Video might be what draws us to the screen, but half of any viewing experience comes through our ears. Of course, we hear the dialogue, but other more subtle auditory elements — music, ambiance, and sound effects — also help bring what we see on screen to life.
Sound design is an often overlooked aspect of filmmaking. Beginner filmmakers, as well as podcasters and video game makers, tend to neglect it. But paying close attention to a few specific audio elements can noticeably elevate your storytelling.
What is sound design?
Sound design is part of the post-production process where the audio elements of a piece of media, like the dialogue, music, and sound effects, are manipulated to enhance a scene’s potency. Used in film, TV, theater, video games, podcasting, radio, and even software development, sound design includes everything from recording to sampling, editing, scoring, and sound mixing. Sound designers do most of their work in post-production, or the editing phase, where they help the project’s director advance their vision.
Sound design emerged with the advent of audio recording technology for film and TV. The term “sound designer” was first used in 1979, when Francis Ford Coppola credited Walter Murch under that title for his work on “Apocalypse Now.” Versus the usual credit at the time, sound editor or supervising sound editor, Coppola felt the new title encompassed Murch’s role in overseeing the film’s broader sound concept and adding to its overall creative intent. Today, most large film projects have dedicated sound designers.
Why is sound design important?
Filmgoers might see audio as something that mostly serves the images on the screen. But for filmmakers, sound is actually the backbone of a film. Here’s why sound design matters:
- Poor audio is a dealbreaker. Sound is arguably more important than image. Research has shown that videos with poor visuals (i.e., shaky, out of focus, or under-exposed) but clear audio are usually perceived as more watchable than videos with strong images but poor audio. The sound helps carry the narrative; you’ll usually have an easier time following a story with only audio than you would following a story that’s purely visual.
- Sound maintains the illusion. Done right, sound design for film gives an otherwise flat, two-dimensional picture layers of authenticity. Filmmaking is the art of illusion. When making a video, your job is to feed your viewers’ senses, make them believe what you’re telling them, and make them think that what they’re watching is real. Sound, and the manipulation of sound, are crucial to this end.
- We hear emotions. Good sound design makes us feel like we’re with the characters on screen. Sound can accentuate their feelings. Horror movies are an obvious example. Think of how suspenseful music and menacing sound effects shape your experience as you watch a character hide in a closet.
- Audio is spatial. The visuals are confined to the four corners of the screen. But sound design can create a sense of off-screen space. There might be more happening beyond the edges of the film, and audio gives your audience a greater sense of what’s going on in the larger world, making the environment you’ve built more fully realized. Adding echoes to voices in a scene in a canyon, for example, can inform your viewers of its expanse, even when the camera angle doesn’t.
The essential elements of sound design
Sound design is multifaceted. There are several ways you can design or manipulate audio in your video or film to immerse your audience. Here are the five main elements of sound design, which are often used in combination.
- Ambiance. Sometimes referred to as atmosphere, ambiance is the background noise that sets the scene and the foundation of your soundtrack. It helps immerse the audience by telling them where the action takes place. The humming of neon lights can make a dingy kitchen scene feel dingier, just as chirping songbirds suggest a calm spring morning. Ambient sounds should inform and describe without overwhelming and distracting viewers from the focus. An easy way to start is to record a “silent take” (i.e. where no one is talking and nothing is happening) for room tone under the same conditions as the other takes. You can use room tone later to fill in gaps and accentuate certain aspects of the environment.
- Foley sounds. You can only capture so much usable sound during the shooting phase. That’s where Foley comes in. Named after Jack Foley, a sound effect designer who worked with Stanley Kubrick, Foley sounds are reproductions of everyday sounds added in post-production to make a scene feel more natural (or exaggerated, depending on the project). A Foley artist, usually working in a studio, uses mics and other objects to make and record real-world sounds. Say a character marches into a kitchen and slaps a newspaper onto a table. To make the scene feel real, a Foley artist might recreate their footsteps, the swish-swish of their jeans, and the thwack of the paper as it hits the surface of the table.
- Audio effects. Also known as sound effects or SFX, audio effects are manufactured sounds — sounds that don’t occur naturally and can’t be recorded solely with a microphone. Audio effects are often computer-generated, made by manipulating recorded sounds by applying filters and oscillators, layering all kinds of sounds on top of one another, or using other sound design tools. Audio effects can sound otherworldly, as in one famous example from “Star Wars.” The film’s sound designer Ben Burtt created the famous swooshing lightsaber sound by waving a mic in front of a video projector’s motor hum and combining it with TV interference.
- Voiceover. Sometimes called off-camera commentary, voiceover is a post-production recording technique in which an unseen voice, or the voice of a visible subject or character, speaks over the video’s main soundtrack to provide additional context or express unspoken thoughts. It’s a way to convey information about what’s happening on screen without using a caption or a talking head. While voiceover is often read from a script, it can also be improvised or pulled from previous recordings.
- Music. Known as scoring, adding music to a video can set the tone, elicit emotions, and raise tension. An elaborate score can do this even without visuals. A score might include songs and instrumental pieces and doesn’t necessarily need to be intricate to be effective. While sound design and music production are closely related, they’re distinct creative processes. Scoring a video can be an excellent opportunity to work with other artists.
What about the mix?
One of the last and most crucial steps of sound design is the mix, in other words, making it all come together. It isn’t enough to simply include ambiance, Foley, effects, voiceover, and music in your video. They need to work in unison. The various audio elements shouldn’t drown out other parts or overpower your storytelling, but instead complement one another. It’s a balancing act.
Sound mixing ensures that all of the levels in the audio are correct relative to other sounds, and that everything comes in at the right time and with the right intensity. This is especially important if you’re going for something realistic — the mix should match what viewers would expect to hear in real life.
Roles and responsibilities of a sound designer
To be a professional sound designer, you need a wide range of creative and technical skills. Of course, you need a good ear, but a strong understanding of acoustics, psychoacoustics, audio distribution systems, and musical structure also helps. These are skills you can pick up through practice or by studying sound design as a specialization.
Sound designers work under the director’s purview. The scope of a sound designer’s role usually depends on the size of the production. On a smaller production, the role may encompass the project’s entire audio component. In contrast, on larger productions, the sound designer usually leads a team of specialists, including audio engineers, Foley artists, dialogue editors, music editors, re-recording mixers, and even composers.
If you’re making a video by yourself, you have to wear all of these hats, which can be a great way to learn the ins and outs of the craft. Here are a sound designer’s (or sound design team’s) primary responsibilities, most, but not all, of which take place during post-production:
- Planning. During pre-production, study the script or project outline with the director to determine essential elements, and attend production meetings.
- Audio recording and design. Source sounds and create original sounds for the mix, and design each scene’s aural environment.
- Mixing and editing. Synchronize sounds with video, add dialogue (including newly designed audio known as automatic dialogue replacement, or ADR), music, and voiceover to the editing timeline, and remove background noises and imperfections.
