Sound Design for Immersive User Experiences
- Published on: 26 June 2023
Share
In the hierarchy of senses, hearing is second only to sight as our strongest sense. Some sounds create instinctive and psychological reactions when heard, others build our understanding of environments to lead powerful experiences. All auditory elements, however, immerse individuals in the bigger picture, meaning they have incredible potential to uplift the user experience of all kinds of digital and physical media.
Sound design is at the heart of it all. Describing the creation of a sonic palette, sound design involves recording, producing, editing and mixing aural stimuli for an end product. Whether it’s for a film, video game, album, website or art installation, sound designers are able to stir action, emotion and understanding.
While music may come to mind first – it’s only a small piece in the sonic puzzle. Sound effects, foley and dialogue also contribute to the immersion, with their effects only as powerful as the way they’re implemented, mixed and synced. Though when it all comes together, users can dive into rich, interactive and navigable experiences that are both impactful and accessible.
The power of sound
Sound possesses a remarkable ability to shape our emotions, thoughts and behaviours. From the gentle ambient sounds of running water, to dissonant crescendos and notification dings, auditory elements can evoke profound psychological responses.
Emotionally, sounds can uplift spirits or evoke feelings of fear. Abrupt or repetitive sound patterns can capture attention, divert focus and facilitate relaxation. Some sounds bring back memories, while others can influence our time perception, movements and decision-making. This is because all aural stimuli can activate different parts of the brain, notably the planum temporale – which sparks action even if an individual isn’t anticipating it – and the amygdala – responsible for a range of emotional responses.
Sound designers play into its power to direct user experiences. By implementing the right music, sounds and ambiance into a media, they can set the scene in a way that visuals alone cannot, and can even replace or reform traditional visual experiences.
For example, the popular fighting video game, Mortal Kombat, has a dedicated community of blind players that rely solely on audio cues to play – some even competitively. The game has such a rich, nuanced and reactive sonic palette that players are able to sense and navigate its environment through its sonic palette.
Key sound design elements and techniques
Sound designers implement a variety of aural elements in their work to communicate experiences effectively to users and audiences.
- Music
Music can set the mood, create suspense, elicit emotions and bring narratives to life. Whether it’s composed specifically for a project or sourced from existing compositions, music commonly adds weight to visuals and animates an immersive environment.
The film score of Twin Peaks: Fire Walk With Me by Angelo Badalementi is often lauded as one of the finest movie accompaniments of the 20th century. The cold, suspenseful and surreal feel of its idiosyncratic dark jazz pairs mysteriously, yet perfectly, with the film’s neo-noir and psychological thriller themes.
- Sound effects and foley sounds
Sound effects and foley sounds encompass sounds separate from the music or dialogue and are essential for creating immersive, reactive experiences. They usually fall into four “flavours” of sound – realistic, symbolic, metaphoric and verbal. Whether it’s the sound of footsteps, the beep of a car horn or the aural feedback of pressing a button in-app, they can enhance realism and provide feedback for the user. Precise sound synchronisation with visual elements is crucial for seamless user experiences.
The UI sounds of the Apple iPhone have become synonymous with modern, Western technology. Each sound, from the camera shutter to the lock sound, are exemplars of representative aural elements. They acknowledge user interactions, notify changes and provide sonic confirmation across all flavours of sound.
- Ambience
Ambience describes the sounds or compositions that establish a media’s overall atmosphere. It can include environmental sounds like wind, rushing water and echoed crowd noise, and also describes musical pieces composed to add depth to the sonic environment.
Bioshock is a first-person shooter game celebrated for its deeply immersive and cinematic sound design. Set in a dystopian underwater city, aqueous sounds, rumbling machinery, clanging metal and crackled loudspeaker voices all come together to create a dilapidated ambience in-line with its horrorful setting.
- Dialogue
Dialogue inherently carries information, advances stories and develops characters in many kinds of digital media. As coherent dialogue is essential for navigable and positive user experiences, sound designers implement careful recording, editing and mixing techniques make every word clear
The New Yorker features audio variations of many of their pieces online. The audio sits alongside the article, with many featuring the voice of the authors themselves. This opens up another realm of the written piece, offering a new layer of emotion as well as another means of engagement for people less in-touch with the written word.
- Spatial audio
Spatial audio techniques create a realistic sense of direction, distance and movement with sound. It taps into the way our ears and brain are able to understand where a sound is coming from and helps enhance immersion and authenticity.
The THX Deep Note is an instantly recognisable jingle that uses deep resonance to engulf the audience – as if the sound is coming from all angles. Composed by Lucasfilm sound engineer Dr. James Moorer, it begins with 30 differently-voiced notes moving randomly between two frequencies before they all morph towards a specific target note in a wild crescendo. The result is a loud, compelling and disorientating chord.
- Mixing:
While each aural element may be magnificent on its own, it’s only through careful mixing and balance that they can work together effectively. This involves volume adjustments, panning and balancing the frequencies of each individual sound element to create a coherent sonic experience.
Harmony in Ultraviolet by ambient composer, Tim Hecker, is a lesson in atmospheric sound design. Purely instrumental, it’s an ominous slab of drones, noises and glitches that weave in and around each other to paint an ethereal canvas of chaos. There’s a lot going on, but there’s beauty and balance at all points of the hypnotic journey.
- Accessibility
As elucidated with the aforementioned Mortal Kombat example, sound designers can uplift user experiences for individuals with a range of different abilities. This involves making visual elements “viewable” through sound without complication or distraction.
Action Audio – 2021 Good Design Best in Class Winner – is an accessibility system for visual sports broadcasts that allows blind and low-vision fans to follow along in real-time. It uses instantaneous ball monitoring technology and spatial sound design to give audiences insight into what’s happening, when it’s happening – without the need for visual information.
Activate your senses with design
The multifaceted power of design can capture, stimulate, solve, challenge and reinvent. From our ears to our hands, to the way we see the world around us, design empowers almost every interaction.
As the 2023 Jurors come together to evaluate, crown and celebrate the brightest designs of this year’s hallmark Award season, why not turn back the clock and discover some innovations of the past? Search by category or have a blind deep dive – find inspiration either way.
DIVE INTO THE GOOD DESIGN INDEX HERE
More News
