Fundamental and Advanced Techniques of Sound Design in Music, Film, and Games

Introduction

Sound design is an art and science that involves creating, manipulating, and integrating auditory elements to enhance the impact and meaning of various media, spanning music production, film, video games, and interactive installations (Cipriani & Giri, 2011). It goes beyond merely recording sounds; it entails sculpting sonic landscapes that evoke emotions, convey narratives, and immerse audiences in captivating experiences (Pinch & Bijsterveld, 2004). Sound design is a critical element that contributes significantly to the overall atmosphere and storytelling in various media (Davies et al., 2012). In essence, sound design is the process of crafting the auditory elements of a project, which includes creating original sounds, manipulating existing recordings, and integrating these elements into a cohesive and compelling soundscape (Salselas & Penha, 2019). The importance of sound design stems from its ability to profoundly influence the audience's perception and emotional response (Stöcker, 1994). In music, it can define genres, create unique sonic textures, and elevate the emotional impact of compositions. In film, sound design is pivotal for creating believable environments, emphasizing dramatic moments, and guiding the audience's emotional journey. In video games, it provides crucial feedback to player actions, enhances immersion, and contributes significantly to the overall gameplay experience. The field of sound studies is a multifaceted discipline that has evolved from various traditions of thought, including philosophy, theology, music, acoustics, psychology, and more (“Keywords in Sound,” 2015).

 Sound design is indispensable for creating immersive and engaging experiences across various media (Kang, 2011). It shapes emotional responses, enhances storytelling, and provides crucial feedback in interactive environments (Kang et al., 2016). Sound design enhances the narrative and emotional impact of a project, whether it is a musical composition, a cinematic masterpiece, or an interactive game (Davies et al., 2012). Soundscapes, which refer to the sonic environment, play a significant role in improving the quality of the environment and enhancing the function of a place (Foale, 2014). The concept of soundscape emphasizes how sounds are perceived and understood by individuals or societies, highlighting the interdisciplinary nature of the field, which includes acoustics, psychology, sociology, and urban planning (Aletta et al., 2017) (Aletta et al., 2016). Soundscapes consider the totality of sounds within a location, emphasizing the relationship between an individual's perception, understanding, and interaction with the sonic environment (Grinfeder et al., 2022).

Key Sound Design Categories

Sound design encompasses a wide range of categories, each with unique characteristics and applications. Diegetic sounds are those that originate from within the story world, such as character speech, the sound of footsteps, or the rumble of an engine; these sounds contribute to the realism and believability of the narrative (Smith & Pijanowski, 2014). Non-diegetic sounds, conversely, are external to the story world, such as the musical score or voice-over narration; they are used to enhance the emotional impact or provide additional information to the audience. Another fundamental distinction lies between synthesized sounds, which are created electronically using synthesizers and software, and recorded sounds, which are captured from the real world using microphones. Synthesized sounds offer unparalleled control and flexibility, allowing designers to create sounds that are entirely unique or impossible to capture naturally. Recorded sounds, on the other hand, provide authenticity and realism, grounding the soundscape in the familiar sonic textures of the real world. Furthermore, sound design involves categorizing sounds based on their function and purpose within a project. Ambient sounds create the background atmosphere, while specific sound effects highlight particular actions or events. Foley sounds are created in post-production to enhance the realism of everyday sounds, such as footsteps or the rustling of clothing.

Essential Sound Design Techniques

Subtractive Synthesis

Subtractive synthesis is a foundational technique in sound design that involves starting with a harmonically rich sound source and then selectively removing frequencies to shape the desired timbre. Typically, a waveform rich in harmonics, such as a sawtooth or square wave, is generated by an oscillator. Filters are then used to attenuate specific frequency ranges, allowing the sound designer to sculpt the sound by removing unwanted harmonics or emphasizing particular frequencies. The most common type of filter used in subtractive synthesis is the low-pass filter, which attenuates frequencies above a certain cutoff point, resulting in a warmer, smoother sound. However, high-pass, band-pass, and notch filters are also employed for more complex timbral shaping. Envelope generators, particularly ADSR envelopes, are crucial for controlling the amplitude and filter cutoff of the sound over time, adding dynamic movement and expressiveness. Subtractive synthesis is widely used for creating a broad spectrum of sounds, from warm, evolving pads to aggressive, distorted leads.

Additive Synthesis

Additive synthesis, conversely, builds complex sounds by combining simple waveforms, typically sine waves, at varying amplitudes and frequencies. This method allows for the creation of highly complex and unique timbres by carefully controlling the individual components of the sound. By meticulously adjusting the amplitude and frequency of each sine wave, sound designers can construct sounds from the ground up, offering unparalleled control over the harmonic content. Additive synthesis is often used to emulate acoustic instruments, create shimmering pads, or generate unusual and otherworldly textures. The process of additive synthesis often requires sophisticated software tools and a deep understanding of spectral analysis, as the designer must determine the precise frequencies and amplitudes needed to achieve the desired sound.

FM Synthesis

FM synthesis is a powerful technique that involves modulating the frequency of one waveform (the carrier) with another waveform (the modulator). This modulation creates complex sidebands and harmonics, resulting in sounds that are often metallic, bright, and harmonically rich (Potamianos & Maragos, 1999). FM synthesis allows for the creation of dynamic and evolving sounds that are difficult to achieve with other methods. The Yamaha DX7, a synthesizer that gained prominence in the 1980s, prominently featured FM synthesis and played a pivotal role in shaping the sound of pop music during that era.

Granular Synthesis

Granular synthesis involves breaking down audio into tiny fragments, or "grains," typically ranging from 1 to 100 milliseconds in duration (Peris et al., 2025). These grains are then manipulated and reassembled to create complex textures and soundscapes (Roads, 2002). The parameters of each grain, such as pitch, amplitude, duration, and position, can be individually controlled, allowing for the creation of a vast range of sonic effects. Granular synthesis is particularly well-suited for creating evolving soundscapes, glitchy textures, and otherworldly effects.

Sampling and Resampling

Sampling involves recording audio from external sources and then manipulating it within a sampler. Resampling, a related technique, involves taking an existing sample and altering its pitch, time, or other characteristics to create new sounds (Yarlagadda, 1976). Samplers allow designers to manipulate these recordings, changing their pitch, duration, and other parameters. Sampling and resampling are versatile techniques used to create a wide range of sounds, from realistic instrument emulations to abstract soundscapes.

Layering and Texturing

Envelope Shaping

Layering and texturing involve combining multiple sounds to create a richer, more complex sonic landscape. Envelope shaping is critical for controlling the dynamic characteristics of a sound (Pulkki & Karjalainen, 2015). Envelope generators, particularly ADSR envelopes, are used to shape the amplitude of a sound over time, creating dynamic movement and expressiveness.

Modulation

Modulation involves using one signal to control another, adding movement and depth to sounds. LFOs (low-frequency oscillators) are commonly used to modulate parameters such as pitch, filter cutoff, and amplitude, creating vibrato, tremolo, and other dynamic effects. Filters, another essential modulation tool, are used to shape the frequency content of a sound.

Reversing and Time-Stretching

Reversing and time-stretching are powerful techniques for creating unusual and otherworldly sounds. Reversing a sound can create a sense of anticipation or unease, while time-stretching can be used to create slow, ethereal textures or fast, glitchy effects. Pipe organs exemplify early acousmatic settings, utilizing complex timbral synthesis (d’Alessandro & Noisternig, 2019). Augmented organs use digital signal processing to capture, transform, and play back sounds in real time (Blackburn, 2010) (d’Alessandro & Noisternig, 2019). Real-time music systems can mitigate the adverse effects of algorithmic procedures in composition to some degree (DuBois, 2003).

Result

Sound design plays a pivotal role in numerous creative fields, impacting how audiences perceive and engage with media (McKee, 2006). In music, it elevates compositions by introducing unique sonic textures and atmospheric elements, enriching the overall auditory experience (Colton & Forrest, 2025). The use of auditory elements can effectively complement other modalities not only in the traditional desktop computer environment but also in virtual and augmented reality, mobile platforms, and other kinds of novel computing environments (Csapó & Wersényi, 2013). In film, sound design is crucial for creating immersive environments, underscoring emotional cues, and heightening tension, thereby enhancing the narrative's impact. It brings together interests from the areas of data mining, exploratory data. Despite great strides in developing the graphical dimension of user interfaces, the auditory dimension has been neglected (Serafin et al., 2011). Sound has the potential to be a subliminal element that focuses attention on the most relevant narrative elements, thereby enhancing storytelling and biasing search in immersive audiovisual environments (Salselas & Penha, 2019). The importance of research and development in the field of audio-visual integration and the design of new interfaces with sound has been recognized (Jeon & Jo, 2019). Moreover, in video games, sound design is essential for providing feedback to player actions, establishing the game's atmosphere, and creating a sense of presence, which significantly contributes to the overall gaming experience.

In today's multimedia-rich environment, creating compelling soundscapes necessitates a combination of hardware, software, and innovative techniques (Koryakin, 2020). The exploration of soundscapes has led to increased interest in the study of sound as a phenomenon, as well as the institutionalization of sound studies as an academic discipline (Gautier, 2016).  Understanding the nuances of sound perception within specific contexts is crucial for effective sound design (Xiu-min, 2009).

The auditory domain, despite its importance to the holistic experience of a place, is often overshadowed in environmental psychology studies (Aletta et al., 2021). The sounds of nature are important factors in determining the quality of recreational experiences (Aletta et al., 2021) (Benfield et al., 2018).

Soundscape ecology has shifted the focus from noise abatement to preserving natural sounds, highlighting the need to understand human perception of soundscapes in various settings. (Aletta et al., 2025). The concept of a soundscape has significantly influenced research on acoustic environments and their design, although it has faced scrutiny (Axelsson, 2020). The soundscape approach has been integrated into noise policies in several countries, yet a gap remains between research and practical design implementation (Hong & Chong, 2023). The absence of a straightforward method for soundscape design contributes to this issue.

References

Aletta, F., Axelsson, Ö., & Kang, J. (2017). Dimensions Underlying the Perceived Similarity of Acoustic Environments. Frontiers in Psychology, 8. https://doi.org/10.3389/fpsyg.2017.01162

Aletta, F., Coensel, B. D., & Lindborg, P. (2021). Editorial: Human Perception of Environmental Sounds. In Frontiers in Psychology (Vol. 12). Frontiers Media. https://doi.org/10.3389/fpsyg.2021.714591

Aletta, F., Kang, J., & Axelsson, Ö. (2016). Soundscape descriptors and a conceptual framework for developing predictive soundscape models. Landscape and Urban Planning, 149, 65. https://doi.org/10.1016/j.landurbplan.2016.02.001

Aletta, F., Zhou, K., Mitchell, A. D., Oberman, T., Pluchinotta, I., Torresin, S., Cerwén, G., Lam, B., Can, A., Guastavino, C., Tarlao, C., Lavandier, C., Schulte‐Fortkamp, B., Cobussen, M., Burgess, M., Nooshin, L., Payne, S. R., Ratcliffe, E., Bernatek, R., … Kang, J. (2025). Exploring the relationships between soundscape quality and public health using a systems thinking approach. 1(1). https://doi.org/10.1038/s44384-025-00003-y

Axelsson, Ö. (2020). Soundscape revisited. Journal of Urban Design, 25(5), 551. https://doi.org/10.1080/13574809.2020.1810006

Benfield, J. A., Taff, B. D., Weinzimmer, D., & Newman, P. (2018). Motorized Recreation Sounds Influence Nature Scene Evaluations: The Role of Attitude Moderators. Frontiers in Psychology, 9. https://doi.org/10.3389/fpsyg.2018.00495

Blackburn, A. (2010). Computer use in music for the pipe organ and real time dsp - or the music of Janus. http://www98.griffith.edu.au/dspace/bitstream/handle/10072/35067/65510_1.pdf;sequence=1

Cipriani, A., & Giri, M. (2011). Innovation, Interaction, experience and imagination in computer music education. International Computer Music Conference, 2011. https://dblp.uni-trier.de/db/conf/icmc/icmc2011.html#CiprianiG11

Colton, J., & Forrest, S. (2025). ‘You Really Have to Get in There and Actually Figure It Out’: Engaging Pre-Service Teachers in Children’s Literature Through Transmodality. Education Sciences, 15(4), 496. https://doi.org/10.3390/educsci15040496

Csapó, Á., & Wersényi, G. (2013). Overview of auditory representations in human-machine interfaces [Review of Overview of auditory representations in human-machine interfaces]. ACM Computing Surveys, 46(2), 1. Association for Computing Machinery. https://doi.org/10.1145/2543581.2543586

d’Alessandro, C., & Noisternig, M. (2019). Of Pipes and Patches: Listening to augmented pipe organs. Organised Sound, 24(1), 41. https://doi.org/10.1017/s1355771819000050

Davies, W. J., Adams, M., Bruce, N., Cain, R., Carlyle, A., Cusack, P., Hall, D. A., Hume, K., Irwin, A., Jennings, P. A., Marselle, M. R., Plack, C. J., & Poxon, J. (2012). Perception of soundscapes: An interdisciplinary approach. Applied Acoustics, 74(2), 224. https://doi.org/10.1016/j.apacoust.2012.05.010

DuBois, R. L. (2003). Applications of Generative String-Substitution Systems in Computer Music. https://music.columbia.edu/~luke/dissertation/dissertation.pdf

Foale, K. (2014). A listener-centered approach to soundscape analysis. https://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.619439

Gautier, A. M. O. (2016). Acoustic Multinaturalism, the Value of Nature, and the Nature of Music in Ecomusicology. Boundary 2, 43(1), 107. https://doi.org/10.1215/01903659-3340661

Grinfeder, E., Lorenzi, C., Haupert, S., & Sueur, J. (2022). What Do We Mean by “Soundscape”? A Functional Description. Frontiers in Ecology and Evolution, 10. https://doi.org/10.3389/fevo.2022.894232

Hong, J. Y., & Chong, K. H. (2023). Designing Public Soundscapes through Social Architecture and Soundscape Approaches: Reflective Review of Architectural Design Studio. Sustainability, 15(16), 12399. https://doi.org/10.3390/su151612399

Jeon, J. Y., & Jo, H. I. (2019). Effects of audio-visual interactions on soundscape and landscape perception and their influence on satisfaction with the urban environment. Building and Environment, 169, 106544. https://doi.org/10.1016/j.buildenv.2019.106544

Kang, J. (2011). Noise Management: Soundscape Approach. In Elsevier eBooks (p. 174). Elsevier BV. https://doi.org/10.1016/b978-0-444-52272-6.00260-9

Kang, J., Aletta, F., Gjestland, T., Brown, L., Botteldooren, D., Schulte‐Fortkamp, B., Lercher, P., Kamp, I. van, Genuit, K., Fiebig, A., Coelho, J. L. B., Maffei, L., & Lavia, L. (2016). Ten questions on the soundscapes of the built environment. Building and Environment, 108, 284. https://doi.org/10.1016/j.buildenv.2016.08.011

Keywords in Sound. (2015). In Duke University Press eBooks. Duke University Press. https://doi.org/10.1215/9780822375494

Koryakin, O. (2020). ТHE METHODOLOGICAL SUPPORT OF SOUND ENGINEERING TRAINING OF FUTURE MASTERS OF MUSICAL ART. OPEN EDUCATIONAL E-ENVIRONMENT OF MODERN UNIVERSITY, 8, 26. https://doi.org/10.28925/2414-0325.2020.8.4

McKee, H. A. (2006). Sound matters: Notes toward the analysis and design of sound in multimodal webtexts. Computers & Composition/Computers and Composition, 23(3), 335. https://doi.org/10.1016/j.compcom.2006.06.003

Peris, E., Ribes, A. M. i, & Tejada, J. (2025). Design and Implementation of a Musical System for the Development of Creative Activities Through Electroacoustics in Educational Contexts. Signals, 6(2), 16. https://doi.org/10.3390/signals6020016

Pinch, T., & Bijsterveld, K. (2004). Sound Studies. Social Studies of Science, 34(5), 635. https://doi.org/10.1177/0306312704047615

Potamianos, A., & Maragos, P. (1999). Speech analysis and synthesis using an AM–FM modulation model. Speech Communication, 28(3), 195. https://doi.org/10.1016/s0167-6393(99)00012-6

Pulkki, V., & Karjalainen, M. (2015). Sound Quality (p. 349). https://doi.org/10.1002/9781119825449.ch17

Roads, C. (2002). Microsound. https://doi.org/10.7551/mitpress/4601.001.0001

Salselas, I., & Penha, R. (2019). The role of sound in inducing storytelling in immersive environments. https://doi.org/10.1145/3356590.3356619

Serafin, S., Franinović, K., Hermann, T., Lemaître, G., Rinott, M., & Rocchesso, D. (2011). The Sonification Handbook. https://vbn.aau.dk/da/publications/sonic-interaction-design(71c27181-b274-437f-b3b5-41faf042fef1).html

Smith, J. W., & Pijanowski, B. C. (2014). Human and policy dimensions of soundscape ecology. Global Environmental Change, 28, 63. https://doi.org/10.1016/j.gloenvcha.2014.05.007

Stöcker, M. (1994). Exhibit sound design for public presentation spaces. Museum Management and Curatorship, 13(2), 177. https://doi.org/10.1080/09647779409515397

Xiu-min, Z. (2009). A Quantification Analysis on Acoustic Landscapes of Waterfront Scenic Areas: A Case Study of Hangzhou City, China. Journal of Asian Architecture and Building Engineering, 8(2), 379. https://doi.org/10.3130/jaabe.8.379

Yarlagadda, R. (1976). Digital signal processing. IEEE Transactions on Acoustics Speech and Signal Processing, 24(6), 586. https://doi.org/10.1109/tassp.1976.1162867