A glimpse into the world of procedural sound design, what is it? Why use it? A quick look at the practise and how to implement procedural sound design by using the Unreal Engine, and a digital audio workstation (DAW).
What is Procedural Sound Design?
Not to be confused by the term or process of procedural audio, both procedural audio and sound design share overlapping principles, but differ in their approach on how to get there. In its essence procedural sound design works with recorded samples or existing audio files, such as ambience and foley recordings. The process maximises the usefulness of these audio files by reducing the length, blending between various takes or intervals of the same recording type. This helps to minimise or remove the repetition of hearing the same sound over and over again. For example in a video game this could be interacting with an object, weapon, or minimising/removing the repetitive nature of character footsteps (Stevens and Raybould, 2015).
Why Procedural Sound Design?
By utilising procedural sound design, a sound designer can take a few audio files and create multiple possibilities and outcomes. One of the many benefits this brings to game design, is that audio assets are no longer repetitive, a design flaw that should not happen in modern games, with the exception of non diegetic ludic sounds. Non diegetic ludic sounds been user prompts, achievements, level ups, new item sounds etc. Other benefits of procedural sound design include timesaving in sample editing. In particular if you set set out to achieve similar results as procedural sound design, a sound designer may have to make multiple recordings of the same sound. For example in procedural sound design three audio recordings could create thirty unique sounds, and possibly more depending on the type of sound. Achieving the same result as procedural sound design, a sound designer would need to record and edit thirty unique recordings. Procedural sound design also leads to the natural benefit of significantly reducing the usage of computer memory (Stevens and Raybould, 2016).
In Practise
Below is a simple walkthrough of the possibilities using procedural sound design, Cubase 9.5 is being used in this demonstration along with the Unreal Engine. In the DAW editing window below there are three snow footstep sounds ready to be edited. The sounds are being prepared for use in a video game, and in this case we will be using The Unreal Engine. Hearing the same three repetitive steps can negatively affect the players immersion within the game. When defining immersion in terms of game design, it's the moments when players enter flow state, players become so immersed they lose track of time. In reality it's not natural to hear the same three steps over and over again, and in video games players can quickly become conscious of this fact which leads to a break in the immersion.
One way to combat this would be to record more footstep audio, import them to your DAW and edit, process and export each individual step. This may sound like a good idea, but it will quickly leach computer memory and cost the sound designer valuable time that could have been spent elsewhere.
A more effective way in resolving this conflict is by applying the practise of procedural sound design. In the example below the waveforms have been cut and separated. This was decided by listening to the audio and identifying the various transients that make up the sound. First the heel, followed by the ball of the foot.
By following this process the three footstep sounds can now be combined to give a total of nine unique sounds. This can be taken a step further by splicing the toe roll transient found at the end of each second waveform.
This principle in regards to the footstep example demonstrated above now allows for twenty seven unique possibilities for each footstep. This will allow for more than enough variety to prevent immersion disconnection caused by the repetitive footstep sounds. Now that the files have been properly edited, it's time to implement them into the Unreal Engine. Below is a snapshot of a custom sound system developed in the Unreal Engine.
The above sound system allows for endless possibilities of what originally began as three footstep sounds. It may look complicated, but it's actually a basic system made up of random choices, delays and subtle modulations to the pitch and volume. An article published by 'A Sound Effect' details a procedural sound design process similar to the one conveyed here, except for explosion sounds. (Stevens, 2016) This shows the adaptability of procedural sound design, from footsteps to explosions, a similar process to deliver endless variations from very different source content.
Procedural sound design can create unique audio systems for delivering ambience to a location, such as a woodland area. A combination of seamless loops and one shots can be used to seamlessly crossfade between various sounds depending on the in game location. For example this layered variation might crossfade between a woodland area of animal life such as insects and birds, and as the player approaches a swamp area, the sounds will crossfade to frogs and water sounds.
Conclusion
It's clear that procedural sound design is a powerful tool in removing the repetitiveness of audio in games. In doing so, it reduces the memory usage, and gives the sound designer more time to be productive and develop more complex audio systems for in game audio.
References
Stevens, R.S., and Raybould, D.R. (2016) Game Audio Implementation A Practical Guide Using The Unreal Engine. Paperback edn. New York and London: Focal Press.
Stevens , R.C., and Raybould, D.R. (2015) 'The Reality Paradox: Authenticity, fidelity and the real in Battlefield 4'. The Sound track 8 (1-2)
Dara Crawford. (2017) Personal Screenshot Gallery [online] 13/02/2017. [Accessed 13/02/2018].
Stevens, R.S. (2016) A Sound Effect, Why Procedural Game Sound Design is so useful – demonstrated in the Unreal Engine. [Blog] 18/01/2016. Available at https://www.asoundeffect.com/procedural-game-sound-design/ [Accessed 12/02/2018].
Aaron McLeran and Dan Reynolds, A.M, D.R. (2017) Aaron McLeran and Dan Reynolds Procedural Audio in the new Unreal Audio Engine [Presentation] Available at <http://proceduralaudionow.com/aaron-mcleran-and-dan-reynolds-procedural-audio-in-the-new-unreal-audio-engine/> [Accessed 12/20/2017].
Nair, A.F. (2012) Procedural Audio: Interview with Andy Farnell. Interviewed by Varun Nair [Article] January 18, 2012.
Huiberts, S.H. (2010) THE ROLE OF AUDIO FOR IMMERSION IN COMPUTER GAMES . PhD Dissertation. Portsmouth: University of Portsmouth.
Farnell, A.F. (2008) Designing Sound. Printed edn. England: Applied Scientific Press.
Farnell, A.F. (2007) 'An introduction to procedural audio and its application in computer games.'. , pp..
Leonard J. Paul. (2010) Procedural Sound Design [online] Canada: Video Game Audio. Available at <VideoGameAudio.com> [Accessed 13/02/2017]