Introduction
The video game industry is the leader in interactive music, which is no surprise as the industry is currently valued in the billions, and growing each year (Woodside Capital Partners, 2015). Despite this, interactive music has not yet been fully explored, and even further innovations in this subject area, are more than likely around the corner. This post will take a look at how interactive music, could be implemented to enhance a players experience, without the need of repetitive dialogue, or visual cues to give direction.
What is Interactive Music?
Interactive music, also known as adaptive, is a form of music that can adapt and change through interaction. In video games for example; the early Super Mario Bros (Nintendo Creative Department, 1985), The main protagonist Mario has specific music that plays and represents his character state, such as the invincibility power-up.
How it's Made?
Currently, there are three main ways to implement interactive music. These methods have sub categories, and can also be combined. To add context, this next section will briefly touch on each category of implementation (Designing Music Now 2016).
Transitional Forms
The general idea behind Transitional Forms, is to select the appropriate track, phrase or segment, to play from the playlist / track pool. This can range from simple crossfades, to a more complex system. Tracks can be cut into many small segments or phrases, that can then transition from one to the next, depending on player actions within the game. In the video game World of Warcraft (Blizzard Entertainment, 2004) the music can be heard crossfading when entering a new location. In the game LeChucks Revenge (LucasArts, 2017), a different and more complex approach, to Transitional Forms is taken. The music is composed into small phrases, that can seamlessly transition into a new phrase, once the current phrase had ended. Phrases are dependant on player actions, such as entering new locations, and meeting with characters etc.
Parallel Forms
Vertical remixing breaks a track down into two or more sections / stems, such as woodwinds, strings, and percussion. Stems are then mixed in realtime to synchronise with player actions. This could be introducing percussion when combat is initiated, then fading out when it ends. Examples of Parallel Forms, can be found in the games; Fallout New Vegas (Obsidian Entertainment, 2010), and Red Dead Redemption (Rockstar Games, 2010). Fallout New Vegas has an interesting approach to Vertical Re-mixing, in that the game introduces more layers as the player gets closer to towns and hot spots. This acts as functional music, informing players that they are travelling in the correct direction.
Procedural Music
Procedural Music, is capable of generating music in realtime using electronic synthesis. From 1980, through to the early1990's, this was the only option available for adding music to a video game. Games such as Super Mario Bros (Nintendo Creative Department, 1985) and Sonic The Hedgehog (Sonic Team, 1991) are still fine examples of what can be achieved. This method was rarely implemented past 1995. This is due to a few different factors, one being the limitation of sound quality. For example, the previously mentioned adaptive music methods, can obtain a film blockbuster quality score. Whether as procedural music, must generate its own sound on the fly, which lacks in realism when compared to the previous methods. However procedural music is more versatile in adapting naturally to player choices and location changes. Another factor is that it can suffer from a lack of skilled composers that can implement music in this fashion (Fournel, N.F, 2017).
Enhancing the Experience Through Functional Music
Currently, the majority of games rely heavily upon visual cues, or dialogue to inform or guide players, which can become repetitive. A recent paper presented at the Conference of the Audio Engineering Society (Stevens, R, Raybould, D, & McDermott, D 2015), described a common flaw found in games, and in particular third person stealth games such as Hitman, Metal Gear Solid and Deus Ex. When a player is seen by the enemy, there tends to be delay before the enemy acts upon seeing you, giving the player a chance to correct their actions. This however is already a failure, as the player has already been seen. In the instance of stealth gameplay, music could play a crucial role. For example, when implementing a vertical music system, one could allow specific music stems to increase in volume as the player nears an enemy. It could also add another stem, which fades in when the enemy is facing in the direction of the player. Then fade out as the enemy turns away, notifying the player that it's possibly safe to progress.
This is one example of how music could have a functional purpose in video games. However, music has the potential to carry out other functions too. Below are some examples of how music could serve as an adaptive function.
- Guide players in direction, which way they should be travelling
- Let the player know they are close to another character for example; character leitmotif
- Notify the player of the characters mood
- Notify the player that they are close to an object or clue
- Notify the player of the characters state, example; ability active
- Notify the player of a time limit etc.
Location & Direction
As previously mentioned, Fallout New Vegas directs players to towns and hotspots by the sound of music, as the player gets closer to a town / hotspot, the music adds more layers. Subsequently, as the player leaves the town, the musical layers drop off, leaving the safety net of the town behind. The use of music in this form is subtle, yet it can guide players unconsciously through open terrain, without the need of dialogue, or visual prompts.
In the game Dantes Inferno (Visceral Games, 2010), there is a similar technique applied. There is a section of the game, were the player controlled character is crossing a bridge. As the player edges closer to the other side, there is an increase of musical brass layers added to the mix. This build up of brass finally hits a triggered climax as the player reaches the other side. The music does the reverse, if the player was to backup from the bridge, and won't progress to the final climax.
The video game Thief (Eidos Montréal, 2014), implemented a musical approach for when an enemy detects the player. The mechanic however, does seem disjointed in terms of it's implementation. The music that's triggered works well, but when the enemy stops looking for the player, it suddenly disperses. Clint Bajakian made some excellent points at the 2010 GDC Conference (Bajakian, 2010), in describing that when music fades out, or suddenly stops, it's not rewarding for the player. In the case of thief, if the player managed to escape, or lose the enemy, then a proper end to the music, would be more satisfying for the player.
Convey Moods & Storytelling
Uncharted has excellent dialogue, but the music also serves an integral role in storytelling. For example, Clint Bajakian, a composer / music implementer for the Uncharted Franchise. Explained at the 2013 GDC presentation, the use of short music stingers, to convey the story, or the characters mood. These short stingers can be anywhere from 10 to 30 seconds in length, and also work with dialogue. Below are two examples of this system taken from Uncharted Drakes Deception (Naughty Dog, 2011), as the player edges close to a discovery or realisation, the following stringers maybe heard.
The first example allows the player to know that there is something significant about their current location, this could be a possible escape route, puzzle or item to discover. The second stinger can notify the player of the characters mood, a realisation, or the significance of what was discovered. Clint described this method best for exploration games, were music is not needed at all times. However elements of these stinger moods, could have potential in other genres. For example, if a game has implemented a Vertical Re-mixing system, and the music and stingers are written in the same key, then a variation of stingers that sync with the current progression and beat, could be used to convey the characters mood, or emphasise a key story moment, item collected etc.
The video game LA Noir (Team Bondi, 2011) implements a similar method, by using short music stringers that notify the character of items that they can interact with. These stingers compliment the background music, while also informing the player, without the need for extra dialogue or visual cues.
Conclusion
This post barely touches the surface of what's possible with interactive music, with today's technology, and middleware bridging the gap between music composer and programmer. If you can imagine a music system, then more than likely it can be implemented. For this reason, it's a good idea to discuss what game functions could be taken over by music, or at the very least, supplemented. By using adaptive music in this manner, a player could soon learn the functions of music within the game, as in the case of LA Noire, musical cues helps the player obtain mastery of the game.
Other advantages include less visual clutter, for example a permanent mini map, health bar and enemy detection visuals, could all be replaced by music. Repetitive dialogue sequences could be removed, if the player is travelling the wrong way, rather than playing a dialogue sequence, music could indicate to the player that they are travelling in the wrong direction, this could be implemented like Fallout, or perhaps in a liner game, the music would start to fall out of pitch, the further the player went from the beaten path.
In any case, listening to the words of industry veteran Clint Bajakian, it's important to ask questions on how the music can serve the gameplay. "Do you know where the music is going to be?", "Do you know where the music is coming from?". Then on paper, design how the system could work in order to meet these functions. At this point, as a composer, you know what you're writing for.
References
Bajakian, C. (2010) Adaptive Music The Secret Lies within Music Itself [Presentation] Available at <http://www.gdcvault.com/play/1012601/Adaptive-Music-The-Secret-Lies> [Accessed 19/03/2018].
Bajakian, C. (2013) Audio Bootcamp: Producing Music for AAA Video Games [Presentation] Available at <http://www.gdcvault.com/play/1017670/Audio-Bootcamp-Producing-Music-for> [Accessed 19/03/2018].
Blizzard Entertainment (2004) World of Warcraft. computer program: Activision Blizzard.
Designing Music Now (2016) Top 6 Adaptive Music Techniques in Games – Pros and Cons [online] Available at <https://www.designingmusicnow.com/2016/06/13/advantages-disadvantages-common-interactive-music-techniques-used-video-games/> [Accessed 19/03/2018].
Fournel, N.F (2017) Procedural Audio for Video Games: Are we there yet ? [online] Available at <https://www.gdcvault.com/play/1012645/Procedural-Audio-for-Video-Games> [Accessed 27/02/2018].
LucasArts (2017) Monkey Island 2: LeChuck's Revenge. computer program: LucasArts.
Naughty Dog (2011) Uncharted 3: Drake's Deception. computer program: Sony Computer Entertainment.
Nintendo Creative Department (1985) Super Mario Bros. computer program: Nintendo .
Obsidian Entertainment (2010) Fallout: New Vegas. computer program: Bethesda Softworks.
Rockstar Games (2010) Red Dead Redemption. computer program: Rockstar Games San Diego.
Sonic Team (1991) Sonic the Hedgehog. computer program: Sega.
Stevens, R, Raybould, D, & McDermott, D 2015, Extreme Ninjas Use Windows Not Doors: Addressing Video Game Fidelity Through Ludo-Narrative Music In The Stealth Genre, n.p.: Leeds Beckett Repository, EBSCOhost, viewed 19 March 2018.
Team Bondi (2011) L.A. Noire. computer program: Rockstar Games.
Eidos Montréal (2014) Thief. computer program: Square Enix.
Visceral Games (2010) Dante's Inferno. computer program: Electronic Arts.
Woodside Capital Partners (2015) Video Game Market Report [online] Available at <http://www.woodsidecap.com/wp-content/uploads/2015/12/WCP-Video-Game-Report-20151104.pdf> [Accessed 20/03/2018].