This post is a top-down look at the updated music system for Black Box VR, I won’t be going into all the nitty-gritty technical details of the scripting and integration of how it works, this will be a top-down look covering the core features and capabilities of the system. First of all, let’s get some obvious questions out of the way (at least what I imagine to be obvious, which may be 100% the opposite).
So what is the difference, why upgrade the current music system?
Previously, our Black Box Music system used linear music, the system itself was somewhat adaptive in that when things got intense during a workout, we would cut the current track, and then follow up with a high-intensity track. By no means ideal, as this abrupt transition to a new track can easily throw a player off their rhythm, which is especially true when players sync their locomotion to the beat of the music, a process better known as Rhythmic Entrainment.
The goal with the new system was to keep players in that Rhythmic Entrainment state for as long as possible without breaking their rhythm, and also without making the music repetitive. Thankfully with modern game audio software, there are lots of possibilities and solutions, as I said without getting too technical, we are currently using FMOD to manage and handle the audio tasks (for any game audio enthusiasts out there, I know you totally get it!).
Our previous music system is still alive and breathing, and will switch on when the adaptive tracks have all played, this is to prevent any of our new music from becoming repetitive. Although, in the future, we aim to shorten this time and eventually phase out the old system altogether.
What’s this Linear Music and Non-Linear Music you Speak of?
When listening to your favourite music, it sounds the same every time, 100% predictable, which is perfect for linear media e.g. film and tv, radio, music streaming platforms etc. But for games, this is far from perfect, and to answer why to that question, well games are non-linear, the player is in control, they are telling their story at their own pace, they make choices and decisions that completely affect and alter the gameplay, and in Black Box, no two workouts are the same, different strategies, different units, different exercises, different power-ups… In short, to tackle this nonlinearity found in video games, audio engineers will implement a form of adaptive music that will respond to the player. In short, Linear music is what it is, it can’t be changed or altered to the extent of non-linear music.
With that out of the way, let's talk about the Black Box Adaptive Music System. There are different forms of Adaptive Music, and at Black Box, I decided to implement a hybrid form that combines Vertical Remixing and Horizontal Resequencing (because that makes perfect sense). Okay, so short explanation, Vertical Remixing is usually adjusting the different levels of musical groups, e.g. a typical drum group would consist of kick, snare, toms and cymbals. There can be many groups e.g. drums, bass, pads, lead, synth, vocals etc. These groups are the core building blocks of a song, and in a video game, these group elements can be controlled during gameplay, which significantly alters how the music sounds. Horizontal Resequencing transitions the music to a different section or even another track. When used within the same track, the original structure is altered to better suit player actions. When done well this technique will sound as smooth as possible within a musical context, changing on time synced to the beat.
Vertical Remixing Context
In short, when you perform direct damage attacks you begin to increase audio parameters that hold their values for a given time, and will eventually decrease their lowest value when no direct damage attacks are being done. By completing slice patterns, players can also delay the time it takes for the audio parameters to reduce. Let's take a look at some examples using FMOD to mimic gameplay triggers.
Example 1: Opening Filters, Adding & Switching Layers
In this example we are using the track “Time To Groove” you will hear the music change from being dark and dull, to sounding open and bright, this all depends on player performance. As the layer’s value is increased you will hear the vocal part switching, one is relaxed sounding, while the other is more driven.
Video 1 - Filters and Layers - Vertical Remixing
Horizontal Resequencing Context
In this example we will use the same track as before, only this time I will simulate the track start when the user is performing well, we want to skip the tracks build up and get right into the meat of the song. Now, this section happens to be at the end of the song, so we want to then transition to the middle section so we can at least hear the rest of the track. During this section, I’ll simulate the round end so that the track will end musically rather than come to an abrupt stop with no conclusion (Woah, that’s a lot of transitions). When a player is in the moment, these transitions flow with their actions and should feel less abrupt than a completely new track starting, and then waiting for this new track to get to the drop.
Video 2 - Horizontal Resequencing
Epic Track Layers “Time to Groove”
Now that you are familiar with the track “Time To Groove”, I’m going to use it for one more example, this track is one of the lower intensity tracks, but it does have a little trick up its sleeve to squeeze out a little extra intensity, which might give the player an extra boost of motivation while Hysterical is triggered! (Deals extra damage but the weight is heavy!)
In Black Box you're surrounded by a crowd who is unaware of the music that the player is hearing, this is commonly referred to as non-diegetic music and is a very common term in games, film and tv. In Black Box on selected tracks, there are existing moments in time when the crowd is aware of the music (known as diegetic) and will chant along in sync to the beat. Now it can be hard to make out exactly what they are saying, but this extra-musical energy that’s coming from the crowd surrounding you, this little spur of excitement, might just be the thing that helps a player squeeze out an extra rep.
Video 3 - Crowd Awareness
Notification Through Music
In Black Box VR, the enemy can be tough, especially when they are destroying your gate and damaging your crystal, to showcase the sense of urgency during these key moments, the music system will replace the kick drum of the song with a synced heartbeat to the tempo of the track. This triggers short bursts when a player's gate is destroyed, or when the player's crystal is taking damage. This keeps the music flowing, and serves as a notification that the player’s defence and round could be at stake. I’ll simulate this using the track “Player One is Ready” (Can you guess what inspired the track name?).
Video 4 - Heartbeat Triggers
If that wasn’t enough already, when players complete slam patterns, the music system will trigger a one-shot/ornament, a short musical decoration like a drum fill or a riser. These are also synced to the music, essentially letting the player add excitement to the track!
Conclusion
By playing Black Box VR, players are altering and creating new and unique interpretations of the music, contributing to the compositions through one-shots, altering the structure and arrangement through their performance and actions, in a sense the player has become part of the composition, the player has literally become the arranger and conductor to their own score!
Thanks, I hope you enjoy the new music system!