狠狠撸

ADAPTIVE
MUSIC
IN GAMES
ADAM J SPORKA
IIM/DCGI@CVUT
WARHORSE STUDIOS
UX MONDAY 03/02/15

@adam_sporka Adaptive Music in Games UX Monday 3
Purpose of Sound in UIs
? Interaction modality (input and output)
? Alerts
? Widget sounds
? Presentation of data
? Mood setters
? Sound branding

Describing Sound and Music
Signal
Sound
field
Tone
/ Stroke
Sequence
Concord
Composition
Environment
Reverberation
Vibration
Distribution
Spectrum
Volume
Periodicity
Pitch
Length
Volume Accord
Interval
Harmony
Rhythm
Tempo
Phrase
Loop
Soundtrack
Intention
Abstraction level
Playlist
Genre
? science art ?

Fuzzy Terminology
? What it does:
– Adaptive music
– Responsive music
– Reactive music
– Interactive music
? How it is done:
– Procedural music
– Algorithmic music

Science: Data Sonification
? Synthesis of sound
controlled by
input data
? ICAD 2004 contest
– http://www.icad.org/websiteV
2.0/Conferences/ICAD2004/c
oncert.htm

https://www.rowperfect.co.uk/a-guide-to-good-rowing-technique/

Sonification of Rowing sounds
? https://www.youtube.com/watch?v=YxWNw9fIizk
? Audible comparison of trainee’s sound and
reference
? Audible feedback while people is rowing

Outline
? Purpose of music in games
? Static music soundtracks
– Music design
? Dynamic music soundtracks
– Design, production, …
? Music production technology
– How the actual music is made

Purpose of Music
? Tradition
– Present in most of the released titles
– Why not in yours?
? Affirmation of the genre
– “Yes, this is an 8bit retro.”
– “Indeed, cowboys in spaceships.”
– “Yay, manga.”

Purpose of Music
? Presentation of emotion
– Easygoing, happy-go-lucky, …
– Depressing, sad, …
? Setting the expectations
– Level difficulty
? Status monitoring
– Changes of music over time indicate changes in
game

Purpose of Music
? Essence of the gameplay
– Dance Dance Revolution (1998)
– Guitar Hero (2005)
– Pugs Luv Beats (2011)
? http://www.youtube.com/watch?v=V0i18_--8Yc
– Not covered by this talk

Game Music Components
? Theme Music
– Often synchronized with visual events on the
screen
– Cinematic experience
? Underscore
– In-game music
– User interface background

Game Music Components
? Stingers
– Specific music triggered by an event
– Transition music
– Level Notification, “Game Over” Screen, “Game Won”
Screen
– Fallout New Vegas:
Pulling out / holstering Magnum
? http://youtu.be/sWLpSpZR6J4?t=41s

Soundtrack
? Diegetic sounds
– Sounds in story space
– On-screen or off-screen
– Voices of characters, sounds of objects, music coming
from objects
? Non-diegetic sounds
– Not implied to be present in the action
– Narrator, sounds for dramatic effect, underscore

Sound in Games
– Voices, diegetic sound effects
– Diegetic ambience
– Non-diegetic ambience
? “Sound design pads”
– Underscore
ABSTRACT
CONCRETE

Game Music Characteristics
? Theme music is feature
– Recognizable theme
– Elaborate
– Genre announcer
– Everyone will hear this
? Underscore is background
– Mood setter
– Does not distract
– Can be listened to throughout the gameplay

Game Music Characteristics
? Silence
– “Dosage” of music
? Used to emphasize music
– Keep the ambient sounds present
– Complete silence ? “something wrong”
? “Are my speakers on? Will I hear anything at all when
I’ll really have to?”

Game Music Interactivity
? Static soundtrack
– Predefined and unchanging
– Loops
– Assigned to specific screens / levels
? Dynamic (adaptive) soundtrack
– More complex control of the music playback
– Engine “aware” of the state of the game

Music Design
? Genre?
? Situations in game?
? Mechanics of music?

Music Design
? Multiple contexts –
typical for open-world RPGs
– Landscape exploration
– Combat
– Dialogs
– …
? Considerations of importance of music
scenes
? Is “combat” always more important than “location”?

Music in FoE by The Overmare Studios

Static Soundtrack
? Early and simpler games
? Level-based games
? Pengon (1984, Atari 800XL)
– http://www.youtube.com/watch?v=MDhLxRLvwvY
– One music loop
– Game over stinger

Static Soundtrack
? Supaplex (1991, Digital Integration, MS-DOS)
– Played throughout the game
– http://www.youtube.com/watch?v=yknubWX2KYI
? Goonies (1985, Datasoft, Atari 800XL)
– Two loops, one used for title, then alternating with each
level
– https://www.youtube.com/watch?v=FKh5b8jcwLk
? Jazz Jackrabbit (1994)
– Separate tracks for levels
– http://youtu.be/b16upFloYak?t=29s

Static Soundtrack

Case Study: Nimble Quest
? 2013; NimbleBit; iOS, Android
? Arcade / RPG
? Early 2000s pixel art graphics

? Music by Whitaker Blackall
? Contents:
– Title / menu
– Loop per level
? Genre:
– Contemporary chiptune
– http://wtrebella.bandcamp.com/album/nimble-quest

? Typical length of a loop: 2’
? Unobtrusive
? Reflects the increasing difficulty of the levels
– L1 http://www.youtube.com/watch?v=6mh7qJgnpJI
– L2 https://www.youtube.com/watch?v=9HaOTX-m0SU
– L4 http://www.youtube.com/watch?v=YKIDg8VkF0A

? Sounds good on small speakers
– Chiptune sounds
– Lot of square waves / triangles
? Better estimation of pitch by the human
low- as well as high-pitched tones
? General remark:
? PC gamers often don’t spend money on a high-quality
speakers

Case Study: Quido (2013)
? Experimental arcade game
– Developed at DCGI/CTU
? Showcasing non-standard haptic modalities
– Presented on Designblok 2013
– http://ulab.cz/naviterier/quido/
? Genre:
– Contemporary chiptune
– https://quido.bandcamp.com/

Quido: Technology used
? Reaper
? VSTs
– SuperWave (leads / chords / bass)
– EXD-80 (beats)
– Some reverb, some volume compressor

Quido: Music
? Theme music
– Exposition of the main theme
– Happy, easygoing
– Not used in the end
? Explicit request from the exhibition organizer: Silence
between the games

Quido: Music
? Level 1: Theme Park
– A player will be likely to be receiving instructions
from a fellow player
– Expected playtime: 3—4 minutes
– Not “dense”
– Diverse in terms of the structure
– Performance of the main theme

Quido: Music
? Level 2: City
– Variation of the theme.
– Structure: [A B A Bbr Avoid]
– Expected playtime: 2 minutes
– Continuous music

Quido: Music
? Level 3: Sky (Heaven)
– Variation of the theme. Barroque.
– Continuous music
– Expected playtime: 2 minutes

Quido: Music
? Level completed
– Joyous jingle
? Advancing to the next level
– Expectation

Quido: Music
? Game Over
– Definitive
– Not too tragic

Quido: Music
? Game won
– Player enters their initials
– Reiteration of the main theme

Adaptive Music Styles
? Resequencing
– Selecting what to play next
? Reorchestration
– Selecting which instruments to play
? Modulation
– Modifying the timbres of notes
– Related to Data Sonification

The Space Game
? Game loops:
– Menu / Idle
– The enemy is approaching
– The enemy opens fire
? <demo>
– http://www.candystand.com/play/the-space-game

Reorchestration
? Multiple tracks
– Turned on/off
– Faded in and out
? Synchronous
<Demo: honza_test_02.sqc>

Reorchestration

Reorchestration
? Fallout: New Vegas
– 2010, Obsidian Entertainment, Bethesda
Softworks
– Multiple renderings of the same track
? “low”, “mid”, “high”
– <demo; NewCaliforniaRepublic>

Resequencing

Resequencing
? Slices chosen according to game state
? Precise timing of transitions
? <Demo: chiptune.sqc>

Resequencing and Reorchestration
? Example: Monkey Island 2
– 1991, LucasArts
– http://www.youtube.com/watch?v=Nsc5nTrCzm
w#t=322
– iMUSE system
? Parallel tracks for different locations
? Transition patterns to return back

Modulation
? Analog variables in the game
– Health
– Completion
– …
? Parameters of sound synthesis
– Volume
– Low-pass filter cut-off
– …

Modulation of Synthesis
? Mapping
game state ? sound synthesis
? <Demo: Angel choir gone wrong>

Modulation via FX Chain

Modulation
? Borderline music/sound effects
? Example: Stun grenade A/V
– Counter Strike Global Offensive
(Valve Corporation, 2012)
? https://www.youtube.com/watch?v=YmwRw8fP_8c
– Payday 2
(505 Games, 2013)
? http://youtu.be/kThVfdZvgz0?t=26m35s

Music Composition
? Movies
– Music traditionally created aside from the entire
project
– Story is defined
– Timing is known
? “Horizontal” production

Music Composition
? Computer Games
– Games are not (very) predictable
– “Horizontal” production
? Advantages
– Music per se works well (classic composition techniques)
– No limitations on instrumentations etc.
? Problems
– Repetition
– Abrupt changes
– Failure to respond

Music Composition
? Computer Games
– “Vertical” production
? Advantages
– Good support of the moment
? Problems
– Difficult to express tensions, developments
– Music forms get disrupted
– Discontiuity
– Can be too descriptive

Music Production
? Input
– Artistic intent
– Gameplay description
? List of supported locations
? List of supported situations
– Available technology
? Platform, available storage, …
– Legal considerations
? Licensing of the game

Music Production
? Orchestration
– What instruments are used
? Structure
– Duration, planning
? Production
– Composing
? Mixdown / Mastering
– “Locking in” the music
– Optimizing the volumes, polishing

Music Production
? Practical problems
– Multiple stakeholders
– Tons of files (as with any content development)
– Hiring composers
– Finding instrumentalists

ADAPTIVE
MUSIC
IN GAMES
ADAM J SPORKA
@adam_sporka

狠狠撸

Adaptive Music in Games

Recommended

More Related Content

What's hot (20)

Similar to Adaptive Music in Games (8)

Recently uploaded (20)

Adaptive Music in Games