GDC: Akira Yamaoka on the disturbing power of audio
The title of Silent Hill composer Akira Yamaoka's GDC panel was "As long as the audio is fun, the game will be too." Kind of a perplexing title, especially if you've enjoyed Yamaoka's work in the horror genre. Don't get us wrong, it's good, we're just not sure fun is the word for it. Yamaoka used the panel to explain precisely why his music and sound effects are so disturbing, and it's more technical than you might think.
"Sound is very close to stimulating emotions in your brain," said Yamaoka via an interpreter. "What do humans fear?" He asked. In the case of audio, he noted that gaps between visual cues and accompanying audio. Specifically, if a sound precedes a visual cue, it can cause anxiety, while a sound that arrives too late is comforting. As an example, he noted the sounds of footsteps being made slightly before the onscreen action in a game. He noted that this effect can be felt even if the sync between sound and visual is off by a few frames of animation. Yamaoka said later in the panel that he used this technique in Silent Hill 3.
Another technique mentioned was the use of silence."When there's no sound, the brain fills in the gap," said Yamaoka. He then played a special version of Beethoven's "Für Elise" which had been injected with tiny, periodic gaps of silence. He then played the same piece, only this time injecting harsh noise instead of silence. Even packed in a fully lit lecture hall with hundreds of GDC attendees, it was unsettling. Unfortunately, Yamaoka wasn't able to give us the sound samples used during the panel, so we've done our best to recreate them:
Thankfully, Yamaoka closed the session with a soothing guitar performance, which we were able to capture for your enjoyment. Listen to it below and stay tuned for an interview with Mr. Yamoaka on Joystiq in the near future.
