Synaesthesia (Games visualizing sound)

Posted: January 17, 2011 by Tim Utley in Editorials

What is Synaesthesia?  Not a bad question to ask because it is not usually used in the American lexicon.  Most simply it is the visualization of sound.  When you turn on your visualizer in iTunes, it will try and simulate or “visualize” the music you are listening to.  That is a very small fragment of the synaesthesic dimension, but an identifiable example of what I am trying to establish.  Through the perfect placement of certain tones and notes some artists have been able to create pictures with sound.  It is truly a sight to behold if you get a chance to see it.

Music has been able to accomplish this phenomenon, but videogames have taken a few stabs at it as well.  The most famous example of synaesthesia being used in a game is Tetsuya Mizuguchi’s “Rez”.  The music based rail shooter lets the player create music through the destruction of enemies you encounter in the game.

Creating music has never been more fun

While Rez might be the epitome of synaestheic gaming Mizuguchi is releasing a spiritual successor in the form of Child of Eden which will be available for both the Xbox 360 and Ps3 utilizing both systems respective motion control functions.

The most contemporary example of synaesthesia in gaming might be less synaesthesic and more rhythmic, but I am willing to argue for the first.  The Bit.Trip franchise is what I am alluding to.  This downloadable franchise from publisher Aksys Games and developer Gaijin Games offers up an unique retro style experience coupled with heavy rhythmic elements (let me explain).  My favorite entry into the series “Runner” will be my shining example of how Bit.Trip is rooted in synaesthesic conventions.  The player controls the infallible Commander Video in his quest to run, jump, kick, and slide through 50 levels of awesomeness.  I am getting side tracked, so how does this game visualize sound?  Every movement that you make with Commander Video creates a sound.  Each level is designed to create particular sounds that accompany the game’s soundtrack and as you move through those levels your performance will create a unique sound.  Through visualizing the level design around sound the player creates his/her own remix of the background track.  Not only is the chip-tune soundtrack a blast to listen to, it is fun to alter the sound with CV’s movement.  Your run through each level is a visual representation of the music, pretty cool huh.

Run, jump, slide, kick to your own groove

Gaming has dabbled in many different areas including synaesthesia.  The titles I have looked at in regards to this phenomenon are not only exemplary in design they are just really enjoyable.  The mix between rhythm, sound, and movement will continue to play a large part in game development and hopefully more developers will utilize this function and continue to create more unique experiences for us all to enjoy.

If you are interested in checking out Bit.Trip Runner or Rez I have posted some videos below and if you have any games you think fall into this category don’t be shy and share them.

Bit. Trip Runner



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s