Game Accessibility

• Home
  • Introduction
  • Aims and Activities
  • Participants
• Research
  • News
  • Resources
  • Project Gallery
• Games
  • Visually disabled gamers
  • Auditory disabled gamers
  • Physically disabled gamers
  • Learning disabled gamers
• Community
  • Forum
• Information
  • About Accessibility
  • Project Stuff
  • Contact
  • Sitemap

The Sound Alternative

by Richard A. van Tol

Game accessibility for players with a hearing impairment often revolves around text-based closed captions. This article discusses ideas to adapt game audio using more than text in order to enhance the immersion of the player in the game.

Game audio and accessibility

In the early days of video gaming hearing impaired players hardly encountered accessibility problems. Games consisted primarily of text and graphics and had very limited audio capabilities. Sound therefore played a very small role in game play. A player did not need to hear the sounds of Pac-Man in order to play it.
But when the audio capabilities of games increased, the use of text was reduced and the role of game audio became more important. A classic example is LucasArts’ adventure game LOOM which, when it first came out in 1989, was accompanied by a 30-minute audio prologue on audiocassette tape that explained the nature and history of the LOOM world. LOOM did not provide an alternative for hearing impaired players, who therefore missed out on a part of the game experience.

The number of problems hearing impaired players encounter in games has gradually grown over the years. In most cases this concerns playing the game with a disadvantage. There are, for example, games which feature puzzles that require hearing. In one level in Prince of Persia: The Sands of Time the main character is trapped in a room with many exits which all seem to be leading back into the same room. The solution to the puzzle is to follow only those exits where the sound of running water is heard when the character stands in front of them. If you get the wrong exit, a gong will sound and you find yourself re-entering the chamber. Without the ability to hear, this level can only be beaten by trying every combinations of exits. Other games, like many first person shooters, use audio to provide important clues (such as the sound of an approaching opponent) which give hearing players an advantage over those players who can’t. Many games feature dialog that provides specific story information, which often is vital to understand the game.

Closed captions

Fortunately there is an easy and cheap solution to solve these problems and it is called closed captions. Closed captions do not only include subtitling of speech, but also a description of sound effects and even music, allowing hearing impaired players to obtain this information.
The term "closed" in closed captioning means that not all viewers see the captions, only those who decode or activate them. This is distinguished from "open captions," where the captions are visible to all viewers. Open captions are sometimes referred to as "in-vision" in the UK. Captions that are permanently visible in a video, film, or other medium are called "burned-in" captions.
Not only players with a hearing impairment can benefit from closed captions in games. Many players appreciate captions to make sure no information is missed. Especially mobile games benefit from closed captions since these games are often played in public places with the sound turned off, either out of courtesy for other people in the same place, to save battery power or because the environment is too noisy to hear the sound anyway.

Closed captions address problems related to the players’ ability to play games. But what about the players’ experience of the game? Game audio not only helps players play a game, it also stimulates the players’ experience of the game. Sound designers aim to immerse the player into the game world using sound effects and environmental audio. Voice actors add fun, personality and realism to the polygon characters. Music is used to emphasize the narrative of the game, define the cultural identity of the game, build tension during a level and amplify the emotional status of the avatar. Game audio seeks to engulf, secretly competing with any real-world sound trying to distract the player.

When it comes to making game audio accessible for players with a hearing impairment, is text-based closed captioning the only alternative?

The sound alternative

Here are several ideas for game audio alternatives that might make games more immersive for hearing impaired players. Most of these techniques already exist in games.

Speaker portraits
When captioning speech in games it is important to distinguish speakers. Otherwise the player might not know which speaker says what. While color coding of the text is quite common, many games show a portrait of the character that is speaking next to the text. A good example of this is Advance Wars: Dual Strike.
The designers of Freedom Force vs. the Third Reich went one step further. In this game the portrait is replaced by an animation of the character speaking. The dynamic facial expressions of the character add more context to the text.

Action Captions
Remember Batman the television series in the 1960’s? During each fight scene, a visual representation of the sound was shown on screen. These consisted of bright colored text superimposed over the scene and positioned roughly at the location of the sound source. Action captions originate from comic books and are usually onomatopoeia - words that imitate the sound it is describing. Examples of games that use action captions are Freedom Force vs. the Third Reich and XIII.

Sound balloons
Another possibility to distinguish speakers is speech balloons, a graphic convention used in comic books. Speech balloons are usually placed next to or above the speaking character. The shape of the balloon indicates the type of speech, if it is spoken, whispered or screamed, or even if it is a thought. There are quite a few games that use speech balloons. An excellent example is Mario & Luigi: Partners in Time. The idea of speech balloons could be extended to 'sound balloons'. Descriptions of sounds are shown in visual containers, which in turn are placed within the game world. The balloons provide context to the sound and the location of balloons show the location of the sound source.

Sound Visualisations
Sound visualisations are single images or animations that are used as a reference to a sound event. Sound visualisations can show a hearing impaired player how a sound behaves, for instance the speed with which a leaking pipe drips. It can even carry additional emotional information. A good example of a game which uses visualized sound is The Sims 2. The game features several objects that make sound, such as a telephone, a stereo and speakers. When an object makes a sound, it is accompanied by an animation. The telephone will shake and emit cartoon-like, circular sound waves, just like the speakers. The stereo produces notes which gently drift upwards like soap bubbles, referring to music. Sound visualisations can easily be combined with other alternatives. To the right is a video-example of how a sound balloon is extended with an animation referring to the type and behaviour of the sound. Such animated sound ballons could even be controlled by the actual sound in real-time using visualization algorithms!

Video Clips
Video clips can provide additional information not present in just icons or text. For example, when a distant helicopter fires a rocket at the player’s character in a game, you could either show “helicopter fires rocket” in text or a small sign depicting a helicopter that fires a rocket. But the information might have more impact using a short clip of the helicopter firing a rocket. Or if in a game a zombie is approaching out of sight, a video clip showing the scene from the zombie’s perspective (with its arms stretched out and ready to turn the corner), is more likely to thrill a player than a text "approaching zombie".
The clip could be shown in a split screen or picture-in-picture (PiP). For a good example of split screen in a game have a look at Indigo Prophecy. An example of a game that uses picture-in-picture animations is XIII.

Meters
Music is the most difficult type of sound to adapt to another medium because of its emotional impact. But game music not only conveys emotion, but also plays a vital role in game play. For example, it is often used to reflect the level of safety or approaching danger. A danger meter would basically perform the same function. There are several games that already use a danger meter of some sort, such as Lupin the 3rd: Treasure of the Sorcerer King. Hitman: Contracts uses a small map which changes colour when the amount of danger changes.

Radars
Another visual indicator is a radar, which can give a detailed overview of where important sound sources are positioned. The Doom3[CC] mod for Doom 3 is a good example of such a ‘visual sound radar’. It gives players an idea of where a sound is coming from by indicating distance and direction.

Sign language
Many deaf people use sign language as their first language and they could benefit if games would also provide sign language instead of plain text. Although there are no examples of sign language in games yet, there have been several projects researching computer characters performing sign language. With more and more games putting extra emphasis on facial expressions and character gestures (like The Godfather) it is not unlikely that future games might provide a sign language option as well, which contributes to the immersion in the game for auditory disabled players.

Conclusion

These are just a few examples of alternatives to text-based closed captions. Many of these can be extended and/or combined. When comparing text-based closed captions to the methods described above, it is important to remember that text does have some benefits as well. Many examples, such as signs and sound balloons, all need to be designed. This is likely to take more time than text, which is simply written. Text is also the least likely to distract players. Several of the examples above also invade the game world by visualizing the location of the sound source. Designers who prefer realism are more likely to want text captions in the game's head-up display than sound balloons in the world of the game or action captions because of their strong tie with comics.

When you are going to add some sort of alternative to game audio to your game, get the audio team involved as well. It is their design that is adapted and they can tell you their intentions with the audio. They can also point out which part of the sound is more important than other sound at what time during game play.

Please remember games are to provide a compelling experience for the players to enjoy. By experimenting with alternatives to text-based closed captioning you might find new ways to create a game that is not only accessible for players with a hearing impairment, but also more immersive.

[Discuss this article in the forum]

(c) 2006 Richard A. van Tol, Accessibility

©2006 Stichting Bartiméus Accessibility. Disclaimer | Privacy policy | XHTML valid.