game environment from an auditive perspective
The point of departure for this paper is to consider the concept of
spatial practice as a possible perspective for the understanding of
computer and videogames. Espen Aarseth (2000, p.169) states that [t]he
problem of spatial representation is of key importance to the genres
aesthetics. Jesper Juuls (1999, p.46) seems to confirm this position
when he says: computer games are almost exclusively set in a space.
He does however not follow up on this important observation.
Sound is not necessarily the first thing that comes to mind when we
think about spatial
representation. Nevertheless, it is a significant factor in the emergence
of specific immersive environments generated by contemporary 3D computer
sound serves other important purposes, which are not exclusively related
to spatial representation: to inform and give feedback, to set the mood,
rhythm and pace, and to convey the narrative. If we consider all of
this and if we agree that the majority of games are audiovisual artefacts,
it is curious why sound is given so little attention in the literature.
This does not necessarily come as a surprise if we remember that it
took decades in the field of film studies to develop a deeper knowledge
of the inner workings of sound practice. Indeed, the marginalisation
of sound and the concentration on vision as the dominating sense can
be found in theoretical approaches to all audiovisual media systems.
methodological model by Lars Konzack (2002, p.89), which sets out to
provide a framework for the complete analysis of any particular computer
game does not even
mention sound. He has used his model to analyse the fighting game Soul
1999), and developed seven layers of analysis: hardware, program code,
functionality, game play, meaning, referentiality and socio culture.
Elements of the visual aesthetics are described as part of game play,
meaning and referentiality. I am convinced that the various effect sounds
connected to the different weapons in a game like Soul Calibur are important
elements of the gameplay as they generate feedback about the players
performance while they are referentially linked to the genre of martial
art films and their distinctive sound design. This is one of many examples
for the marginalisation of sound in the current literature.
sound does turn up on the agenda, it is very often in relation to film
and music. Poole (2000, p.80) for example argues that a superficial
similarity between films and video games exists because they communicate
to eyes and ears of the audience and both share methods of sound production.
There are indeed some of similarities between the sound practice of
film and computer games, which makes use of film theory relevant in
this context. Of course, the complex issues of the use of sound in film
are still discussed and remain controversial. However, there is one
unifying element among the many unanswered questions, [one] claim
on which we all can agree: the image has been theorised earlier, longer,
and more fully than sound.(Altman, 1992, p.171).
I would like to argue that computer games have a very specific way of
which is different from film as we will see in the following.
to Brenda Laurel (1991, p.161), [t]ight linkage between visual,
kinaesthetic, and auditory modalities is the key to the sense of immersion
that is created by many computer games, simulations and virtual-reality
In this paper we will attempt to a closer look at this linkage
between the different sensual modalities in relation to the spatial
nature of computer games.
Even though we take an auditory perspective on the game environment,
it is crucial for our undertaking that sound cannot be analysed in isolation
and that visual and kinaesthetic modalities must be taken into consideration.
will discuss the notions of the game environment and the user environment.
Then we will define a number of sound objects. Finally, the spatial
functions which are
regulating the deployment of sound objects in the game environment will
be described. As a basis for our examination and for demonstration purposes,
we will use the game Metal Gear Solid 2, Sons of Liberty (MGS2). The
reason for choosing this game is that although it can be seen as part
of a larger genre it uses sound in an innovative way. MGS2 was developed
by Kazuki Muraoka at Konami and primarily uses 3rd person perspective.
The main theme is tactical espionage and infiltration. Hideo Kojima
led the sound production team.
the impact of the technological evolution on the concepts and strategies
sounddesign for computergames, we will deliberately avoid to discuss
them here. There are numerous sources covering the history of technological
development. A good example is the account given on Joerg Weskes
website (Weske 2000). We will assume that the reader is familiar with
the concepts of stereo sound and Dolby Surround 5.1 which is used by
The distinction between user and game environment
It is important to first define the categories influencing the players
spatial experience. One category is constituted by the actual real world
space in which the player plays the game and listens to its audio. It
is quite obvious that external factors such as the size and the nature
of the room (game arcade, living room or game boy on a bus), as well
as the specific hardware used (stereo headphones or home theatre) influence
the auditory spatial experience of a game. The nature of this type of
space that I will in the following refer to as the user environment
has a very high variability and essentially differs from player to player.
Although this could open an interesting field for research into the
acoustic qualities of the real world surroundings of players
and their influences on the game experience, e.g. in public spaces or
at home, this study intends to look at the mechanics that can be observed
in the game itself. This
means that for our purposes, we will assume an ideal user
environment based on the hardware needs derived from the game manual
and an ideal room that is free of external auditory influences.
to that there is the space brought forward by the game itself. It can
be understood as a collection of sound elements organised as culturally
coded representations of space. This space that we will call the game
environment, encompasses the sounds originating from the game during
play. This also includes the sound of the credits before and after the
game, all the sounds related to its interface as well as the program
or software that defines their deployment or generation.
Although our perspective is directed towards sound, most computer games
are audiovisual artefacts. It is therefore sensible to widen the category
of the game environment to include the visual elements in a game, as
well as the program defining the relations between visual and sound
environment is usually generated by a very consistent set of elements
that will not vary from player to player. As most games are packaged
as a product in a specific form on a storage device the distribution
of the same data to all the players is ensured. However, there is number
a growing number of games that reside on dedicated internet servers
and are thus altered and updated periodically. Game modifications, patches,
add-ons and updates are also changing the game environment. The following
is a sketch for a method of game audio analysis in relation to spatial
practice and it focuses on the elements and functions in the game environment.
3. Sound Objects
Now that we have established the game environment as the origin of sound
and graphics, it is necessary to develop a way of describing and differentiating
the sounds it generates. In this respect it might be interesting to
consider the status given to sound in the production of games and the
concepts arising from it.
The structure that defines the relations between all the elements in
a game is commonly
referred to as the game architecture. Following the concept of object
oriented programming, all the separate elements of the game, such as
pre-recorded sound files and textures are understood as objects. These
are organised in classes. The software that makes the interaction between
these objects possible is called the game engine. A game engine is first
and foremost a set of software libraries. The architecture consists
of the objects within classes on the one hand and the libraries defining
how they are implemented in the game environment on the other. Following
this concept, every sound in the sound library is referred to as an
object and it is treated on the same level as a graphic file, as one
of the many objects comprising the game.
to emphasise the discrete nature of the sound elements used in the game
environment, we will use the term sound object for our purposes. As
they can also be
produced dynamically by a program it is important not to limit our understanding
term solely to the actual sound files on a CD-ROM or harddrive, because
they could also be generated by a dedicated algorithm for a soundchip.
How is it possible to access and
describe these sound objects if we do not have a direct access to the
games sound library?
researchers are familiar with the dilemma of being confronted with the
box of the program on the one hand, and the output of the game on the
other. Jesper Juuls (1999, p.35). triadic model of games describes this
as the interaction between
program (game architecture) material (sound, text, graphics) and output.
He says that
The interesting focus in a system like this regards the relationship
represented and the rules for the combinations of material. and
claims that [in] the
computer game: the material and the program can be taken apart(1999,p.36).
He is using this dichotomy between program and material to set computer
games apart from traditional narrative text, following Aarseths (1997,
p. 94) model of ergodic text.
Juuls even argues (1999, p.36) that the rules of the game are of higher
importance than the material, for the skilled player of an action game.
This might hold true for a limited number of games but it is certainly
not true for a game like MGS2 that has a very idiosyncratic and aesthetically
diverse material side. In other words one could produce a new game based
on the rules present in MGS2 but it would be a very different experience.
we can say that it is precisely the dynamic relation between material
and program that separates sound practice in a computer game from other
media and thus we will concentrate on it. In our context this means
that the sound objects, of which
many belong to the material, can first of all be grouped according to
their use in the game. Because the program defines the use of sound
objects in the game, we have to focus on the relation between program
and material all the time. With the advent of RDSP (Real-time Digital
Signal Processing) this dynamic relation between program and material
is emphasized because the program has the ability to dynamically alter
the aesthetic appearance of sound objects.
of sound objects in the game architecture reveals one of the major differences
between the structuring of sound in a game and the sound practice in
film. All the sounds or objects are part of a dynamic environment. Their
qualities, such as pitch, volume, reverberation and other effects, as
well as the relations among them are defined by a program that can also
be influenced by user action. Every sound object can potentially enter
a temporal and spatial relation with every other sound or graphic object.
Obviously, the program differs to a great extent between different types
of computer games, and the following typology might have to be adapted
The notion of the sound object clearly resonates with meaning from the
field of music
theory and its use in the game context has to be clarified. The objet
introduced by the French theorist and musician Pierre Schaeffer in his
Traité des Objets Musicaux(1966). Schaeffer proposed
the method of reduced listening; a way of listening that would avoid
the habit of searching for the semantic properties of sounds and instead
try to find ways of describing their specific properties and perceptual
characteristics. It is quite obvious that a mode of reduced listening
will not be achieved when we are playing an audiovisual game, simply
because we are drawn to construct relations between the visual and auditory
information we are receiving. However, it is possible to describe sound
qualities, which generate a spatial understanding independent of an
indexical connection to their source. Ulf Wilhelmsson (2001, p. 119)
has shown that sounds can contain orientational information, such as
up-down or approaching-leaving schemes, via the change of their pitch.
He claims that sounds have spatial qualities without being indexically
linked to their material source and describes how changes in loudness
and pitch can generate the illusion of movement. Sound objects could
be analysed according to these inherent qualities as Wilhelmsson has
shown. Yet, this paper intends to focus on the use of sound objects
in the game environment and there is no room to describe the inherent
qualities of the sounds themselves in detail here.
instead attempt develop a typology of sound objects according to their
use in the game MGS2. Pre-rendered cinematics are excluded from this
analysis, as most of their characteristics could be analysed fully by
taking a film theoretical approach. This has been demonstrated by Sacha
Howells (2002, p.110). Following this initial typology we will try to
explain how sound objects are used in relation to each other as well
as in relation to the visual elements in the game and the user action.
Types of sound objects in the game environment
This typology of sound objects aims to identify the inherent qualities
of different types of sound objects present in the game environment.
We will discuss five different types of
sound objects, namely speech, effect, zone, score and interface sound
objects. This is a
preliminary classification based on the observation of MGS2, and it
might be possible to
define more or other sound objects in other games.
Speech sound objects
Speech is used in a variety of ways in computer and videogames.
Mostly it is employed as an intrinsical element of the diegetic system,
narrative of the game. Speech will either be recorded speech, spoken
by voice-over actors, or synthesized by the FM chip, producing the type
of speech that is usually associated with computers or robots. In MGS2
speech sound objects are used in a various ways. All the important characters
in the game, such as Solid Snake, Raiden, Vamp, Rose, Otacon are linked
to speech sound objects derived from voice-over acting. It is interesting
to note that speech sound objects in MGS2 are always accompanied by
written text in the style of film subtitles. They are indeed the core
transport elements of the games narrative. Most options and objectives
are explained via speech. The game employs a radio/video communication
device that contacts the main character on a regular basis and enables
the player to reach other characters in order to receive directives
or help. Even the save interface is an element of the described communication
device. Each time the user saves a gamestate he is drawn into a conversation
with a person. This trick cleverly incorporates the save dialogue into
the diegetic system of the game.
sound objects are also constantly employed for the description and mapping
locations in the game environment. They are often qualitatively transformed
in order to represent other media systems and their specific aural qualities,
such as telephone, radio or TV. In MGS2 this is present in the radio
communication, which is accompanied by hissing and crackling noises.
The communication device is also used to emphasize the spatial separation
between the user and game characters. It enables the construction of
complex spatial relations between speakers, who are supposed to be in
separate locations in the game environment.
2 very different ways in which speech sound objects influence the spatial
practice of a game. One the one hand the spatial information can be
transported by the text in the form of language. Whenever a character
in MGS2 gives us directions, where to move, which object or place to
look for, it influences our movement in the game environment. On the
other hand speech sound objects can move through the game environment,
which generates a sense of the location of characters. This is especially
strong in a 3D sound environment, such as Dolby Surround. The movement
of sound objects in the game environment can be understood as a spatial
function, and we will discuss specific spatial functions following our
typology of sound objects.
Effect sound objects
Effect sound objects are sounds, which are cognitively linked to visual
objects or events in the game environment by the player. They are in
other words perceived as being produced by or attributed to visual objects
or events within the diegetic part of the game environment. Visual objects
in this context mean all the visual objects that are part of the game
environment, whether moving or static elements, directly interactable
numerous examples for visual objects, which are linked to effect sound
objects, such as opponents in the game, consumable objects, doors, transportation
devices and so on. Sometimes effect sound objects are connected to direct
user action, sometimes they are synchronised to visual events in the
game and at other times they are merely used to generate the impression
of an action without a visual equivalent.
The realm of the effect sound objects is generally constituted by all
the sounds, which are at the forefront of the users attention
with the exception of intelligible speech. They are often used to signal
changes in the game state. They can provide feedback about changes of
conditions in the game, such as the points gained, the health status,
birth (spawning) or death events. Here it is important to note that
they do not have to refer to a game object that is visually represented.
one hears in synchronisation to the movement of the avatar, motoric
sounds such as footsteps or motor sounds are included within this type.
In spatial terms effect sound objects have the ability of situating
objects in the game environment.
Just as speech sound objects, they can also be moved through the game
environment, which is in most cases created by panning in a stereo situation
from left to right, as well as modulating loudness in order to generate
the illusion of objects approaching or leaving. In the case of MGS2
there are large number of effect sound objects. They could be classified
as being linked to the avatar, the game characters, objects, and events.
However, this system is not rigid and the placing of some of these objects
in the classification could be discussed further. From this perspective,
the following list is not intended to be exhaustive:
sound objects linked to the avatar:
- External body related sounds generated by the avatars movement,
such as footsteps,
sounds produced by fighting or martial arts moves (a swishing type of
through the air), the sound of the avatar swimming, the sound of knocking
(intended to confuse guards), cries of pain when the player is hurt.
- Internal body sounds such as heartbeat - which is used in an intriguing
way to strengthen identification with the avatar in connection with
the vibration of the controller when the player is hiding in cupboards,
breathing sound when smoking a cigarette.
sounds of usable objects carried by the avatar:
- This includes all of the weapon sounds, such as a variety of guns,
grenades, a sword but also objects like binoculars and different types
of sensors, throwable objects and clothes.
sound objects linked to game characters:
- Movement sounds such as Footsteps, Weapon sounds, a particular sound
snoring sounds, yawning sounds.
sound objects linked to other entities in the game environment:
- Opening and closing of doors, hatches, cupboards.
- Elevators and other transport devices, servo sounds of cameras, flying
drones, helicopters, planes, ships, birds.
sound objects linked to events in the game environment:
- Sounds produced while objects are consumed by the avatar: power up
objects, ammunition objects, weapons and tools.
- Sounds produced by bombs before they explode (ticking) as well as
the explosion itself.
Zone sound objects
Zone sound objects are sounds, which are connected to locations in the
game environment. Zones or locations can be understood as different
spatial settings that contain a finite number of visual and sound objects
in the game environment. A zone might be a whole level in a given game,
or part of a set of zones constituting the level.
Zones are separated by differing causally linked visual, kinaesthetic
or auditory qualities. In special cases different zones overlap. Zone
sound objects are aurally defining zones within the game environment.
They can have an indexical or non-indexical connection with the number
of visual objects or events present in the zone. They share a lot of
qualities with the type of film sounds Michel Chion describes as ambient
or territory sound when he suggests to [
] call ambient sound,
sound that envelops a scene and inhabits its space, without raising
the question of the identification or visual embodiment of its source:
birds singing, churchbells ringing. We might also call them Territory
sounds, because they serve to identify a particular locale through their
pervasive and continuous presence.(1994, p.75).
in our context the spatial metaphor of the zone is preferable to the
notion of the territory. MGS2 contains a number of different zone sound
There are two main outside zones: an oilrig and a ship. On the platforms
of the oil rig the zone sound is generated by waves, wind and the sounds
of seabirds. The outside zone sound on the ship is mainly characterised
by the sound of raindrops that is particularly immersive. The inside
zone sound objects are usually dominated by ambient mechanical sounds,
such as the humming of an air conditioning system and in one particular
case a conveyor belt.
A very good example for a zone sound object is the sound in the underwater
level, a flooded part of the oil rig (Shell 2 Core) that has a very
particularly muffled sound quality that reproduces perfectly the immersive
experience of being under water.
Score sound objects
The game score or music consists of a number of sound objects that belong
to the nondiegetic part of the game environment. In numerous games the
player can decide to switch the music on or off independently from the
Game music is a very complex area and we will therefore only consider
the qualities of the score, which are important for the spatial apparatus
of the game.
Score sound objects can also be connected to locations in the game,
which makes them
significant for the spatial practice of a game. Generally the score
has a huge emotional
impact on the player and it can enhance the feeling of immersion. This
means there should not be too many gaps within the musical score of
a game as this would threaten the immersive bond with the player. Score
sound objects are often used to mask transitions and to veil load times
or idle situations. There are number of interesting aspects that remain
to be analysed, of which the possible dynamic relations between the
game-score and game events might be the most interesting.
of MGS2 has been produced by the Hollywood composer Harry Gregson
Williams, who has written the sound for action films such as The Rock,
Killers and Enemy of the State. It was his first game score and in an
on the making-of-DVD (Konami, 2002), that is shipped with the game,
he says that it was interesting for him to produce music without having
a visual reference for it. Instead he chose to write themes linked to
different actions or states in the game, such as sneaking, alert, action,
ambient as well as general feelings, like being watched watching, tension
and so on. Additionally he produced individual themes for the main game
characters, which is a pattern reminiscent of traditional film soundtracks.
He then delivered these elements as 1-minute clips, which were built
into the game by Konamis sound department. Overall the music is
synthesizer based and is built around several different drum patterns
related to the different states of alert in the gameplay.
Interface sound objects
Interface sound objects share most of the qualities of effect sound
objects with the notable exception that they are usually not perceived
as belonging to the diegetic part of the game environment. However,
it cannot be ignored that a number of games have managed to include
interface elements in very clever ways into the diegetic part of the
game environment. MGS2 is a perfect example for this because it manages
to include load/save dialogues into the overall game narrative.
Interface sound objects are all the sounds connected with saving or
loading gamestates, and with changing the settings for the game. In
the case of MGS2, all the sounds one hears when changing the different
game settings, such as the controller functions, image and sound settings,
onscreen representation of objects carried by the user can be understood
as interface sound objects. Here the sounds are short bleeps, which
can be heard whenever a setting is changed.
objects enhance the knowledge of the present location within a metaphorical
structure and give feedback about actions. Interface designers even
coined the term of the earmark, the sound equivalent of the icon. Current
research (Maaso, 2001) is trying to define the possibilities of adding
sound options to information media systems in order to improve usability.
This process is termed sonification.
Now that we have distinguished between different types of sound objects
we will move on to describe their interrelations as well as how they
function when they are linked to visual objects or events as well as
Spatialising functions in the game environment
Without doubt, one of the most important contributions to the understanding
of sound in
film has been made by the french theorist and musician Michel Chion.
In a number of texts he analysed the internal workings of visual and
auditive elements in film and TV. He
describes the relationship between sound and image, the audiovisual
contract, as a sort of symbolic pact to which the audio-spectator
agrees when he or she considers the elements of sound and image to be
participating in one and the same entity or world(1994, p.222).
This notion emphasizes the aspect of construction on the one hand and
underlines that we are facing cultural conventions that are subject
to change on the other. It is quite obvious that computer games are
offering an audiovisual contract as well. Yet, we have to ask ourselves
in which way the rules constituting this contract in a computer game
differ from film. We have already mentioned one major difference between
the two media earlier in the text: In film, the audiovisual contract
inscribes static relations between the elements, whereas in games they
are dynamic and potentially user driven. Following this logic, dynamic
relations between sound objects, visual objects and user action which
defining the spatial practice of a game can be understood as spatialising
function defines all the participants of a relation, as well as the
nature of it, over time.
the most important of these spatialising functions that we have already
mentioned, defines the movement of sound objects in the game environment.
following we will discuss how some other functions can be described.
many distinctive spatial functions, we will concentrate on the two most
employed by MGS2, namely the dynamic acousmatic and the spatial signature
It is important to note in this context that the use of a sound object
can be influenced by either one function or by combinations of them.
The dynamic acousmatic function
The acousmatic is a pythagorean term, which considers the
distance separating sounds from their origin. It refers to the situation
of the disciples listening to the words of the priest while he is hidden
behind a curtain. For some, the term is very precise and refers specifically
to this listening situation. However, it has gained wider usage, in
describing a genre, which, to a large extent derives from the Musique
Concrète tradition and is founded upon this specific listening
situation. The acousmatic has been very important for Pierre Schaeffer
in his discussion of the sound object, and Michel Chion transformed
it in order to describe the specific relations of sound and vision in
film. Radio, phonograph as well as telephone can be identified as purely
acousmatic media systems. Film on the other hand has the possibility
of showing the source of a sound. In a film an acousmatic situation
can develop along two different scenarios: either a sound is visualized
first, and subsequently acousmatized, or it is acousmatic to start with,
and it is visualized only afterwards.(Chion 1994, p.72).
of the acousmatic situation for the representation of a specific spatial
setup can be understood if one considers the nature of how people localize
sounds. They are usually trying to identify the source of a particular
sound. In a perceptual sense sound is surrounding us completely and,
even if there is spatialising at work through phase difference between
our ears, hearing is not directional in the same way as seeing. The
natural everyday action of accurately locating a sound that one hears
from a place that is not part of the visual field, would be to move
ones head in the general direction and to try to visually locate the
source of the sound. A film does not devolve that action to the viewer,
it is using the apparatus of the spatially fixed container of vision,
the frame and the possibility of sound editing to create different spatial
situations, which are then frozen and generate one possible temporal
and spatial narrative stream or order of events.
of sound as being either off or on screen is used frequently to describe
these acousmatic situations in film. It has been argued that it is not
really the sound that is on or off screen, because the sound can either
be heard or not, but that the reference is always being made to the
visual source of the sound. Christian Metz (1985, p.157) states that
[we] tend to forget that a sound in itself is never off:
either it is audible or it doesnt exist and he goes on to
say, [t]he situation is clear: the language used by technicians
and studios, without realizing it, conceptualizes sound in a way that
makes sense only for the image.(1985, p.158). Thus, the concept
of off and on screen sound seems to be problematic, or at least confusing
in our context. We will refer to sound objects that are related to visual
objects in the players field of vision as visualized sound objects.
In contrast, those sound objects linked to visual objects outside the
players field of vision will be called acousmatised.
computer games, and especially first person perspective 3D games, allow
the user to actively visualize or acousmatise sound objects. The player
of such a game
constantly has to make the conscious decision if the visual source of
a sound object is worth seeing, if the sound object should be visualized
within the game context. In other words, when playing a game such as
Quake (ID Software 1996), one constantly has to scan the aural field
for effect sound objects that are have been identified with opponents.
If one hears such a sound one will usually turn his head
(move the visual field of the virtual camera) in the direction of the
sound to visualize the sound and to precisely locate the source of it.
control over the acousmatisation and visualisation of sound objects
in the game environment is a key factor in creating unique spatial experiences
computer games. Whenever one might have wished to be able to identify
the source of a
strange sound, e.g. in a horror film, one had to acknowledge that the
timing of this
identification is up to the films author. A game like MGS2, which
is appropriating a lot of
conventions derived from film, gives the user the option to locate and
visualise the source of a sound. This is indeed an important element
of the gameplay. Even if we are only focus on how acousmatic situations
are being constructed temporally, the functional differences between
films and computer games immediately take shape.
In film the relations between sound and image are usually chronologically
defined after the filming has taken place. In the process of adding
the sound to the image, each connection has to be defined for each frame,
and the final outcome of the process is a product constituted by the
fixed bond of aural and visual elements.
In a computer
game we can also find a number of fixed relations between sound objects
and visual objects, but the temporal process of visualization and acousmatisation
of sound objects is a dynamic process that can be subject to the players
action. It can also be different each time a game is played. This state
of affairs makes it necessary to think about the mechanics at work as
dynamic functions. The user controlled dynamic acousmatic function is
one of the most important spatialising functions in video and computer
The sound objects used in these situations are in most of the cases
either speech sound
objects or effect sound objects. Dynamic acousmatic functions are very
important for a
stealth intrusion game like MGS2. The gameplay consists to a large extent
of hiding from
guards and opponents and sneaking past them without alerting them. In
order to remain
undetected the player has to stay hidden, which means that most of the
time he cannot see his opponents. He can however hear effect sound objects
indicating their approach such as footsteps as well as speech sound
objects i.e. conversations. Whenever the player has somehow attracted
the interest of the guards and alerted them the gamestate changes. This
is additionally indicated by an alarm sound and the small map that is
used to show the locations of opponents in the game environment disappears.
In these situations the player has no way of locating the enemy other
that listening for footsteps. Another interesting acousmatic situation
emerges from the avatar hiding in a cupboard. It is possible to get
a very limited amout of visual information through small slits in the
cupboard door, but one will only see the enemy when he is directly in
front of the cupboard. In these moments the acousmatic state shifts
and the effect sound object of enemy footsteps is visualized. These
functions are crucial for actively locating or situating objects in
the game environment and are thus very important features of the games
of a directional microphone as a feature that can be used in particular
parts of the game signifies the importance of sound in MGS2. In one
short episode of the game the player has to locate a specific hostage
held by terrorists in a room with thirty other hostages (Oil Rig Shell
1, Core 1 B). Due to the fact that the hostage has a heart pacemaker,
a directional microphone is used to listen for any unusual cardiac pattern.
In another situation (Oil Rig Shell 2, Core 1, Air Purification Room)
the same microphone is used to listen to a conversation that takes place
behind a wall. Because the source of the voice is moving, the player
has to move the directional microphone in the right direction to listen
This is a perfect example for a dynamic acousmatic function based on
speech sound objects that even defines a specific subtype of gameplay.
It generates a complex spatial setup that is dynamically linked to user
action, moving the microphone in the right direction. The idea of the
directional microphone has since also been used in the game Splinter
Cell (UbiSoft Entertainment 2002), to listen to a conversation in a
moving elevator and another one taking place inside an embassy building.
The spatial signature function
The fact that one particular sound event will have very different qualities
if it is heard by
different people in a different environment is familiar.
Rick Altman (1992, p.24) gives the example of a baseball that broke
his window and that
sounded very different for him, who was outdoors than for his father
in the house or for his mother in the basement. In his article about
the material heterogeneity of recorded sound, he deals with the problem
of perspective within recorded sound and he introduces the term spatial
signature in order to explain that [
] every recording
carries elements of this spatial signature, carried in the audible signs
of each hearings particularities (Altman 1992, p.24).Andrea
Truppin (1992, p.241) defines this phenomenon as follows: Spatial
Signature can be defined as a sounds auditory fingerprint that
is never absolute, but subject to the sounds placement in a particular
physical environment. These markers include reverb level, volume, frequency,
and timbre that allow auditors to interpret the sounds identity
in terms of distance or the type of space in which it has been produced
and/or is being heard. According to this definition, it is clear
that recorded sounds can have multiple signatures: There is the specific
spatial context of the original sound, the space at the source of the
recording, and the space of playback and its particularities.
spatial signature appear in the game environment? First and foremost
we can differentiate between the spatial signatures of recorded sounds
(qualities inherent to the sound object itself) and the functions that
allow computer games to simulate the spatial signature connected with
Most interesting for our analysis is the fact that contemporary sound
especially real-time DSP are able to simulate certain qualities that
define the spatial
signature of sounds. If the amount of reverb of any given sound object
(whether recorded or synthesized) is changed, this will simulate the
experience of hearing the sound in a different type of environment.
As large rooms suggest strong reverberation, a sound that is changed
by a reverb filter will bear the spatial signature of being reflected
in a large room. This simulation process literally turns the relation
between source and surrounding, as observed by Rick Altman on its head.
The user follows an inductive process from the isolated sound to an
assumption about the surrounding space. These simulations of particular
spatial signatures are used by many contemporary games. The most common
factors are the amount of reverb or echo of sound objects, but as the
game architecture gets more sophisticated, complex simulations like
the reflection of sound from walls or objects are being introduced.
This observation hints at the relation between representation and simulation
that is at work in computer games. Here is however not the time and
place further discuss this particular subject in further detail.
the spatial signature of sound objects is defined by the space surrounding
sound source, it can be understood as a function that operates on the
level of the zone sound object. A zone within the game environment shares
the rules defining its spatial signature, suggesting a coherent spatial
structure. In other words, if the zone sound object is defined as a
particular room in a house, the qualities of reverb and echo within
that room will be shared by all the sound objects it contains. Functions
defining the spatial signatures of sound objects can greatly enhance
the immersive qualities of a location in the game environment. If we
return to MGS2, we can note that the game does indeed present us with
places that have general auditive qualities.
We have referred to them as zone sound objects above, but spatial signature
functions also define the qualities of effect sound objects. For example,
all the sounds we hear in the flooded level of MGS2 (effect sound objects
linked to avatar movement as well as
exploding underwater mines) will be muffled in the same way and thus
convey a particular spatial signature.
An interesting example, that shows the range of influences of the spatial
signature function on different sound objects and that can literally
be understood as a type of audio perspective, can be found
in the outside levels of the oil rig.
MGS2 uses two optional visual perspectives, a third person perspective
mode, which usually shows the avatar from behind and above, as well
as a first person perspective.
The user switches between these two visual perspective modes throughout
The two visual perspectives correspond to two different aural perspectives.
If one switches to the first person perspective in the outside level
of the oil rig, one will hear a much louder rendition of the sound produced
by the wind. Essentially there are two different auditive spatial signatures
at work, which are related to the virtual camera. This function greatly
enhances the feeling of being in the place, while it refers to film
We have seen that sound is an important element of the spatial apparatus
of contemporary computer games. The sounds in the game environment have
been classified according to their use. The acousmatic spatialising
function and the spatial signature function have been proposed and described.
Although a lot remains to be said, I hope that this short paper has
managed to generate a point of departure for an auditive perspective
on games and that the research into this particular aspect will attract
more interest in the future.
As sound technology is evolving with new gaming platforms we will most
see more games that make innovative use of sound in the near future.
Quake (1996). ID Software.
Soul Calibur (1999). NAMCO.
Solid 2 Sons of Liberty (2001). Konami Computer Entertainment Japan,
Including: DVD The Making of Metal gear Solid 2 (2001).Fun TV.
Splinter Cell (2002). Ubi Soft Entertainment.
E. (2001). Allegories of Space, The Question of Spatiality in Computergames.
Cybertext Yearbook 2000. R. K. Markku Eskelinen. Jyväskylä,
Research Center for Contemporary Culture University of Jyväskylä.
E. (1997). Cybertext, Perspectives on Ergodic Literature. London, Johns
Hopkins University Press.
R. (1992). Material Heterogeneity of Recorded Sound. In: Sound Theory
Sound Practice. Ed R. Altman. New York, Routledge.
M. (1994).Audio-vision : Sound on Screen. New York ; Chichester, Columbia
S. A. (2002). Watching a Game, Playing a Movie: When Media Collide.
In: ScreenPlay, Cinema/Videogames/Interfaces. Eds. T. Krzywinska, G.
King. London, Wallflower Press.
J. (1999). A Clash between Game and Narrative, Institute of Nordic Language
and Literature. Copenhagen, University of Copenhagen.
L. (2002). Computer Game Criticism: A Method for Computer Game Analysis.
CGDC Conference Proceedings. F. Mäyrä. Tampere, Tampere University
B. (1991). Computers as Theatre. Menlo Park, Ca, Addison Wesley.
A. (2001). Sonification in web design: auditive possibilities for navigation.
(1985). Aural Objects. Film Sound: Theory and Practice. J. B. Elisabeth
Weis. New York, Columbia University Press.
S. (2000). Trigger Happy. The Inner Life of Videogames. London, Fourth
P. (1966). Traité des objets musicaux. Essai interdisciplines.
Paris, Editions du Seuil.
A. (1992). And Then There Was Sound: The Films of Andrei Tarkovsky.
in: Sound Theory Sound Practice. Ed. R. Altman. New York, Routledge.
J. (2000). Digital Sound and Music in Computergames.
U. (2001). Enacting the Point of Being: Computer games, interactivity
and film theory. PhD Thesis. Department of Film and Mediastudies. Copenhagen,
University of Copenhagen.
Axel Stockburger 2003
this document has been adapted for the sake of accessibility. You can
find the original .pdf document here.