Guidelines for Sound Design in Computer Games

The inconsequential exploitation of sound in most computer games, both in extent and nature, contrasts with its prominence in our daily lives and with the kind of associations that have been explored in domains such as music and cinema. Sound design remains the craft of a talented minority and the unavailability of a public body of knowledge on the subject has greatly contributed to this state of affairs. This leads to a mix of alienation and best-judgment improvisation in the broader development community. A sensitivity to the potential of sound for the enrichment of the experience—with emphasis on game specifics—is, therefore, necessary. This study presents a contribution to the practice of sound design for computer games. An approach to intentional sound design, informed by multi-disciplinary interpretations of concepts including emotion, context, acoustic ecology, soundscape, resonance, and entrainment, is distilled into a set of design guidelines that holistically address the different sound layers.

New Wine in New Skins

With the disappearance of technological constraints and their often predetermining impact upon design, computer game sound has the opportunity to develop into many innovative and unique aesthetic directions. This article reflects upon related discourse and design practice, which seems strongly influenced by mainstream Hollywood film and by a striving for naturalism and the simulation of “reality.” It is proposed that this constitutes an unnecessary limitation to the development and maturation of game sound. Interestingly, a closer understanding of aesthetic innovations of film sound, in particular in relation to what can be termed “liberation of the soundtrack,” can indicate thus far unexploited potential for game sound. Combined with recent innovations in creative practice and technology, they serve as inspiration to propose new directions for game sound design, taking into account the inherent qualities of the interactive medium and the technological and aesthetic possibilities associated with it.

Spatial Sound for Computer Games and Virtual Reality

In this chapter, we discuss spatial sound within the context of Virtual Reality and other synthetic environments such as computer games. We review current audio technologies, sound constraints within immersive multi-modal spaces, and future trends. The review process takes into consideration the wide-varying levels of audio sophistication in the gaming and VR industries, ranging from standard stereo output to Head Related Transfer Function implementation. The level of sophistication is determined mostly by hardware/system constraints (such as mobile devices or network limitations), however audio practitioners are developing novel and diverse methods to overcome many of these challenges. No matter what approach is employed, the primary objectives are very similar—the enhancement of the virtual scene and the enrichment of the user experience. We discuss how successful various audio technologies are in achieving these objectives, how they fall short, and how they are aligned to overcome these shortfalls in future implementations.

Behaviour, Structure and Causality in Procedural Audio1

This chapter expands some key concepts and problems in the emerging field of procedural audio. In addition to historical, philosophical, commercial, and technological themes, it examines why procedural audio differs from earlier “computer music” and “computer sound”. In particular, the extension of sound synthesis to the general case of ordinary, everyday objects in a virtual world, and the requirements for interactivity and real-time computation are examined.

Physical Modelling for Sound Synthesis

While the first computer games synthesised all their sound effects, a desire for realism led to the widespread use of sample playback when technology matured enough to allow it. However, current research points to many advantages of procedural audio which is generated at run time from information on sound producing events using various synthesis techniques. A specific type of synthesis known as physical modelling has emerged, primarily from research into musical instruments, and this has provided audio synthesis with an intuitive link to a system’s virtual physical parameters. Various physical modelling techniques have been developed, each offering particular advantages, and some of these have been used to synthesise audio in interactive virtual environments. Refinements of these techniques have improved their efficiency by exploiting human audio perception. They have been implemented in large virtual environments and linked to third party physics engines, unveiling the potential for more realistic audio, reduced production costs, faster prototyping, and new gaming possibilities.

Player-Game Interaction Through Affective Sound

This chapter treats computer game playing as an affective activity, largely guided by the audio-visual aesthetics of game content (of which, here, we concentrate on the role of sound) and the pleasure of gameplay. To understand the aesthetic impact of game sound on player experience, definitions of emotions are briefly discussed and framed in the game context. This leads to an introduction of empirical methods for assessing physiological and psychological effects of play, such as the affective impact of sonic player-game interaction. The psychological methodology presented is largely based on subjective interpretation of experience, while psychophysiological methodology is based on measurable bodily changes, such as context-dependent, physiological experience. As a means to illustrate both the potential and the difficulties inherent in such methodology we discuss the results of some experiments that investigate game sound and music effects and, finally, we close with a discussion of possible research directions based on a speculative assessment of the future of player-game interaction through affective sound.

Emotion, Content, and Context in Sound and Music

Computer game sound is particularly dependent upon the use of both sound artefacts and music. Sound and music are media rich in information. Audio and music processing can be approached from a range of perspectives which may or may not consider the meaning and purpose of this information.Computer music and digital audio are being advanced through investigations into emotion, content analysis, and context, and this chapter attempts to highlight the value of considering the information content present in sound, the context of the user being exposed to the sound, and the emotional reactions and interactions that are possible between the user and game sound. We demonstrate that by analysing the information present within media and considering the applications and purpose of a particular type of information, developers can improve user experiences and reduce overheads while creating more suitable, efficient applications. Some illustrated examples of our research projects that employ these theories are provided. Although the examples of research and development applications are not always examples from computer game sound, they can be related back to computer games. We aim to stimulate the reader’s imagination and thought in these areas, rather than attempt to drive the reader down one particular path.

Uncanny Speech

With increasing sophistication of realism for human-like characters within computer games, this chapter investigates player perception of audio-visual speech for virtual characters in relation to the Uncanny Valley. Building on the findings from both empirical studies and a literature survey, a conceptual framework for the uncanny and speech is put forward which includes qualities of speech sound, lip-sync, human-likeness of voice, and facial expression. A cross-modal mismatch for the fidelity of speech with image can increase uncanniness and as much attention should be given to speech sound qualities as aesthetic visual qualities by game developers to control how uncanny a character is perceived to be.

Listening to Fear

This chapter aims to explain how sound in horror computer games works towards eliciting emotions in the gamer: namely fear and dread. More than just analyzing how the gamer produces meaning with horror game sound in relation to its overarching generic context, it will look at how the inner relations of the sonic structure of the game and the different functions of computer game sound are manipulated to create the horrific strategies of the games. This chapter will also provide theoretical background on sound, gameplay, and the reception of computer games to support my argument.

Perceived Quality in Game Audio

This chapter reviews game audio from a Quality of Experience point of view. It describes cross-modal interaction of auditory and visual stimuli, re-introduces the concept of plausibility, and discusses issues of interactivity and attention as the basis for the qualitative, high-level salience model being suggested here. The model is substantiated by experimental results indicating that interaction or task located in the audio domain clearly influences the perceived audio quality. Cross-modal influence, with interaction or task located in a different (for example, visual) domain, is possible, but is significantly harder to predict and evaluate.