An Interactive Approach to Emotional Expression Through Musical Cues

Previous literature suggests that structural and expressive cues affect the emotion expressed in music. However, only a few systematic explorations of cues have been done, usually focussing on a few cues or a limited amount of predetermined arbitrary cue values. This paper presents three experiments investigating the effect of six cues and their combinations on the music's perceived emotional expression. Twenty-eight musical pieces were created with the aim of providing flexible, ecologically valid, unfamiliar, new stimuli. In Experiment 1, 96 participants assessed which emotions were expressed in the pieces using Likert scale ratings. In Experiment 2, a subset of the stimuli was modified by participants (N = 42) via six available cues (tempo, mode, articulation, pitch, dynamics, and brightness) to convey seven emotions (anger, sadness, fear, joy, surprise, calmness, and power), addressing the main aim of exploring the impact of cue levels to expressions. Experiment 3 investigated how well the variations of the original stimuli created by participants in Experiment 2 expressed their intended emotion. Participants (N = 91) rated them alongside the seven original pieces, allowing the exploration of similarities and differences between the two sets of related pieces. An overall pattern of cue combinations was identified for each emotion. Some findings corroborate previous studies: mode and tempo were the most impactful cues in shaping emotions, and sadness and joy were amongst the most accurately recognised emotions. Novel findings include soft dynamics being used to convey anger, and dynamics and brightness being the least informative cues. These findings provide further motivation to investigate the effect of cues on emotions in music as combinations of multiple cues rather than as individual cues, as one cue might not give enough information to portray a specific emotion. The new findings and discrepancies are discussed in relation to current theories of music and emotions.

Effects of emotionally incongruent musical excerpts on memory retrieval

Music’s power to trigger memories has rarely been tested; in particular, it is not clear what mechanisms govern memory retrieval elicited by musical cues. Previous research has suggested that memory retrieval is underpinned by two mechanisms: (1) distinctiveness—the probability that a cue will retrieve a memory declines with the number of stimuli previously encoded with that same cue and (2) incongruence—a cue encoded with emotionally incongruent targets triggers more memories of the stimuli associated with it than a cue encoded with emotionally congruent stimuli. Our participants experienced an implicit encoding phase where they were presented with auditory-visual pairs of stimuli (pieces of music and images of facial expressions). In the retrieval phase, participants were asked to remember encoded stimuli triggered by music. As expected, musical cues encoded with emotionally incongruent facial expressions triggered more memories than cues encoded with congruent facial expressions. Contrary to our prediction and to previous findings, music from distinctive pairs of stimuli triggered fewer memories than cue pairs displayed multiple times in the encoding phase. Our finding suggests that the manipulation of stimuli at encoding is crucial when using musical cues to trigger memories.

Instrument Timbre Enhances Perceptual Segregation in Orchestral Music

Timbre perception and auditory grouping principles can provide a theoretical basis for aspects of orchestration. In Experiment 1, 36 excerpts contained two streams and 12 contained one stream as determined by music analysts. Streams—the perceptual connecting of successive events—comprised either single instruments or blended combinations of instruments from the same or different families. Musicians and nonmusicians rated the degree of segregation perceived in the excerpts. Heterogeneous instrument combinations between streams yielded greater segregation than did homogeneous ones. Experiment 2 presented the individual streams from each two-stream excerpt. Blend ratings on isolated individual streams from the two-stream excerpts did not predict global segregation between streams. In Experiment 3, Experiment 1 excerpts were reorchestrated with only string instruments to determine the relative contribution of timbre to segregation beyond other musical cues. Decreasing timbral differences reduced segregation ratings. Acoustic and score-based descriptors were extracted from the recordings and scores, respectively, to statistically quantify the factors involved in these effects. Instrument family, part crossing, consonance, spectral factors related to timbre, and onset synchrony all played a role, providing evidence of how timbral differences enhance segregation in orchestral music.

Adaptation, Authenticity, and Ethics in Carl Davis’s Score to the Thief of Bagdad

Contributor Geoffrey Wilson notes Davis’s self-proclaimed practice of reviving the score the films might have had in their original release but he faults the leading composer in the silent film revival movement for failing to realize his aim. Wilson cautions that a number of contemporary practices mark Davis’s scores, including his use of leitmotivs to denote characters, a willingness to allow those leitmotivs to shape the overall narrative, and his readiness to adopt a unified musical style for the film. Each practice belongs to a more contemporary period of film scoring. Wilson contends that one of the only aspects of silent scoring Davis keeps is a willingness to create musical cues that present non-Western characters and settings in essentialist if not outright racist ways. The choice makes Davis’s claim to authenticity misleading. While Davis’s scores may make a historical film more accessible to modern audiences, they do so by promoting reductive stereotypical constructs.

Surprise me Softly: The Element of Surprise in Designing Museum Experiences

Experience design has become a widely discussed topic. Museums use experience design for engaging their visitors and culture offers exceptional tools for it. Visual arts and music are particularly effective in eliciting visitors’ emotions. However, there are a number of visual and acoustic cues that influence museum visitor response behaviours. Understanding the ways in which the human brain processes information provides a basis for furthering experience design principles. This study focuses on the emotion of surprise, considered especially effective for engaging visitor attention, providing meaning and affecting memory. The methodology involved monitoring psychophysiological responses and self-reports to assess research participants’ reactions to visual/acoustic stimuli. The aim was to confirm/detect types of sensory stimuli that generate the emotion of surprise, to see if participants have similar reactions to stimuli and whether individuals’ self-reports are aligned with their psychophysiological reactions. The results showed that musical stimuli are more effective than visual arts in eliciting surprise. While the study showed no clear indications that visual cues have an effect on surprise, musical cues, such as rapid attack, large pitch variation, higher harmonics, slow tempo with a sudden interruption, and sudden change in loudness do seem to play a role. Other cues, such as major key, 4/4 meter, timbral difference, and diatonic harmony also have an impact on the elicitation of surprise. These are important implications for designing museum experiences.

All Eyes on Me

Duo musicians exhibit a broad variety of bodily gestures, but it is unclear how soloists’ and accompanists’ movements differ and to what extent they attract observers’ visual attention. In Experiment 1, seven musical duos’ body movements were tracked while they performed two pieces in two different conditions. In a congruent condition, soloist and accompanist behaved according to their expected musical roles; in an incongruent condition, the soloist behaved as accompanist and vice versa. Results revealed that behaving as soloist, regardless of the condition, led to more, smoother, and faster head and shoulder movements over a larger area than behaving as accompanist. Moreover, accompanists in the incongruent condition moved more than soloists in the congruent condition. In Experiment 2, observers watched videos of the duo performances with and without audio, while eye movements were tracked. Observers looked longer at musicians behaving as soloists compared to musicians behaving as accompanists, independent of their respective musical role. This suggests that visual attention was allocated to the most salient visuo-kinematic cues (i.e., expressive bodily gestures) rather than the most salient musical cues (i.e., the solo part). Findings are discussed regarding auditory-motor couplings and theories of motor control as well as auditory-visual integration and attention.

Musical emotions in the absence of music: A cross-cultural investigation of emotion communication in music by extra-musical cues

Research in music and emotion has long acknowledged the importance of extra-musical cues, yet has been unable to measure their effect on emotion communication in music. The aim of this research was to understand how extra-musical cues affect emotion responses to music in two distinguishable cultures. Australian and Cuban participants (N = 276) were instructed to name an emotion in response to written lyric excerpts from eight distinct music genres, using genre labels as cues. Lyrics were presented primed with genre labels (original priming and a false, lured genre label) or unprimed. For some genres, emotion responses to the same lyrics changed based on the primed genre label. We explain these results as emotion expectations induced by extra-musical cues. This suggests that prior knowledge elicited by lyrics and music genre labels are able to affect the musical emotion responses that music can communicate, independent of the emotion contribution made by psychoacoustic features. For example, the results show a lyric excerpt that is believed to belong to the Heavy Metal genre triggers high valence/high arousal emotions compared to the same excerpt primed as Japanese Gagaku, without the need of playing any music. The present study provides novel empirical evidence of extra-musical effects on emotion and music, and supports this interpretation from a multi-genre, cross-cultural perspective. Further findings were noted in relation to fandom that also supported the emotion expectation account. Participants with high levels of fandom for a genre reported a wider range of emotions in response to the lyrics labelled as being a song from that same specific genre, compared to lower levels of fandom. Both within and across culture differences were observed, and the importance of a culture effect discussed.

A Response to Michael Spitzer's Commentary

The authors respond to the commentary by Michael Spitzer which appeared in Vol. 14, No. 1-2 of Empirical Musicology Review. The response 1) points out the problem with equating nostalgia and tension in the perception of single chords, 2) makes a case for why studying the role of vertical harmony, isolated from other musical cues, is insightful, and 3) questions the relevance of appoggiaturas as an explanation for the perception of nostalgia in single chords.

Could Self-Consciousness Be Enhanced in Alzheimer’s Disease? An Approach from Emotional Sensorial Stimulation

Alzheimer’s disease (AD) provides a valuable field of research into impairment of self-consciousness (SC), because AD patients have a reduced capacity to understand their mental world, to experience and relive previous personal events, as well as to interpret thoughts, feelings, and beliefs about themselves. Several studies observed that AD patients had an altered SC, but not a complete abolition of it. Emotions are an integral part of the construction of personal identity, therefore of Self. In general, most studies on emotion in AD patients have observed that emotion is not completely abolished and it lets them better remember autobiographical events with greater emotional charge. The positive effect of autobiographical memories rich in emotional content, evoked directly/automatically by sensorial stimuli such as familiar odors or music, could be used to reestablish/reinforce the permanence and coherence of the Self in AD. We studied the research of empirical evidence supporting the power of the sensorial cues associated with emotion, which could be capable of enhancing the SC in AD. We presented the studies about “Emotional stimulations” using odor, music, or taste cues in AD. All studies have shown to have a positive impact on SC in AD patients such as odor-evoked autobiographical memories, taste/odor-evoked autobiographical memories, emotional sensorial stimulation using musical cues, and multi-sensorial stimulations using healing gardens. We found research supporting the notion that emotional sensorial stimulations can even temporarily exalt memory, affective state, and personal identity, that is, the SC in AD. The emotional sensory stimulations could be used as a tool to activate the SC in AD and hence improve the quality of life of patients and caregivers.

The Baby Shark (Songs Heard Affecting Resuscitation Kinetics) study

Previous research has examined the utilisation of musical cues to improve the performance of cardiopulmonary resuscitation (CPR) delivered in training environments. We postulated a musical cue that is both contemporary and transcends cultures may improve CPR performance. Our aim was to establish whether chest compressions are performed with improved rate and depth if a song of a fixed beat (PinkFong’s ‘Baby Shark’ with a tempo of 115 beats per minute (bpm) and 15 beats in each verse) is played to a healthcare professional immediately before undertaking CPR compared to whale noises (a non-metronomic rhythm). 58 Participants of a paediatric conference (majority doctors) were randomly assigned to listen to a minute of Baby Shark (28) or whale song (30) and then undertake a minute of CPR. There was no significant difference in the mean compression rate between the Baby Shark and control groups, with the groups achieving 121 and 125 bpm, respectively (p=0.18). In relation to compression depth within the target zone, the Baby Shark group had more compressions completed within the target zone (55%) than the control group (39%) although this difference was not significant (p=0.08). Listening to Baby Shark prior to undertaking simulated CPR does not improve overall performance, but there is a potential tendency to improve adequate compression depth which may be beneficial in training exercises.