Sound-Motion Bonding in Body and Mind

2018 ◽  
pp. 144-160
Author(s):  
Rolf Inge Godøy
Keyword(s):  
Mental Simulation ◽  
Body Motion ◽  
Visual Features ◽  
Musical Experience ◽  
Subjective Experiences ◽  
Musical Sound ◽  
Motor Theory ◽  
Sound Motion

This chapter focuses on the links between sound and body motion in music. It can readily be observed that musical sound is produced by body motion and also triggers body motion in many contexts, meaning scholars have an inexhaustible supply of sound-motion bonding available for research. The main challenges here are to get an overview of the different kinds of sound-motion bonding at work in music, and to go deeper into the subjective experiences of sound-motion bonding. To this end, the chapter presents sound-motion bonding in a so-called motor theory perspective on perception, suggesting that whatever humans perceive of sound, motion, and/or visual features is spontaneously re-enacted in our minds, meaning active mental simulation of whatever it is that we are perceiving. This leads to the idea of sound-motion objects, entities that fuse sensations of sound and motion into salient and holistically perceived units in musical experience.

scholarly journals Motor Constraints Shaping Musical Experience

2018 ◽  
Vol 24 (3) ◽  
Author(s):  
Rolfe Inge Godøy
Keyword(s):  
Electronic Music ◽  
Music Perception ◽  
Music Cognition ◽  
Music Performance ◽  
Body Motion ◽  
Musical Experience ◽  
Musical Sound ◽  
Perceptual Features ◽  
Motor Constraints

In recent decades, we have seen a surge in published work on embodied music cognition, and it is now broadly accepted that musical experience is intimately linked with experiences of body motion. It is also clear that music performance is not something abstract and without restrictions, but something traditionally (i.e., before the advent of electronic music) constrained by our possibilities for body motion. The focus of this paper is on these various constraints of sound-producing body motion that shape the emergent perceptual features of musical sound, as well as on how these constraints may enhance our understanding of agency in music perception.

scholarly journals Ubiquitous Motor Cognition in Musical Experience: Open Peer Review of Jacques Launay’s “Musical Sounds, Motor Resonance, and Detectable Agency”

2015 ◽  
Vol 10 (1-2) ◽  
pp. 41
Author(s):  
Rolf Inge Godoy
Keyword(s):  
Peer Review ◽  
Essential Element ◽  
Body Motion ◽  
Musical Experience ◽  
Mental Images ◽  
Motor Resonance ◽  
Motor Cognition ◽  
Musical Sound ◽  
Evolution Of Music ◽  
Musical Sounds

Motor cognition, defined as the capacity to conceive, plan, control, perceive, and imagine body motion, is here seen as an ubiquitous element in music: music is produced by body motion, people often move in various ways when listening to music, and images of body motion seem to be integral to mental images of musical sound. Given this ubiquity of motor cognition in musical experience, it could be argued that motor cognition is a fundamental element in music, and thus could be hypothesized to also have been an essential element in the evolution of music, regardless of whether music is seen as primarily a social or as a more solitary phenomenon. It could furthermore be argued that music in all cases has intersubjective significance because of shared motor cognition among people, and also that this motor cognition may be applied to most perceptually salient features of music.

scholarly journals Sonifying the Shape of Human Body Motion using Motiongrams

2013 ◽  
Vol 8 (2) ◽  
pp. 73 ◽  
Author(s):  
Alexander Refsum Jensenius ◽  
Rolf Inge Godøy
Keyword(s):  
Human Body ◽  
Video Recording ◽  
Visual Display ◽  
Human Motion ◽  
Visual Similarity ◽  
Body Motion ◽  
Auditory Displays ◽  
Musical Sound ◽  
Starting Point ◽  
Spatial Shape

<p class="author">The paper presents sonomotiongram, a technique for the creation of auditory displays of human body motion based on motiongrams. A motiongram is a visual display of motion, based on frame differencing and reduction of a regular video recording. The resultant motiongram shows the spatial shape of the motion as it unfolds in time, somewhat similar to the way in which spectrograms visualise the shape of (musical) sound. The visual similarity of motiongrams and spectrograms is the conceptual starting point for the sonomotiongram technique, which explores how motiongrams can be turned into sound using &ldquo;inverse FFT&rdquo;. The paper presents the idea of shape-sonification, gives an overview of the sonomotiongram technique, and discusses sonification examples of both simple and complex human motion.</p>

scholarly journals Thinking Sound and Body-Motion Shapes in Music: Public Peer Review of “Gesture and the Sonic Event in Karnatak Music” by Lara Pearson

2013 ◽  
Vol 8 (1) ◽  
pp. 15
Author(s):  
Rolfe Inge Godøy
Keyword(s):  
Peer Review ◽  
Vocal Music ◽  
Body Motion ◽  
Western Music ◽  
Future Research ◽  
Hand Motion ◽  
Sound Motion ◽  
Insight Into

<p>It seems that the majority of research on music-related body motion has so far been focused on Western music, so this paper by Lara Pearson on music-re&shy;lated body motion in Indian vocal music is a most welcome contribution to this field. But research on music-related body motion does present us with a number of chal&shy;lenges, ranging from issues of method to fundamental issues of perception and multi&shy;modal integration in music. In such research, thinking of perceptually salient fea&shy;tures in different modalities (sound, motion, touch, etc.) as shapes seems to go well with our cognitive apparatus, and also be quite practical in representing the fea&shy;tures in question. The research reported in this paper gives us an insight into how trac&shy;ing shapes by hand motion is an integral part of teaching Indian vocal music, and the approach of this paper also holds promise for fruitful future research.</p> <p>&nbsp;</p>

scholarly journals Ecological Constraints of Timescales, Production, and Perception in Temporal Experiences of Music

2015 ◽  
Vol 9 (3-4) ◽  
pp. 224
Author(s):  
Rolf Inge Godøy
Keyword(s):  
The Other ◽  
Body Motion ◽  
Ecological Constraints ◽  
Musical Sound ◽  
Musical Imagery ◽  
Musical Works ◽  
Musical Discourse ◽  
The One ◽  
Different Order ◽  
At Will

In trying to structure our discussions of temporal experience in music, it could be useful to have a look at some basic ecological constraints of timescales, produc&shy;tion, and perception of music. This may hopefully help us to distinguish between on the one hand readily perceived features of sound and music-related body motion, i.e. con&shy;crete sonic, kinematic, and proprioceptive features, and on the other hand, more generic, amodal, and abstract elements in musical discourse, manifest in various symbolic representa&shy;tions such as notation, numbers, and diagrams. Given easily accessible music tech&shy;nologies, it is actually possible to experiment with different editions of musical works, i.e. concatenate fragments in different order and then evaluate the emergent contex&shy;tual effects in listening experiments. Also, given the faculties of musical imagery (de&shy;fined as our ability to mentally re-experience musical sound and body motion in the ab&shy;sence of physically present sound and body motion), we can at will recombine chunks of music in our minds and mentally scan through large musical works. The contention here is that such recombination in actual re-editing of musical sound or in musical im&shy;agery, will still be related to the basic ecological constraints of the timescales, production and perception in music.

scholarly journals Constraint-Based Sound-Motion Objects in Music Performance

2021 ◽  
Vol 12 ◽  
Author(s):  
Rolf Inge Godøy
Keyword(s):  
Motor Control ◽  
Crucial Role ◽  
Music Performance ◽  
Building Blocks ◽  
Body Motion ◽  
Control Constraints ◽  
Practical Applications ◽  
Sound Motion ◽  
Generative Processes

The aim of this paper is to present principles of constraint-based sound-motion objects in music performance. Sound-motion objects are multimodal fragments of combined sound and sound-producing body motion, usually in the duration range of just a few seconds, and conceived, produced, and perceived as intrinsically coherent units. Sound-motion objects have a privileged role as building blocks in music because of their duration, coherence, and salient features and emerge from combined instrumental, biomechanical, and motor control constraints at work in performance. Exploring these constraints and the crucial role of the sound-motion objects can enhance our understanding of generative processes in music and have practical applications in performance, improvisation, and composition.

Digiti Sonus and Eyes

Public ◽  
2020 ◽  
Vol 30 (60) ◽  
pp. 156-163
Author(s):  
Yoon Chung Han
Keyword(s):  
User Interface ◽  
Data Visualization ◽  
Design Process ◽  
Interface Design ◽  
Feature Analysis ◽  
Visual Features ◽  
Visual Feature ◽  
Biometric Data ◽  
Musical Sound ◽  
Multimodal Interactions

This paper discusses design process and motivations of two interactive biometric data artworks (Digiti Sonus and Eyes). Narratives and artistic explorations using two forms of biometric data, from fingerprints and the iris, are discussed based on insights from various fields such as genetics, visual feature analysis, user interface design and data visualization and sonification. Digiti Sonus and Eyes extracted the unique visual features of each type of biometric data and transformed the data into musical sound with multimodal interactions so that the result was real time experimental sound. Various aspects of the two artworks are compared in this paper.

Exploring Sound-Motion Similarity in Musical Experience

2016 ◽  
Vol 45 (3) ◽  
pp. 210-222 ◽  
Author(s):  
Rolf Inge Godøy ◽  
Minho Song ◽  
Kristian Nymoen ◽  
Mari Romarheim Haugen ◽  
Alexander Refsum Jensenius
Keyword(s):  
Musical Experience ◽  
Sound Motion

The Pursuit of Intensity

2021 ◽  
pp. 223-294
Author(s):  
Braxton D. Shelley
Keyword(s):  
Music Analysis ◽  
Phenomenological Approach ◽  
Gospel Music ◽  
Musical Experience ◽  
Musical Sound ◽  
The Third ◽  
Tonal Modulation ◽  
Musical Syntax ◽  
The Relationship ◽  
The Way

This chapter offers a phenomenological approach to the vamp’s form, arguing that gospel vamps emerge as repetition and intensification become musical conduits of belief. Beginning with an analytical essay on the live recording of Smallwood’s “Anthem of Praise,” the chapter elucidates the interpenetration of compositional strategy and religious expectation in the gospel tradition. Its second section interrogates the phenomenological implications of gospel’s participatory character and analyzes a performance of Brenda Joyce Moore’s “Perfect Praise” by Lecresia Campbell and the Houston Chapter of the Gospel Music Workshop of America in order to clarify the relationship between musical syntax and musical experience—“the gospel stance.” The third part of this chapter weaves together analytical vignettes and theories of repetition, groove, and teleology, theorizing the vamp’s “affective trajectory.” In so doing, this section pays special attention to tonal modulation, “inversion,” and textural accumulation, three techniques that pervade the gospel choral repertory. The chapter’s fourth move reflects on the practice of music analysis, using Kurt Carr’s “For Every Mountain” and Thomas Whitfield’s “Soon as I Get Home” to assert that the chapter’s concern with the way sound is organized provides a deeper understanding of the way musical sound structures believers’ traffic between the seen world and another. This interchange motivates the gospel song’s relentless pursuit of intensity, a quest that comes into particularly clear relief in the chapter’s concluding analysis of Smallwood’s “I Will Sing Praises.”

scholarly journals Is color an integral part of a rich mental simulation?

2017 ◽  
Author(s):  
Lara Natasha Hoeben Mannaert ◽  
Katinka Dijkstra ◽  
Rolf Antonius Zwaan
Keyword(s):  
Object Recognition ◽  
Language Comprehension ◽  
Mental Simulation ◽  
Original Study ◽  
Visual Features ◽  
Perceptual Information ◽  
Current Investigation ◽  
Color Saturation ◽  
Match Effect ◽  
Previous Sentence

Research suggests that language comprehenders simulate visual features such as shape during language comprehension. In sentence-picture verification tasks, whenever pictures match the shape or orientation implied by the previous sentence, responses are faster than when the pictures mismatch implied visual aspects. However, mixed results have been demonstrated when the sentence-picture paradigm was applied to color (Connell, Cognition, 102(3), 476–485, 2007; Zwaan &amp; Pecher, PLOS ONE, 7(12), e51382, 2012). One of the aims of the current investigation was to resolve this issue. This was accomplished by conceptually replicating the original study on color, using the same paradigm but a different stimulus set. The second goal of this study was to assess how much perceptual information is included in a mental simulation. We examined this by reducing color saturation, a manipulation that does not sacrifice object identifiability. If reduction of one aspect of color does not alter the match effect, it would suggest that not all perceptual information is relevant for a mental simulation. Our results did not support this: We found a match advantage when objects were shown at normal levels of saturation, but this match advantage disappeared when saturation was reduced, yet still aided in object recognition compared to when color was entirely removed. Taken together, these results clearly show a strong match effect for color, and the perceptual richness of mental simulations during language comprehension.

Sign in / Sign up

Export Citation Format

Share Document