Does serial memory of locations benefit from spatially congruent audiovisual stimuli? investigating the effect of adding spatial sound to visuospatial sequences

Purpose The purpose of this study was twofold: (a) to identify a set of communication-based predictors (including both acoustic and gestural variables) of masculinity–femininity ratings and (b) to explore differences in ratings between audio and audiovisual presentation modes for transgender and cisgender communicators. Method The voices and gestures of a group of cisgender men and women ( n = 10 of each) and transgender women ( n = 20) communicators were recorded while they recounted the story of a cartoon using acoustic and motion capture recording systems. A total of 17 acoustic and gestural variables were measured from these recordings. A group of observers ( n = 20) rated each communicator's masculinity–femininity based on 30- to 45-s samples of the cartoon description presented in three modes: audio, visual, and audio visual. Visual and audiovisual stimuli contained point light displays standardized for size. Ratings were made using a direct magnitude estimation scale without modulus. Communication-based predictors of masculinity–femininity ratings were identified using multiple regression, and analysis of variance was used to determine the effect of presentation mode on perceptual ratings. Results Fundamental frequency, average vowel formant, and sound pressure level were identified as significant predictors of masculinity–femininity ratings for these communicators. Communicators were rated significantly more feminine in the audio than the audiovisual mode and unreliably in the visual-only mode. Conclusions Both study purposes were met. Results support continued emphasis on fundamental frequency and vocal tract resonance in voice and communication modification training with transgender individuals and provide evidence for the potential benefit of modifying sound pressure level, especially when a masculine presentation is desired.

Download Full-text

The Effect of Path Length and Display Size on Memory for Spatial Information

Experimental Psychology (formerly Zeitschrift für Experimentelle Psychologie) ◽

10.1027/1618-3169/a000137 ◽

2012 ◽

Vol 59 (3) ◽

pp. 147-152 ◽

Cited By ~ 11

Author(s):

Katherine Guérard ◽

Sébastien Tremblay

Keyword(s):

Path Length ◽

Potential Role ◽

Spatial Information ◽

Recall Performance ◽

Minor Role ◽

Length Effect ◽

Serial Memory ◽

Fixed Reference ◽

A Minor

In serial memory for spatial information, some studies showed that recall performance suffers when the distance between successive locations increases relatively to the size of the display in which they are presented (the path length effect; e.g., Parmentier et al., 2005) but not when distance is increased by enlarging the size of the display (e.g., Smyth & Scholey, 1994). In the present study, we examined the effect of varying the absolute and relative distance between to-be-remembered items on memory for spatial information. We manipulated path length using small (15″) and large (64″) screens within the same design. In two experiments, we showed that distance was disruptive mainly when it is varied relatively to a fixed reference frame, though increasing the size of the display also had a small deleterious effect on recall. The insertion of a retention interval did not influence these effects, suggesting that rehearsal plays a minor role in mediating the effects of distance on serial spatial memory. We discuss the potential role of perceptual organization in light of the pattern of results.

Download Full-text

Configural Determinants of Visuospatial Serial Memory Performance

PsycEXTRA Dataset ◽

10.1037/e537052012-048 ◽

2004 ◽

Author(s):

Fabrice B. R. Parmentier ◽

Greg Elford ◽

Dylan M. Jones

Keyword(s):

Memory Performance ◽

Serial Memory

Download Full-text

Target Detection in a Scenario of Natural Sounds: The Role of Spatial Sound Source Separation

PsycEXTRA Dataset ◽

10.1037/e633262013-526 ◽

2013 ◽

Author(s):

Susanne Mayr ◽

Gunnar Regenbrecht ◽

Kathrin Lange ◽

Albertgeorg Lang ◽

Axel Buchner

Keyword(s):

Target Detection ◽

Sound Source ◽

Source Separation ◽

Natural Sounds ◽

Sound Source Separation ◽

Spatial Sound

Download Full-text

Foregrounding Expectation: Spatial Sound-image Composition in Virtual Environments

The New Soundtrack ◽

10.3366/sound.2017.0102 ◽

2017 ◽

Vol 7 (2) ◽

pp. 95-110

Author(s):

Angela McArthur ◽

Stefano Kalonaris

Keyword(s):

Virtual Environments ◽

Image Composition ◽

Sound Image ◽

Spatial Sound

Download Full-text

Alpha EEG Frontal Asymmetries during Audiovisual Perception in Cochlear Implant Users

Methods of Information in Medicine ◽

10.3414/me15-01-0005 ◽

2015 ◽

Vol 54 (06) ◽

pp. 500-504 ◽

Cited By ~ 9

Author(s):

A. G. Maglione ◽

A. Scorpecci ◽

P. Malerba ◽

P. Marsella ◽

S. Giannantonio ◽

...

Keyword(s):

Cochlear Implant ◽

Alpha Rhythm ◽

Normal Hearing ◽

Experimental Conditions ◽

Audiovisual Perception ◽

Audiovisual Stimuli ◽

The Difference ◽

Eeg Alpha ◽

The One ◽

Hearing Children

SummaryObjectives: The aim of the present study is to investigate the variations of the electroencephalographic (EEG) alpha rhythm in order to measure the appreciation of bilateral and unilateral young cochlear implant users during the observation of a musical cartoon. The cartoon has been modified for the generation of three experimental conditions: one with the original audio, another one with a distorted sound and, finally, a mute version.Methods: The EEG data have been recorded during the observation of the cartoons in the three experimental conditions. The frontal alpha EEG imbalance has been calculated as a measure of motivation and pleasantness to be compared across experimental populations and conditions.Results: The EEG frontal imbalance of the alpha rhythm showed significant variations during the perception of the different cartoons. In particular, the pattern of activation of normal-hearing children is very similar to the one elicited by the bilateral implanted patients. On the other hand, results related to the unilateral subjects do not present significant variations of the imbalance index across the three cartoons.Conclusion: The presented results suggest that the unilateral patients could not appreciate the difference in the audio format as well as bilaterally implanted and normal hearing subjects. The frontal alpha EEG imbalance is a useful tool to detect the differences in the appreciation of audiovisual stimuli in cochlear implant patients.

Download Full-text

Visual Anticipatory Information Modulates Multisensory Interactions of Artificial Audiovisual Stimuli

Journal of Cognitive Neuroscience ◽

10.1162/jocn.2009.21308 ◽

2010 ◽

Vol 22 (7) ◽

pp. 1583-1596 ◽

Cited By ~ 87

Author(s):

Jean Vroomen ◽

Jeroen J. Stekelenburg

Keyword(s):

Visual Information ◽

Speech Sound ◽

Human Action ◽

Visual Signal ◽

Multisensory Interactions ◽

Temporal Occurrence ◽

Audiovisual Stimuli ◽

The Moment ◽

Integrative Processing ◽

Audiovisual Interaction

The neural activity of speech sound processing (the N1 component of the auditory ERP) can be suppressed if a speech sound is accompanied by concordant lip movements. Here we demonstrate that this audiovisual interaction is neither speech specific nor linked to humanlike actions but can be observed with artificial stimuli if their timing is made predictable. In Experiment 1, a pure tone synchronized with a deformation of a rectangle induced a smaller auditory N1 than auditory-only presentations if the temporal occurrence of this audiovisual event was made predictable by two moving disks that touched the rectangle. Local autoregressive average source estimation indicated that this audiovisual interaction may be related to integrative processing in auditory areas. When the moving disks did not precede the audiovisual stimulus—making the onset unpredictable—there was no N1 reduction. In Experiment 2, the predictability of the leading visual signal was manipulated by introducing a temporal asynchrony between the audiovisual event and the collision of moving disks. Audiovisual events occurred either at the moment, before (too “early”), or after (too “late”) the disks collided on the rectangle. When asynchronies varied from trial to trial—rendering the moving disks unreliable temporal predictors of the audiovisual event—the N1 reduction was abolished. These results demonstrate that the N1 suppression is induced by visual information that both precedes and reliably predicts audiovisual onset, without a necessary link to human action-related neural mechanisms.

Download Full-text