Audiovisual Events in Sensory Memory

The functional properties of the auditory sensory memory have been extensively studied using the mismatch negativity (MMN) component of the auditory event-related potential (ERP) and its magnetic counterpart recorded using magneto-encephalography (MEG). It has been found that distinct auditory features (such as frequency or intensity) are encoded separately in sensory memory. Nevertheless, the conjunction of these features (auditory “gestalts”) can also be encoded in auditory sensory memory. Here we investigated how auditory and visual features of bimodal events are represented in sensory memory by recording audiovisual MMNs in two different audiovisual oddball paradigms. The results of a first ERP experiment showed that the sensory memory representations of auditory and visual features of audiovisual events lie within the temporal and occipital cortex, respectively, yet with possible interactions between the processing of the unimodal features. In a subsequent MEG experiment, we found some evidence that audiovisual feature conjunctions could also be represented in sensory memory. These results, thus, extend to the audiovisual domain a number of properties of sensory memory already established within the auditory system.

Download Full-text

Separate Neural Processing of Timbre Dimensions in Auditory Sensory Memory

Journal of Cognitive Neuroscience ◽

10.1162/jocn.2006.18.12.1959 ◽

2006 ◽

Vol 18 (12) ◽

pp. 1959-1972 ◽

Cited By ~ 65

Author(s):

Anne Caclin ◽

Elvira Brattico ◽

Mari Tervaniemi ◽

Risto Näätänen ◽

Dominique Morlet ◽

...

Keyword(s):

Mismatch Negativity ◽

Event Related Potential ◽

Sensory Memory ◽

Sound Sources ◽

Auditory Event ◽

Auditory Sensory Memory ◽

Complex Tones ◽

Dipole Modeling ◽

Sound Sequence ◽

Perceptual Attribute

Timbre is a multidimensional perceptual attribute of complex tones that characterizes the identity of a sound source. Our study explores the representation in auditory sensory memory of three timbre dimensions (acoustically related to attack time, spectral centroid, and spectrum fine structure), using the mismatch negativity (MMN) component of the auditory event-related potential. MMN is elicited by a discriminable change in a sound sequence and reflects the detection of the discrepancy between the current stimulus and traces in auditory sensory memory. The stimuli used in the present study were carefully controlled synthetic tones. MMNs were recorded after changes along each of the three timbre dimensions and their combinations. Additivity of unidimensional MMNs and dipole modeling results suggest partially separate MMN generators for different timbre dimensions, reflecting their mainly separate processing in auditory sensory memory. The results expand to timbre dimensions a property of separation of the representation in sensory memory that has already been reported between basic perceptual attributes (pitch, loudness, duration, and location) of sound sources.

Download Full-text

Familiarity Affects the Processing of Task-irrelevant Auditory Deviance

Journal of Cognitive Neuroscience ◽

10.1162/089892905774589262 ◽

2005 ◽

Vol 17 (11) ◽

pp. 1704-1713 ◽

Cited By ~ 49

Author(s):

Thomas Jacobsen ◽

Erich Schröger ◽

István Winkler ◽

János Horváth

Keyword(s):

Change Detection ◽

Mismatch Negativity ◽

Sensory Memory ◽

Auditory Stimuli ◽

Brain Potential ◽

Detection Process ◽

Auditory Sensory Memory ◽

Memory Representations ◽

Auditory Change Detection ◽

Task Irrelevant

The effects of familiarity on auditory change detection on the basis of auditory sensory memory representations were investigated by presenting oddball sequences of sounds while participants ignored the auditory stimuli. Stimulus sequences were composed of sounds that were familiar and sounds that were made unfamiliar by playing the same sounds backward. The roles of frequently presented stimuli (standards) and infrequently presented ones (deviants) were fully crossed. Deviants elicited the mismatch negativity component of the event-related brain potential. We found an enhancement in detecting changes when deviant sounds appeared among familiar standard sounds compared when they were delivered among unfamiliar standards. Familiarity with the deviant sounds also enhanced the change-detection process. We suggest that tuning to familiar items sets up preparatory processes that affect change detection in familiar sound sequences.

Download Full-text