Effects of Spatial Congruity on Audio-Visual Multimodal Integration

2005 ◽  
Vol 17 (9) ◽  
pp. 1396-1409 ◽  
Author(s):  
W. A. Teder-Sälejärvi ◽  
F. Di Russo ◽  
J. J. McDonald ◽  
S. A. Hillyard
Keyword(s):  
Multisensory Integration ◽  
Amplitude Modulation ◽  
Animal Studies ◽  
Temporal Cortex ◽  
Event Related Potentials ◽  
Temporal Region ◽  
Modal Interactions ◽  
Neural Basis ◽  
Related Potentials ◽  
Evoked Activity

Spatial constraints on multisensory integration of auditory (A) and visual (V) stimuli were investigated in humans using behavioral and electrophysiological measures. The aim was to find out whether cross-modal interactions between A and V stimuli depend on their spatial congruity, as has been found for multisensory neurons in animal studies (Stein & Meredith, 1993). Randomized sequences of unimodal (A or V) and simultaneous bimodal (AV) stimuli were presented to right-or left-field locations while subjects made speeded responses to infrequent targets of greater intensity that occurred in either or both modalities. Behavioral responses to the bimodal stimuli were faster and more accurate than to the uni-modal stimuli for both same-location and different-location AV pairings. The neural basis of this cross-modal facilitation was studied by comparing event-related potentials (ERPs) to the bimodal AV stimuli with the summed ERPs to the unimodal A and V stimuli. These comparisons revealed neural interactions localized to the ventral occipito-temporal cortex (at 190 msec) and to the superior temporal cortical areas (at 260 msec) for both same-and different-location AV pairings. In contrast, ERP interactions that differed according to spatial congruity included a phase and amplitude modulation of visual-evoked activity localized to the ventral occipito-temporal cortex at 100-400 msec and an amplitude modulation of activity localized to the superior temporal region at 260-280 msec. These results demonstrate overlapping but distinctive patterns of multisensory integration for spatially congruent and incongruent AV stimuli.

scholarly journals Neural Basis of Anticipatory Multisensory Integration

2021 ◽  
Vol 11 (7) ◽  
pp. 843
Author(s):  
Linda Fiorini ◽  
Marika Berchicci ◽  
Elena Mussini ◽  
Valentina Bianco ◽  
Stefania Lucia ◽  
...  
Keyword(s):  
Multisensory Integration ◽  
Sensory Processing ◽  
Sensory Information ◽  
Event Related Potentials ◽  
Event Perception ◽  
Neural Basis ◽  
Stimulus Processing ◽  
Related Potentials ◽  
Salient Event ◽  
The Brain

The brain is able to gather different sensory information to enhance salient event perception, thus yielding a unified perceptual experience of multisensory events. Multisensory integration has been widely studied, and the literature supports the hypothesis that it can occur across various stages of stimulus processing, including both bottom-up and top-down control. However, evidence on anticipatory multisensory integration occurring in the fore period preceding the presentation of the expected stimulus in passive tasks, is missing. By means of event-related potentials (ERPs), it has been recently proposed that visual and auditory unimodal stimulations are preceded by sensory-specific readiness activities. Accordingly, in the present study, we tested the occurrence of multisensory integration in the endogenous anticipatory phase of sensory processing, combining visual and auditory stimuli during unimodal and multimodal passive ERP paradigms. Results showed that the modality-specific pre-stimulus ERP components (i.e., the auditory positivity -aP- and the visual negativity -vN-) started earlier and were larger in the multimodal stimulation compared with the sum of the ERPs elicited by the unimodal stimulations. The same amplitude effect was also present for the early auditory N1 and visual P1 components. This anticipatory multisensory effect seems to influence stimulus processing, boosting the magnitude of early stimulus processing. This paves the way for new perspectives on the neural basis of multisensory integration.

Abnormal Neural Basis of Emotional Conflict Control in Treatment-Resistant Depression

2016 ◽  
Vol 48 (2) ◽  
pp. 103-110 ◽  
Author(s):  
Song Xue ◽  
Shanshan Wang ◽  
Xia Kong ◽  
Jiang Qiu
Keyword(s):  
Stroop Task ◽  
Event Related Potentials ◽  
Frontal Region ◽  
Healthy Controls ◽  
Treatment Resistant Depression ◽  
Treatment Resistant ◽  
Neural Basis ◽  
Emotional Conflict ◽  
Related Potentials ◽  
Resistant Depression

Emotional conflict has received increased attention as a research topic. The objective of this study is to confirm that the processing of emotional conflict is impaired in treatment-resistant depression (TRD). We compared the event-related potentials of 17 patients with TRD and 17 healthy controls during the face-word Stroop task, which is an effective way of assessing the effects of emotional conflict directly. Compared with healthy controls, the accuracy scores of the TRD patients were lower in both “congruent stimuli” and “incongruent stimuli” conditions, and their response times were longer. The TRD patients also had larger N2 amplitudes over the frontal region, regardless of stimulus condition, which might reflect that TRD patients pay more attention to emotional information. A larger P3 amplitude over the frontal region for “incongruent stimuli minus congruent stimuli” was also found among patients with TRD, which indicates interference effects in the Stroop task. The results of this study provide novel behavioral and neurophysiological evidence of anomalies in cognitive inhibition among patients with TRD using the word-face task. These findings not only improve our understanding of deficient inhibition in TRD, but also pave the way for a cognitive neuropsychiatric model of depression.

scholarly journals The Influence of Subclinical Neck Pain on Neurophysiological and Behavioral Measures of Multisensory Integration

2019 ◽  
Vol 9 (12) ◽  
pp. 362
Author(s):  
Antonia M. Karellas ◽  
Paul Yielder ◽  
James J. Burkitt ◽  
Heather S. McCracken ◽  
Bernadette A. Murphy
Keyword(s):  
Reaction Time ◽  
Neck Pain ◽  
Multisensory Integration ◽  
Performance Measures ◽  
Brain Activity ◽  
Event Related Potentials ◽  
Control Group ◽  
Neural Processing ◽  
Significant Group ◽  
Related Potentials

Multisensory integration (MSI) is necessary for the efficient execution of many everyday tasks. Alterations in sensorimotor integration (SMI) have been observed in individuals with subclinical neck pain (SCNP). Altered audiovisual MSI has previously been demonstrated in this population using performance measures, such as reaction time. However, neurophysiological techniques have not been combined with performance measures in the SCNP population to determine differences in neural processing that may contribute to these behavioral characteristics. Electroencephalography (EEG) event-related potentials (ERPs) have been successfully used in recent MSI studies to show differences in neural processing between different clinical populations. This study combined behavioral and ERP measures to characterize MSI differences between healthy and SCNP groups. EEG was recorded as 24 participants performed 8 blocks of a simple reaction time (RT) MSI task, with each block consisting of 34 auditory (A), visual (V), and audiovisual (AV) trials. Participants responded to the stimuli by pressing a response key. Both groups responded fastest to the AV condition. The healthy group demonstrated significantly faster RTs for the AV and V conditions. There were significant group differences in neural activity from 100–140 ms post-stimulus onset, with the control group demonstrating greater MSI. Differences in brain activity and RT between individuals with SCNP and a control group indicate neurophysiological alterations in how individuals with SCNP process audiovisual stimuli. This suggests that SCNP alters MSI. This study presents novel EEG findings that demonstrate MSI differences in a group of individuals with SCNP.

Visual Gnosis and Face Perception

2012 ◽  
Vol 3 (4) ◽  
pp. 11-20
Author(s):  
Shozo Tobimatsu
Keyword(s):  
Spatial Frequency ◽  
Facial Expressions ◽  
Face Perception ◽  
Time Windows ◽  
High Spatial Frequency ◽  
Early Time ◽  
Event Related Potentials ◽  
Temporal Region ◽  
The Face ◽  
Related Potentials

There are two major parallel pathways in humans: the parvocellular (P) and magnocellular (M) pathways. The former has excellent spatial resolution with color selectivity, while the latter shows excellent temporal resolution with high contrast sensitivity. Visual stimuli should be tailored to answer specific clinical and/or research questions. This chapter examines the neural mechanisms of face perception using event-related potentials (ERPs). Face stimuli of different spatial frequencies were used to investigate how low-spatial-frequency (LSF) and high-spatial-frequency (HSF) components of the face contribute to the identification and recognition of the face and facial expressions. The P100 component in the occipital area (Oz), the N170 in the posterior temporal region (T5/T6) and late components peaking at 270-390 ms (T5/T6) were analyzed. LSF enhanced P100, while N170 was augmented by HSF irrespective of facial expressions. This suggested that LSF is important for global processing of facial expressions, whereas HSF handles featural processing. There were significant amplitude differences between positive and negative LSF facial expressions in the early time windows of 270-310 ms. Subsequently, the amplitudes among negative HSF facial expressions differed significantly in the later time windows of 330–390 ms. Discrimination between positive and negative facial expressions precedes discrimination among different negative expressions in a sequential manner based on parallel visual channels. Interestingly, patients with schizophrenia showed decreased spatial frequency sensitivities for face processing. Taken together, the spatially filtered face images are useful for exploring face perception and recognition.

scholarly journals Neurobiological Effects of Binge Drinking Help in Its Detection and Differential Diagnosis from Alcohol Dependence

2018 ◽  
Vol 2018 ◽  
pp. 1-9 ◽  
Author(s):  
Napoleon Waszkiewicz ◽  
Beata Galińska-Skok ◽  
Anastasiya Nestsiarovich ◽  
Agnieszka Kułak-Bejda ◽  
Karolina Wilczyńska ◽  
...  
Keyword(s):  
Alcohol Dependence ◽  
Binge Drinking ◽  
Frontal Lobe ◽  
Inhibitory Control ◽  
Animal Studies ◽  
Event Related Potentials ◽  
Limbic Cortex ◽  
Parietal Lobes ◽  
Related Potentials ◽  
Binge Drinkers

The prevalence of binge drinking in the general population is 3-4 times higher than that of alcohol dependence. Neuroimaging studies show that binge drinking in adolescence impairs brain development and white matter integrity. Regions with reduced functional activity include the limbic system, ventral diencephalon, frontal lobe, and middle and inferior temporal lobes, whereas the right superior frontal and parietal lobes are typically hyperactivated. The observed activation of the frontoparietal areas might reflect the alternative memory system operating, whereas the reduced occipito-hippocampal response is associated with impaired visual and linguistic processing/learning. Some other findings from literature research include a decrease of N-acetylaspartate (NAA) in the frontal lobe and its increase in the parietal lobes, as well as the reduced components of event-related potentials, reflecting deficit in attention, working memory, inhibition, and executive functioning. Animal studies show that even a single day of binge drinking results in a neurodegeneration and reactive gliosis in the limbic cortex as well as in gene expression dysregulation and histone acetylation. Another biological evidence on binge drinking effect include inflammatory response, oxidative stress, formation of toxic ceramides, activation of caspase 3, and secretion of corticoliberin. Some of the binge drinking-induced cognitive abnormalities can be reversible after three weeks of abstinence. Although binge drinkers have a similar pattern of neuropsychological deficits with chronic alcohol consumers (mainly memory deficits), binge drinkers have prominent impairment of inhibitory control, which may be a marker of binge pattern of alcohol drinking. The optimal therapeutic strategies should target the inhibitory control processes to facilitate discontinuation of alcohol consumption and to block its possible progression to the alcohol dependence syndrome.

scholarly journals Seeing and Hearing Meaning: ERP and fMRI Evidence of Word versus Picture Integration into a Sentence Context

2008 ◽  
Vol 20 (7) ◽  
pp. 1235-1249 ◽  
Author(s):  
Roel M. Willems ◽  
Aslı Özyürek ◽  
Peter Hagoort
Keyword(s):  
Visual Information ◽  
Time Course ◽  
Semantic Information ◽  
Temporal Cortex ◽  
Event Related Potentials ◽  
Semantic Integration ◽  
Sentence Context ◽  
N400 Effect ◽  
Related Potentials ◽  
Pictures And Words

Understanding language always occurs within a situational context and, therefore, often implies combining streams of information from different domains and modalities. One such combination is that of spoken language and visual information, which are perceived together in a variety of ways during everyday communication. Here we investigate whether and how words and pictures differ in terms of their neural correlates when they are integrated into a previously built-up sentence context. This is assessed in two experiments looking at the time course (measuring event-related potentials, ERPs) and the locus (using functional magnetic resonance imaging, fMRI) of this integration process. We manipulated the ease of semantic integration of word and/or picture to a previous sentence context to increase the semantic load of processing. In the ERP study, an increased semantic load led to an N400 effect which was similar for pictures and words in terms of latency and amplitude. In the fMRI study, we found overlapping activations to both picture and word integration in the left inferior frontal cortex. Specific activations for the integration of a word were observed in the left superior temporal cortex. We conclude that despite obvious differences in representational format, semantic information coming from pictures and words is integrated into a sentence context in similar ways in the brain. This study adds to the growing insight that the language system incorporates (semantic) information coming from linguistic and extralinguistic domains with the same neural time course and by recruitment of overlapping brain areas.

scholarly journals Aversive conditioning in oddball paradigm modulates multisensory integration, attention and emotional processing

2018 ◽  
Author(s):  
Yuri G. Pavlov ◽  
Boris Kotchoubey
Keyword(s):  
Associative Learning ◽  
Multisensory Integration ◽  
Emotional Processing ◽  
Aversive Conditioning ◽  
Event Related Potentials ◽  
Oddball Paradigm ◽  
Beta Activity ◽  
Time Frequency ◽  
Electrical Shocks ◽  
Related Potentials

AbstractThe nature of cortical plasticity in the course of learning is one of the most intriguing questions of the modern cognitive neuroscience. Aversive conditioning is a type of associative learning produced by continuous pairing of neutral and aversive stimuli. Aversive conditioning and electroencephalography together provide a good framework for expanding our knowledge about fast learning-related cortical changes. In our experiment we tested a novel paradigm to study associative learning where aversive conditioning was combined with passive oddball. We employed conditioned auditory neutral stimuli and unconditioned aversive electrical shocks and used time-frequency, connectivity and event-related potentials (ERP) analyses to explore their interaction. First, we observed changes in the cortical activity in the form of conditioning-induced multisensory integration. The integration manifested itself in (1) desynchronization of lower beta activity in the contralateral to expected electrical shocks hemisphere and (2) enhanced functional connectivity between auditory and somatosensory cortex in the gamma frequency band. Second, we found a larger amplitude of P3a and the late posterior positivity (LPP) components of ERP to conditioned stimuli, which may be related to increased attentional and emotional significance of these stimuli. Our results reproduced and extended previous findings about multisensory integration in classical conditioning and demonstrated the improved discriminability of ERP responses through incorporation of the oddball paradigm in associative learning.

Neural Basis of Semantically Dependent and Independent Cross-Modal Boosts on the Attentional Blink

2020 ◽  
Author(s):  
Song Zhao ◽  
Chengzhi Feng ◽  
Xinyin Huang ◽  
Yijun Wang ◽  
Wenfeng Feng
Keyword(s):  
Object Recognition ◽  
Attentional Blink ◽  
Temporal Dynamics ◽  
Recognition Task ◽  
Event Related Potentials ◽  
Visual Object ◽  
Visual Object Recognition ◽  
Neural Basis ◽  
Related Potentials ◽  
The Cross

Abstract The present study recorded event-related potentials (ERPs) in a visual object-recognition task under the attentional blink paradigm to explore the temporal dynamics of the cross-modal boost on attentional blink and whether this auditory benefit would be modulated by semantic congruency between T2 and the simultaneous sound. Behaviorally, the present study showed that not only a semantically congruent but also a semantically incongruent sound improved T2 discrimination during the attentional blink interval, whereas the enhancement was larger for the congruent sound. The ERP results revealed that the behavioral improvements induced by both the semantically congruent and incongruent sounds were closely associated with an early cross-modal interaction on the occipital N195 (192–228 ms). In contrast, the lower T2 accuracy for the incongruent than congruent condition was accompanied by a larger late occurring cento-parietal N440 (424–448 ms). These findings suggest that the cross-modal boost on attentional blink is hierarchical: the task-irrelevant but simultaneous sound, irrespective of its semantic relevance, firstly enables T2 to escape the attentional blink via cross-modally strengthening the early stage of visual object-recognition processing, whereas the semantic conflict of the sound begins to interfere with visual awareness only at a later stage when the representation of visual object is extracted.

N200 in the Eriksen-Task: Inhibitory Executive Processes?

2000 ◽  
Vol 14 (4) ◽  
pp. 218-225 ◽  
Author(s):  
Martin Heil ◽  
Allen Osman ◽  
Juliane Wiegelmann ◽  
Bettina Rolke ◽  
Erwin Hennighausen
Keyword(s):  
Amplitude Modulation ◽  
Flanker Task ◽  
Event Related Potentials ◽  
Neural Correlates ◽  
Executive Processes ◽  
Response Priming ◽  
Left Hand ◽  
Readiness Potentials ◽  
Related Potentials ◽  
Abstract Event

Abstract Event-related potentials were recorded (N = 18) in a hybrid go/no-go Eriksen flanker task to study the neural correlates of response inhibition. Three letters were assigned to either a left-hand, a right-hand, or a no-go response. These three letters appeared either as targets signaling the assigned response or as flankers surrounding the target. The lateralized readiness potentials revealed erroneous cortical response priming on go trials, in which the target and flankers were assigned to different hands, as well as on no-go trials, in which the flankers primed one of the two hands. Exactly these two conditions were accompanied by a fronto-central amplitude modulation of the N200, suggesting that this ERP component may reflect inhibitory executive functions. The data replicate and extend recent studies by Kopp, Rist, and Mattler (1996) and Kopp, Mattler, Goertz, and Rist (1996) .

scholarly journals Neural Basis of Intrinsic Motivation: Evidence from Event-Related Potentials

2015 ◽  
Vol 2015 ◽  
pp. 1-6 ◽  
Author(s):  
Jia Jin ◽  
Liping Yu ◽  
Qingguo Ma
Keyword(s):  
Intrinsic Motivation ◽  
Social Factor ◽  
Event Related Potentials ◽  
P300 Amplitude ◽  
Neural Basis ◽  
Outcome Feedback ◽  
Quantitative Measurements ◽  
Machine Learning Model ◽  
Stop Watch ◽  
Related Potentials

Human intrinsic motivation is of great importance in human behavior. However, although researchers have focused on this topic for decades, its neural basis was still unclear. The current study employed event-related potentials to investigate the neural disparity between an interesting stop-watch (SW) task and a boring watch-stop task (WS) to understand the neural mechanisms of intrinsic motivation. Our data showed that, in the cue priming stage, the cue of the SW task elicited smaller N2 amplitude than that of the WS task. Furthermore, in the outcome feedback stage, the outcome of the SW task induced smaller FRN amplitude and larger P300 amplitude than that of the WS task. These results suggested that human intrinsic motivation did exist and that it can be detected at the neural level. Furthermore, intrinsic motivation could be quantitatively indexed by the amplitude of ERP components, such as N2, FRN, and P300, in the cue priming stage or feedback stage. Quantitative measurements would also be convenient for intrinsic motivation to be added as a candidate social factor in the construction of a machine learning model.

Sign in / Sign up

Export Citation Format

Share Document