scholarly journals Exploiting Electrophysiological Measures of Semantic Processing for Auditory Attention Decoding

2020 ◽  
Author(s):  
Karen Dijkstra ◽  
Peter Desain ◽  
Jason Farquhar
Keyword(s):  
Semantic Processing ◽  
Content Word ◽  
Auditory Attention ◽  
Added Value ◽  
Acoustic Features ◽  
Semantic Description ◽  
Brain Responses ◽  
Word Onset ◽  
Analyse Data ◽  
Current Word

ABSTRACTIn Auditory Attention Decoding, a user’s electrophysiological brain responses to certain features of speech are modelled and subsequently used to distinguish attended from unattended speech in multi-speaker contexts. Such approaches are frequently based on acoustic features of speech, such as the auditory envelope. A recent paper shows that the brain’s response to a semantic description (i.e., semantic dissimilarity) of narrative speech can also be modelled using such an approach. Here we use the (publicly available) data accompanying that study, in order to investigate whether combining this semantic dissimilarity feature with an auditory envelope approach improves decoding performance over using the envelope alone. We analyse data from their ‘Cocktail Party’ experiment in which 33 subjects attended to one of two simultaneously presented audiobook narrations, for 30 1-minute fragments. We find that the addition of the dissimilarity feature to an envelope-based approach significantly increases accuracy, though the increase is marginal (85.4% to 86.6%). However, we subsequently show that this dissimilarity feature, in which the degree of dissimilarity of the current word with regard to the previous context is tagged to the onsets of each content word, can be replaced with a binary content-word-onset feature, without significantly affecting the results (i.e., modelled responses or accuracy), putting in question the added value of the dissimilarity information for the approach introduced in this recent paper.

The time-course of morphosyntactic and semantic priming in late bilinguals: A study of German adjectives

2016 ◽  
Vol 20 (3) ◽  
pp. 435-456 ◽  
Author(s):  
SINA BOSCH ◽  
HELENA KRAUSE ◽  
ALINA LEMINEN
Keyword(s):  
Semantic Processing ◽  
Time Course ◽  
Semantic Information ◽  
Temporal Dynamics ◽  
Native Speaker ◽  
Behavioral Experiment ◽  
Lexical Semantic ◽  
Brain Responses ◽  
L2 Learners ◽  
Spatially Extended

How do late proficient bilinguals process morphosyntactic and lexical-semantic information in their non-native language (L2)? How is this information represented in the L2 mental lexicon? And what are the neural signatures of L2 morphosyntactic and lexical-semantic processing? We addressed these questions in one behavioral and two ERP priming experiments on inflected German adjectives testing a group of advanced late Russian learners of German in comparison to native speaker (L1) controls. While in the behavioral experiment, the L2 learners performed native-like, the ERP data revealed clear L1/L2 differences with respect to the temporal dynamics of grammatical processing. Specifically, our results show that L2 morphosyntactic processing yielded temporally and spatially extended brain responses relative to L1 processing, indicating that grammatical processing of inflected words in an L2 is more demanding and less automatic than in the L1. However, this group of advanced L2 learners showed native-like lexical-semantic processing.

scholarly journals Localization of Syntactic and Semantic Brain Responses using Magnetoencephalography

2007 ◽  
Vol 19 (7) ◽  
pp. 1193-1205 ◽  
Author(s):  
Elisabet Service ◽  
Päivi Helenius ◽  
Sini Maury ◽  
Riitta Salmelin
Keyword(s):  
Temporal Lobe ◽  
Semantic Processing ◽  
Sentence Comprehension ◽  
Right Hemisphere ◽  
Temporal Cortex ◽  
Word Class ◽  
Sentence Reading ◽  
Word Onset ◽  
Semantic Errors ◽  
The Right

Electrophysiological methods have been used to study the temporal sequence of syntactic and semantic processing during sentence comprehension. Two responses associated with syntactic violations are the left anterior negativity (LAN) and the P600. A response to semantic violation is the N400. Although the sources of the N400 response have been identified in the left (and right) temporal lobe, the neural signatures of the LAN and P600 have not been revealed. The present study used magnetoencephalography to localize sources of syntactic and semantic activation in Finnish sentence reading. Participants were presented with sentences that ended in normally inf lected nouns, nouns in an unacceptable case, verbs instead of nouns, or nouns that were correctly inflected but made no sense in the context. Around 400 msec, semantically anomalous last words evoked strong activation in the left superior temporal lobe with significant activation also for word class errors (N400). Weaker activation was seen for the semantic errors in the right hemisphere. Later, 600-800 msec after word onset, the strongest activation was seen to word class and morphosyntactic errors (P600). Activation was significantly weaker to semantically anomalous and correct words. The P600 syntactic activation was localized to bilateral sources in the temporal lobe, posterior to the N400 sources. The results suggest that the same general region of the superior temporal cortex gives rise to both LAN and N400 with bilateral reactivity to semantic manipulation and a left hemisphere effect to syntactic manipulation. The bilateral P600 response was sensitive to syntactic but not semantic factors.

scholarly journals Semantic Processing in Deaf and Hard-of-Hearing Children: Large N400 Mismatch Effects in Brain Responses, Despite Poor Semantic Ability

2016 ◽  
Vol 7 ◽  
Author(s):  
Petter Kallioinen ◽  
Jonas Olofsson ◽  
Cecilia Nakeva von Mentzer ◽  
Magnus Lindgren ◽  
Marianne Ors ◽  
...  
Keyword(s):  
Hard Of Hearing ◽  
Semantic Processing ◽  
Brain Responses ◽  
Hearing Children

The Processing of Printed Language by Aphasic Adults

1982 ◽  
Vol 25 (2) ◽  
pp. 314-319 ◽  
Author(s):  
John L. Locke ◽  
John W. Deck
Keyword(s):  
Word Recognition ◽  
Oral Reading ◽  
Semantic Processing ◽  
Semantic Analysis ◽  
Content Word ◽  
Function Word ◽  
Phonological Recoding ◽  
Reading Theory ◽  
The Face

Eight aphasic and eight brain-damaged nonaphasie patients silently "read" a short passage while performing an internal search for specified consonant letters of varying phonological and syntactic salience. The nonaphasie patients showed the phonological and syntactic effects customarily achieved by normal readers. For example, they were more likely to find a letter if it were pronounced than if it were silent, and they were more likely to find a letter if it were in a content word than in a function word. The aphasics had reliable phonological effects hut no observable syntactic effects. Those aphasics with relatively large phonological effects performed better on a separate task requiring the oral reading of isolated words. For reading theory, the primary message from this study is that phonological recoding may occur between word recognition and the completion of semantic analysis, and that recoding may not by itself be sufficient to reading for meaning. For aphasia theory, the main implication of this study is that aphasics read by applying the appropriate phonological strategies, but that such strategies are limited in the face of ineffective syntactic and semantic processing, as occurs in aphasia.

scholarly journals Neural tracking to go: auditory attention decoding and saliency detection with mobile EEG

2021 ◽  
Author(s):  
Lisa Straetmans ◽  
B. Holtze ◽  
Stefan Debener ◽  
Manuela Jaeger ◽  
Bojana Mirkovic
Keyword(s):  
Hearing Aids ◽  
Saliency Detection ◽  
Assistive Technologies ◽  
Auditory Attention ◽  
Acoustic Features ◽  
Auditory Scene ◽  
Neural Information ◽  
Natural Movement ◽  
Salient Event ◽  
First Time

Abstract Objective. Neuro-steered assistive technologies have been suggested to offer a major advancement in future devices like neuro-steered hearing aids. Auditory attention decoding methods would in that case allow for identification of an attended speaker within complex auditory environments, exclusively from neural data. Decoding the attended speaker using neural information has so far only been done in controlled laboratory settings. Yet, it is known that ever-present factors like distraction and movement are reflected in the neural signal parameters related to attention. Approach. Thus, in the current study we applied a two-competing speaker paradigm to investigate performance of a commonly applied EEG-based auditory attention decoding (AAD) model outside of the laboratory during leisure walking and distraction. Unique environmental sounds were added to the auditory scene and served as distractor events. Main results. The current study shows, for the first time, that the attended speaker can be accurately decoded during natural movement. At a temporal resolution of as short as 5-seconds and without artifact attenuation, decoding was found to be significantly above chance level. Further, as hypothesized, we found a decrease in attention to the to-be-attended and the to-be-ignored speech stream after the occurrence of a salient event. Additionally, we demonstrate that it is possible to predict neural correlates of distraction with a computational model of auditory saliency based on acoustic features. Conclusion. Taken together, our study shows that auditory attention tracking outside of the laboratory in ecologically valid conditions is feasible and a step towards the development of future neural-steered hearing aids.

scholarly journals Target of selective auditory attention can be robustly followed with MEG

2019 ◽  
Author(s):  
Dovilė Kurmanavičiūtė ◽  
Antti Rantala ◽  
Mainak Jas ◽  
Anne Välilä ◽  
Lauri Parkkonen
Keyword(s):  
Stimulus Onset ◽  
Auditory Attention ◽  
Support Vector ◽  
Time Resolved ◽  
Auditory Evoked Responses ◽  
Spatially Resolved ◽  
Auditory Responses ◽  
Brain Responses ◽  
Source Level ◽  
The Right

AbstractSelective auditory attention enables filtering relevant from irrelevant acoustic information. Specific auditory responses, measurable by electro- and magnetoencephalography (EEG/MEG), are known to be modulated by attention to the evoking stimuli. However, these attention effects are typically demonstrated in averaged responses and their robustness in single trials is not studied extensively.We applied decoding algorithms to MEG to investigate how well the target of auditory attention could be determined from single responses and which spatial and temporal aspects of the responses carry most of the information regarding the target of attention. To this end, we recorded brain responses of 15 healthy subjects with MEG when they selectively attended to one of the simultaneously presented auditory streams of words “Yes” and “No”. A support vector machine was trained on the MEG data both at the sensor and source level to predict at every trial which stream was attended.Sensor-level decoding of the attended stream using the entire 2-s epoch resulted in a mean accuracy of 93%±1% (range 83–99% across subjects). Time-resolved decoding revealed that the highest accuracies were obtained 200–350 ms after the stimulus onset. Spatially-resolved source-level decoding indicated that the cortical sources most informative of the attended stream were located primarily in the auditory cortex, especially in the right hemi-sphere.Our result corroborates attentional modulation of auditory evoked responses also to naturalistic stimuli. The achieved high decoding accuracy could enable the use of our experimental paradigm and classification method in a brain–computer interface.

scholarly journals Phonotactic Knowledge and Lexical-Semantic Processing in One-year-olds: Brain Responses to Words and Nonsense Words in Picture Contexts

2005 ◽  
Vol 17 (11) ◽  
pp. 1785-1802 ◽  
Author(s):  
Manuela Friedrich ◽  
Angela D. Friederici
Keyword(s):  
Semantic Processing ◽  
Semantic Integration ◽  
Control Group ◽  
First Year ◽  
Brain Potentials ◽  
Lexical Priming ◽  
Lexical Semantic ◽  
Brain Responses ◽  
One Year ◽  
First Year Of Life

During their first year of life, infants not only acquire probabilistic knowledge about the phonetic, prosodic, and phonotactic organization of their native language, but also begin to establish first lexical-semantic representations. The present study investigated the sensitivity to phonotactic regularities and its impact on semantic processing in 1-year-olds. We applied the method of event-related brain potentials to 12-and 19-month-old children and to an adult control group. While looking at pictures of known objects, subjects listened to spoken nonsense words that were phonotactically legal (pseudowords) or had phonotactically illegal word onsets (nonwords), or to real words that were either congruous or incongruous to the picture contents. In 19-month-olds and in adults, incongruous words and pseudowords, but not non-words, elicited an N400 known to reflect mechanisms of semantic integration. For congruous words, the N400 was attenuated by semantic priming. In contrast, 12-month-olds did not show an N400 difference, neither between pseudo-and nonwords nor between incongruous and congruous words. Both 1-year-old groups and adults additionally displayed a lexical priming effect for congruous words, that is, a negativity starting around 100 msec after words onset. One-year-olds, moreover, displayed a phonotactic familiarity effect, that is, a widely distributed negativity starting around 250 msec in 19-month-olds but occurring later in 12-month-olds. The results imply that both lexical priming and phonotactic familiarity already affect the processing of acoustic stimuli in children at 12 months of age. In 19-month-olds, adult-like mechanisms of semantic integration are present in response to phonotactically legal, but not to phonotactically illegal, nonsense words, indicating that children at this age treat pseudo-words, but not nonwords, as potential word candidates.

scholarly journals Music with Concurrent Saliences of Musical Features Elicits Stronger Brain Responses

2021 ◽  
Vol 11 (19) ◽  
pp. 9158
Author(s):  
Lorenzo J. Tardón ◽  
Ignacio Rodríguez-Rodríguez ◽  
Niels T. Haumann ◽  
Elvira Brattico ◽  
Isabel Barbancho
Keyword(s):  
Statistical Analysis ◽  
Real Life ◽  
Acoustic Features ◽  
Eeg Signals ◽  
Experimental Conditions ◽  
Brain Responses ◽  
Spectral Flux ◽  
Initial Hypothesis ◽  
Brain Reaction

Brain responses are often studied under strictly experimental conditions in which electroencephalograms (EEGs) are recorded to reflect reactions to short and repetitive stimuli. However, in real life, aural stimuli are continuously mixed and cannot be found isolated, such as when listening to music. In this audio context, the acoustic features in music related to brightness, loudness, noise, and spectral flux, among others, change continuously; thus, significant values of these features can occur nearly simultaneously. Such situations are expected to give rise to increased brain reaction with respect to a case in which they would appear in isolation. In order to assert this, EEG signals recorded while listening to a tango piece were considered. The focus was on the amplitude and time of the negative deflation (N100) and positive deflation (P200) after the stimuli, which was defined on the basis of the selected music feature saliences, in order to perform a statistical analysis intended to test the initial hypothesis. Differences in brain reactions can be identified depending on the concurrence (or not) of such significant values of different features, proving that coterminous increments in several qualities of music influence and modulate the strength of brain responses.

scholarly journals Experience-based Auditory Predictions Modulate Brain Activity to Silence as Do Real Sounds

2015 ◽  
Vol 27 (10) ◽  
pp. 1968-1980 ◽  
Author(s):  
Leila Chouiter ◽  
Athina Tzovara ◽  
Sebastian Dieguez ◽  
Jean-Marie Annoni ◽  
David Magezi ◽  
...  
Keyword(s):  
Temporal Dynamics ◽  
Brain Activity ◽  
Internal Models ◽  
Active Listening ◽  
Acoustic Features ◽  
Noisy Environments ◽  
Brain Responses ◽  
Pitch Direction ◽  
Electrophysiological Responses ◽  
Left Posterior

Interactions between stimuli's acoustic features and experience-based internal models of the environment enable listeners to compensate for the disruptions in auditory streams that are regularly encountered in noisy environments. However, whether auditory gaps are filled in predictively or restored a posteriori remains unclear. The current lack of positive statistical evidence that internal models can actually shape brain activity as would real sounds precludes accepting predictive accounts of filling-in phenomenon. We investigated the neurophysiological effects of internal models by testing whether single-trial electrophysiological responses to omitted sounds in a rule-based sequence of tones with varying pitch could be decoded from the responses to real sounds and by analyzing the ERPs to the omissions with data-driven electrical neuroimaging methods. The decoding of the brain responses to different expected, but omitted, tones in both passive and active listening conditions was above chance based on the responses to the real sound in active listening conditions. Topographic ERP analyses and electrical source estimations revealed that, in the absence of any stimulation, experience-based internal models elicit an electrophysiological activity different from noise and that the temporal dynamics of this activity depend on attention. We further found that the expected change in pitch direction of omitted tones modulated the activity of left posterior temporal areas 140–200 msec after the onset of omissions. Collectively, our results indicate that, even in the absence of any stimulation, internal models modulate brain activity as do real sounds, indicating that auditory filling in can be accounted for by predictive activity.

scholarly journals Incorporating Context into Language Encoding Models for fMRI

2018 ◽  
Author(s):  
Shailee Jain ◽  
Alexander G Huth
Keyword(s):  
Human Brain ◽  
Language Processing ◽  
Contextual Information ◽  
Brain Area ◽  
Language Model ◽  
Language Models ◽  
Brain Responses ◽  
Learning Functions ◽  
High Level ◽  
Current Word

AbstractLanguage encoding models help explain language processing in the human brain by learning functions that predict brain responses from the language stimuli that elicited them. Current word embedding-based approaches treat each stimulus word independently and thus ignore the influence of context on language understanding. In this work, we instead build encoding models using rich contextual representations derived from an LSTM language model. Our models show a significant improvement in encoding performance relative to state-of-the-art embeddings in nearly every brain area. By varying the amount of context used in the models and providing the models with distorted context, we show that this improvement is due to a combination of better word embeddings learned by the LSTM language model and contextual information. We are also able to use our models to map context sensitivity across the cortex. These results suggest that LSTM language models learn high-level representations that are related to representations in the human brain.

Sign in / Sign up

Export Citation Format

Share Document