scholarly journals Age-related decreases of cortical visuo-phonological transformation of unheard spectral fine-details

2021 ◽  
Author(s):  
Nina Suess ◽  
Anne Hauswald ◽  
Patrick Reisinger ◽  
Sebastian Rösch ◽  
Anne Keitel ◽  
...  
Keyword(s):  
Visual Cortex ◽  
Fundamental Frequency ◽  
Visual Information ◽  
Visual Speech ◽  
Phonological Information ◽  
Noisy Environments ◽  
Neural Basis ◽  
Resonant Frequencies ◽  
Acoustic Information ◽  
Speech Envelope

AbstractThe integration of visual and auditory cues is crucial for successful processing of speech, especially under adverse conditions. Recent reports have shown that when participants watch muted videos of speakers, the phonological information about the acoustic speech envelope is tracked by the visual cortex. However, the speech signal also carries much richer acoustic details, e.g. about the fundamental frequency and the resonant frequencies, whose visuo-phonological transformation could aid speech processing. Here, we investigated the neural basis of the visuo-phonological transformation processes of these more fine-grained acoustic details and assessed how they change with ageing. We recorded whole-head magnetoencephalography (MEG) data while participants watched silent intelligible and unintelligible videos of a speaker. We found that the visual cortex is able to track the unheard intelligible modulations of resonant frequencies and the pitch linked to lip movements. Importantly, only the processing of intelligible unheard formants decreases significantly with age in the visual and also in the cingulate cortex. This is not the case for the processing of the unheard speech envelope, the fundamental frequency or the purely visual information carried by lip movements. These results show that unheard spectral fine-details (along with the unheard acoustic envelope) are transformed from a mere visual to a phonological representation. Aging affects especially the ability to derive spectral dynamics at formant frequencies. Since listening in noisy environments should capitalize on the ability to track spectral fine-details, our results provide a novel focus on compensatory processes in such challenging situations.Significance statementThe multisensory integration of speech cues from visual and auditory modalities is crucial for optimal speech perception in noisy environments or for elderly individuals with progressive hearing loss. It has already been shown that the visual cortex is able to extract global acoustic information like amplitude modulations from silent visual speech, but whether this extends to fine-detailed spectral acoustic information remains unclear. Here, we demonstrate that the visual cortex is indeed able to extract fine-detailed phonological cues just from watching silent lip movements. Furthermore, this tracking of acoustic fine-details is deteriorating with age. These results suggest that the human brain is able to transform visual information into useful phonological information, and this process might be crucially affected in ageing individuals.

scholarly journals Auditory and Language Contributions to Neural Encoding of Speech Features in Noisy Environments

2018 ◽  
Author(s):  
Jiajie Zou ◽  
Jun Feng ◽  
Tianyong Xu ◽  
Peiqing Jin ◽  
Cheng Luo ◽  
...  
Keyword(s):  
Language Processing ◽  
Noise Intensity ◽  
Gain Control ◽  
Neural Representation ◽  
Noisy Environments ◽  
Envelope Tracking ◽  
Neural Basis ◽  
Speech Envelope ◽  
Noise Robust ◽  
Temporal Profiles

AbstractRecognizing speech in noisy environments is a challenging task that involves both auditory and language mechanisms. Previous studies have demonstrated noise-robust neural tracking of the speech envelope, i.e., fluctuations in sound intensity, in human auditory cortex, which provides a plausible neural basis for noise-robust speech recognition. The current study aims at teasing apart auditory and language contributions to noise-robust envelope tracking by comparing 2 groups of listeners, i.e., native listeners of the testing language and foreign listeners who do not understand the testing language. In the experiment, speech is mixed with spectrally matched stationary noise at 4 intensity levels and the neural responses are recorded using electroencephalography (EEG). When the noise intensity increases, an increase in neural response gain is observed for both groups of listeners, demonstrating auditory gain control mechanisms. Language comprehension creates no overall boost in the response gain or the envelope-tracking precision but instead modulates the spatial and temporal profiles of envelope-tracking activity. Based on the spatio-temporal dynamics of envelope-tracking activity, the 2 groups of listeners and the 4 levels of noise intensity can be jointly decoded by a linear classifier. All together, the results show that without feedback from language processing, auditory mechanisms such as gain control can lead to a noise-robust speech representation. High-level language processing, however, further modulates the spatial-temporal profiles of the neural representation of the speech envelope.

scholarly journals Frontal cortex selects representations of the talker’s mouth to aid in speech perception

eLife ◽  
2018 ◽  
Vol 7 ◽  
Author(s):  
Muge Ozker ◽  
Daniel Yoshor ◽  
Michael S Beauchamp
Keyword(s):  
Visual Cortex ◽  
Speech Perception ◽  
Frontal Cortex ◽  
Visual Information ◽  
Brain Regions ◽  
Visual Speech ◽  
Auditory Information ◽  
Sources Of Information ◽  
Audiovisual Speech Perception ◽  
Strong Connectivity

Human faces contain multiple sources of information. During speech perception, visual information from the talker’s mouth is integrated with auditory information from the talker's voice. By directly recording neural responses from small populations of neurons in patients implanted with subdural electrodes, we found enhanced visual cortex responses to speech when auditory speech was absent (rendering visual speech especially relevant). Receptive field mapping demonstrated that this enhancement was specific to regions of the visual cortex with retinotopic representations of the mouth of the talker. Connectivity between frontal cortex and other brain regions was measured with trial-by-trial power correlations. Strong connectivity was observed between frontal cortex and mouth regions of visual cortex; connectivity was weaker between frontal cortex and non-mouth regions of visual cortex or auditory cortex. These results suggest that top-down selection of visual information from the talker’s mouth by frontal cortex plays an important role in audiovisual speech perception.

scholarly journals Visual Information Shapes the Dynamics of Corticobasal Ganglia Pathways during Response Selection and Inhibition

2015 ◽  
Vol 27 (7) ◽  
pp. 1344-1359 ◽  
Author(s):  
Sara Jahfari ◽  
Lourens Waldorp ◽  
K. Richard Ridderinkhof ◽  
H. Steven Scholte
Keyword(s):  
Visual Cortex ◽  
Visual Information ◽  
Response Selection ◽  
Selection Process ◽  
Effective Connectivity ◽  
Action Selection ◽  
Visual Input ◽  
Fmri Data ◽  
Response Strategies ◽  
Fast Flow

Action selection often requires the transformation of visual information into motor plans. Preventing premature responses may entail the suppression of visual input and/or of prepared muscle activity. This study examined how the quality of visual information affects frontobasal ganglia (BG) routes associated with response selection and inhibition. Human fMRI data were collected from a stop task with visually degraded or intact face stimuli. During go trials, degraded spatial frequency information reduced the speed of information accumulation and response cautiousness. Effective connectivity analysis of the fMRI data showed action selection to emerge through the classic direct and indirect BG pathways, with inputs deriving form both prefrontal and visual regions. When stimuli were degraded, visual and prefrontal regions processing the stimulus information increased connectivity strengths toward BG, whereas regions evaluating visual scene content or response strategies reduced connectivity toward BG. Response inhibition during stop trials recruited the indirect and hyperdirect BG pathways, with input from visual and prefrontal regions. Importantly, when stimuli were nondegraded and processed fast, the optimal stop model contained additional connections from prefrontal to visual cortex. Individual differences analysis revealed that stronger prefrontal-to-visual connectivity covaried with faster inhibition times. Therefore, prefrontal-to-visual cortex connections appear to suppress the fast flow of visual input for the go task, such that the inhibition process can finish before the selection process. These results indicate response selection and inhibition within the BG to emerge through the interplay of top–down adjustments from prefrontal and bottom–up input from sensory cortex.

Adult Cortical Dynamics

1998 ◽  
Vol 78 (2) ◽  
pp. 467-485 ◽  
Author(s):  
CHARLES D. GILBERT
Keyword(s):  
Visual Cortex ◽  
Functional Properties ◽  
Primary Visual Cortex ◽  
Visual Information ◽  
Visual Space ◽  
Visual Pathway ◽  
Local Features ◽  
Spatial Integration ◽  
Cortical Dynamics ◽  
Wide Range

Gilbert, Charles D. Adult Cortical Dynamics. Physiol. Rev. 78: 467–485, 1998. — There are many influences on our perception of local features. What we see is not strictly a reflection of the physical characteristics of a scene but instead is highly dependent on the processes by which our brain attempts to interpret the scene. As a result, our percepts are shaped by the context within which local features are presented, by our previous visual experiences, operating over a wide range of time scales, and by our expectation of what is before us. The substrate for these influences is likely to be found in the lateral interactions operating within individual areas of the cerebral cortex and in the feedback from higher to lower order cortical areas. Even at early stages in the visual pathway, cells are far more flexible in their functional properties than previously thought. It had long been assumed that cells in primary visual cortex had fixed properties, passing along the product of a stereotyped operation to the next stage in the visual pathway. Any plasticity dependent on visual experience was thought to be restricted to a period early in the life of the animal, the critical period. Furthermore, the assembly of contours and surfaces into unified percepts was assumed to take place at high levels in the visual pathway, whereas the receptive fields of cells in primary visual cortex represented very small windows on the visual scene. These concepts of spatial integration and plasticity have been radically modified in the past few years. The emerging view is that even at the earliest stages in the cortical processing of visual information, cells are highly mutable in their functional properties and are capable of integrating information over a much larger part of visual space than originally believed.

scholarly journals Increased connectivity among sensory and motor regions during visual and audiovisual speech perception

2020 ◽  
Author(s):  
Jonathan E Peelle ◽  
Brent Spehar ◽  
Michael S Jones ◽  
Sarah McConkey ◽  
Joel Myerson ◽  
...  
Keyword(s):  
Visual Cortex ◽  
Speech Perception ◽  
Brain Activity ◽  
Premotor Cortex ◽  
Temporal Cortex ◽  
Primary Auditory Cortex ◽  
Visual Speech ◽  
Auditory Signal ◽  
Audiovisual Speech ◽  
Audiovisual Speech Perception

In everyday conversation, we usually process the talker's face as well as the sound of their voice. Access to visual speech information is particularly useful when the auditory signal is degraded. Here we used fMRI to monitor brain activity while adults (n = 60) were presented with visual-only, auditory-only, and audiovisual words. As expected, audiovisual speech perception recruited both auditory and visual cortex, with a trend towards increased recruitment of premotor cortex in more difficult conditions (for example, in substantial background noise). We then investigated neural connectivity using psychophysiological interaction (PPI) analysis with seed regions in both primary auditory cortex and primary visual cortex. Connectivity between auditory and visual cortices was stronger in audiovisual conditions than in unimodal conditions, including a wide network of regions in posterior temporal cortex and prefrontal cortex. Taken together, our results suggest a prominent role for cross-region synchronization in understanding both visual-only and audiovisual speech.

scholarly journals LIP-READING VIA DEEP NEURAL NETWORKS USING HYBRID VISUAL FEATURES

2018 ◽  
Vol 37 (2) ◽  
pp. 159 ◽  
Author(s):  
Fatemeh Vakhshiteh ◽  
Farshad Almasganj ◽  
Ahmad Nickabadi
Keyword(s):  
Speech Intelligibility ◽  
Deep Neural Networks ◽  
Visual Speech ◽  
Visual Features ◽  
Noisy Environments ◽  
Phone Recognition ◽  
Facial Information ◽  
Visual Speech Recognition ◽  
Lip Reading ◽  
Reading System

Lip-reading is typically known as visually interpreting the speaker's lip movements during speaking. Experiments over many years have revealed that speech intelligibility increases if visual facial information becomes available. This effect becomes more apparent in noisy environments. Taking steps toward automating this process, some challenges will be raised such as coarticulation phenomenon, visual units' type, features diversity and their inter-speaker dependency. While efforts have been made to overcome these challenges, presentation of a flawless lip-reading system is still under the investigations. This paper searches for a lipreading model with an efficiently developed incorporation and arrangement of processing blocks to extract highly discriminative visual features. Here, application of a properly structured Deep Belief Network (DBN)- based recognizer is highlighted. Multi-speaker (MS) and speaker-independent (SI) tasks are performed over CUAVE database, and phone recognition rates (PRRs) of 77.65% and 73.40% are achieved, respectively. The best word recognition rates (WRRs) achieved in the tasks of MS and SI are 80.25% and 76.91%, respectively. Resulted accuracies demonstrate that the proposed method outperforms the conventional Hidden Markov Model (HMM) and competes well with the state-of-the-art visual speech recognition works.

scholarly journals Using perceptual tasks to selectively measure magnocellular and parvocellular performance: Rationale and a user’s guide

2021 ◽  
Author(s):  
Mark Edwards ◽  
Stephanie C. Goodhew ◽  
David R. Badcock
Keyword(s):  
Visual Cortex ◽  
Visual System ◽  
Lateral Geniculate Nucleus ◽  
Cognitive Functions ◽  
Visual Information ◽  
Visual Stimuli ◽  
Visual Scene ◽  
Ongoing Debate ◽  
Parvocellular Pathway ◽  
Lesion Studies

AbstractThe visual system uses parallel pathways to process information. However, an ongoing debate centers on the extent to which the pathways from the retina, via the Lateral Geniculate nucleus to the visual cortex, process distinct aspects of the visual scene and, if they do, can stimuli in the laboratory be used to selectively drive them. These questions are important for a number of reasons, including that some pathologies are thought to be associated with impaired functioning of one of these pathways and certain cognitive functions have been preferentially linked to specific pathways. Here we examine the two main pathways that have been the focus of this debate: the magnocellular and parvocellular pathways. Specifically, we review the results of electrophysiological and lesion studies that have investigated their properties and conclude that while there is substantial overlap in the type of information that they process, it is possible to identify aspects of visual information that are predominantly processed by either the magnocellular or parvocellular pathway. We then discuss the types of visual stimuli that can be used to preferentially drive these pathways.

scholarly journals Thalamic inputs determine functionally distinct gamma bands in mouse primary visual cortex

2020 ◽  
Author(s):  
Nicolò Meneghetti ◽  
Chiara Cerri ◽  
Elena Tantillo ◽  
Eleonora Vannini ◽  
Matteo Caleo ◽  
...  
Keyword(s):  
Visual Cortex ◽  
Primary Visual Cortex ◽  
Visual Information ◽  
Narrow Band ◽  
Broad Band ◽  
Gamma Band ◽  
Neuron Network ◽  
Sensory Stimuli ◽  
Thalamic Input ◽  
Visual Contrast

AbstractGamma band is known to be involved in the encoding of visual features in the primary visual cortex (V1). Recent results in rodents V1 highlighted the presence, within a broad gamma band (BB) increasing with contrast, of a narrow gamma band (NB) peaking at ∼60 Hz suppressed by contrast and enhanced by luminance. However, the processing of visual information by the two channels still lacks a proper characterization. Here, by combining experimental analysis and modeling, we prove that the two bands are sensitive to specific thalamic inputs associated with complementary contrast ranges. We recorded local field potentials from V1 of awake mice during the presentation of gratings and observed that NB power progressively decreased from low to intermediate levels of contrast. Conversely, BB power was insensitive to low levels of contrast but it progressively increased going from intermediate to high levels of contrast. Moreover, BB response was stronger immediately after contrast reversal, while the opposite held for NB. All the aforementioned dynamics were accurately reproduced by a recurrent excitatory-inhibitory leaky integrate-and-fire network, mimicking layer IV of mouse V1, provided that the sustained and periodic component of the thalamic input were modulated over complementary contrast ranges. These results shed new light on the origin and function of the two V1 gamma bands. In addition, here we propose a simple and effective model of response to visual contrast that might help in reconstructing network dysfunction underlying pathological alterations of visual information processing.Significance StatementGamma band is a ubiquitous hallmark of cortical processing of sensory stimuli. Experimental evidence shows that in the mouse visual cortex two types of gamma activity are differentially modulated by contrast: a narrow band (NB), that seems to be rodent specific, and a standard broad band (BB), observed also in other animal models.We found that narrow band correlates and broad band anticorrelates with visual contrast in two complementary contrast ranges (low and high respectively). Moreover, BB displayed an earlier response than NB. A thalamocortical spiking neuron network model reproduced the aforementioned results, suggesting they might be due to the presence of two complementary but distinct components of the thalamic input into visual cortical circuitry.

scholarly journals Changes in thalamocortical connectivity as a potential mechanism of cross-modal plasticity in congenitally blind individuals

2018 ◽  
Author(s):  
Theo Marins ◽  
Maite Russo ◽  
Erika Rodrigues ◽  
jorge Moll ◽  
Daniel Felix ◽  
...  
Keyword(s):  
Visual Cortex ◽  
Cortical Thickness ◽  
Visual Information ◽  
Brain Plasticity ◽  
Temporal Cortex ◽  
Occipital Cortex ◽  
Brain Structures ◽  
Congenitally Blind ◽  
Blind Individuals ◽  
Neuroimaging Techniques

ABSTRACTEvidence of cross-modal plasticity in blind individuals has been reported over the past decades showing that non-visual information is carried and processed by classical “visual” brain structures. This feature of the blind brain makes it a pivotal model to explore the limits and mechanisms of brain plasticity. However, despite recent efforts, the structural underpinnings that could explain cross-modal plasticity in congenitally blind individuals remain unclear. Using advanced neuroimaging techniques, we mapped the thalamocortical connectivity and assessed cortical thickness and integrity of white matter of congenitally blind individuals and sighted controls to test the hypothesis that aberrant thalamocortical pattern of connectivity can pave the way for cross-modal plasticity. We described a direct occipital takeover by the temporal projections from the thalamus, which would carry non-visual information (e.g. auditory) to the visual cortex in congenitally blinds. In addition, the amount of thalamo-occipital connectivity correlated with the cortical thickness of primary visual cortex (V1), supporting a probably common (or related) reorganization phenomena. Our results suggest that aberrant thalamocortical connectivity as one possible mechanism of cross-modal plasticity in blinds, with potential impact on cortical thickness of V1.SIGNIFICANT STATEMENTCongenitally blind individuals often develop greater abilities on spared sensory modalities, such as increased acuity in auditory discrimination and voice recognition, when compared to sighted controls. These functional gains have been shown to rely on ‘visual’ cortical areas of the blind brain, characterizing the phenomenon of cross-modal plasticity. However, its anatomical underpinnings in humans have been unsuccessfully pursued for decades. Recent advances of non-invasive neuroimaging techniques allowed us to test the hypothesis of abnormal thalamocortical connectivity in congenitally blinds. Our results showed an expansion of the thalamic connections to the temporal cortex over those that project to the occipital cortex, which may explain, the cross-talk between the visual and auditory systems in congenitally blind individuals.

Sign in / Sign up

Export Citation Format

Share Document