On the Timing of Signals in Multisensory Integration and Crossmodal Interactions: a Scoping Review

2019 ◽  
Vol 32 (6) ◽  
pp. 533-573
Author(s):  
Philip Sanders ◽  
Benjamin Thompson ◽  
Paul Corballis ◽  
Grant Searchfield
Keyword(s):  
Scoping Review ◽  
Temporal Processing ◽  
Computational Models ◽  
Visual Stimuli ◽  
Multisensory Perception ◽  
Early Integration ◽  
Crossmodal Interactions ◽  
Temporal Relationships ◽  
The Brain ◽  
Neuroimaging Techniques

Abstract A scoping review was undertaken to explore research investigating early interactions and integration of auditory and visual stimuli in the human brain. The focus was on methods used to study low-level multisensory temporal processing using simple stimuli in humans, and how this research has informed our understanding of multisensory perception. The study of multisensory temporal processing probes how the relative timing between signals affects perception. Several tasks, illusions, computational models, and neuroimaging techniques were identified in the literature search. Research into early audiovisual temporal processing in special populations was also reviewed. Recent research has continued to provide support for early integration of crossmodal information. These early interactions can influence higher-level factors, and vice versa. Temporal relationships between auditory and visual stimuli influence multisensory perception, and likely play a substantial role in solving the ‘correspondence problem’ (how the brain determines which sensory signals belong together, and which should be segregated).

Synaesthesia

2018 ◽  
Author(s):  
Bruno and
Keyword(s):  
Experimental Results ◽  
Multisensory Perception ◽  
Related Phenomenon ◽  
Perceptual Modality ◽  
Sensory Channel ◽  
Multisensory Interactions ◽  
Crossmodal Associations ◽  
The Brain

Synaesthesia is a curious anomaly of multisensory perception. When presented with stimulation in one sensory channel, in addition to the percept usually associated with that channel (inducer) a true synaesthetic experiences a second percept in another perceptual modality (concurrent). Although synaesthesia is not pathological, true synaesthetes are relatively rare and their synaesthetic associations tend to be quite idiosyncratic. For this reason, studying synaesthesia is difficult, but exciting new experimental results are beginning to clarify what makes the brain of synaesthetes special and the mechanisms that may produce the condition. Even more importantly, the related phenomenon known as ‘natural’ crossmodal associations is instead experienced by everyone, providing another useful domain for studying multisensory interactions with important implications for understanding our preferences for products in terms of spontaneously evoked associations, as well as for choosing appropriate names, labels, and packaging in marketing applications.

scholarly journals Power Failure of Mitochondria and Oxidative Stress in Neurodegeneration and Its Computational Models

Antioxidants ◽  
2021 ◽  
Vol 10 (2) ◽  
pp. 229
Author(s):  
JunHyuk Woo ◽  
Hyesun Cho ◽  
YunHee Seol ◽  
Soon Ho Kim ◽  
Chanhyeok Park ◽  
...  
Keyword(s):  
Oxidative Stress ◽  
Mitochondrial Dysfunction ◽  
Computational Models ◽  
Neuronal Damage ◽  
Living Organism ◽  
The Body ◽  
Mitochondrial Functions ◽  
A Cell ◽  
The Brain ◽  
And Oxidative Stress

The brain needs more energy than other organs in the body. Mitochondria are the generator of vital power in the living organism. Not only do mitochondria sense signals from the outside of a cell, but they also orchestrate the cascade of subcellular events by supplying adenosine-5′-triphosphate (ATP), the biochemical energy. It is known that impaired mitochondrial function and oxidative stress contribute or lead to neuronal damage and degeneration of the brain. This mini-review focuses on addressing how mitochondrial dysfunction and oxidative stress are associated with the pathogenesis of neurodegenerative disorders including Alzheimer’s disease, amyotrophic lateral sclerosis, Huntington’s disease, and Parkinson’s disease. In addition, we discuss state-of-the-art computational models of mitochondrial functions in relation to oxidative stress and neurodegeneration. Together, a better understanding of brain disease-specific mitochondrial dysfunction and oxidative stress can pave the way to developing antioxidant therapeutic strategies to ameliorate neuronal activity and prevent neurodegeneration.

scholarly journals Pain Detection and the Privacy of Subjective Experience

2007 ◽  
Vol 33 (2-3) ◽  
pp. 433-456 ◽  
Author(s):  
Adam J. Kolber
Keyword(s):  
Subjective Experience ◽  
Brain Regions ◽  
Physical Pain ◽  
Medical Evidence ◽  
Functional Magnetic Resonance ◽  
Positron Emission ◽  
Pain Symptoms ◽  
The Brain ◽  
Insight Into ◽  
Neuroimaging Techniques

A neurologist with abdominal pain goes to see a gastroenterologist for treatment. The gastroenterologist asks the neurologist where it hurts. The neurologist replies, “In my head, of course.” Indeed, while we can feel pain throughout much of our bodies, pain signals undergo most of their processing in the brain. Using neuroimaging techniques like functional magnetic resonance imaging (“fMRI”) and positron emission tomography (“PET”), researchers have more precisely identified brain regions that enable us to experience physical pain. Certain regions of the brain's cortex, for example, increase in activation when subjects are exposed to painful stimuli. Furthermore, the amount of activation increases with the intensity of the painful stimulus. These findings suggest that we may be able to gain insight into the amount of pain a particular person is experiencing by non-invasively imaging his brain.Such insight could be particularly valuable in the courtroom where we often have no definitive medical evidence to prove or disprove claims about the existence and extent of pain symptoms.

scholarly journals Inferring brain-computational mechanisms with models of activity measurements

2016 ◽  
Vol 371 (1705) ◽  
pp. 20160278 ◽  
Author(s):  
Nikolaus Kriegeskorte ◽  
Jörn Diedrichsen
Keyword(s):  
Computational Models ◽  
Brain Activity ◽  
Visual Object ◽  
Visual Object Recognition ◽  
Summary Statistics ◽  
Activity Data ◽  
Individual Measurement ◽  
Activity Measurements ◽  
The Brain ◽  
The Way

High-resolution functional imaging is providing increasingly rich measurements of brain activity in animals and humans. A major challenge is to leverage such data to gain insight into the brain's computational mechanisms. The first step is to define candidate brain-computational models (BCMs) that can perform the behavioural task in question. We would then like to infer which of the candidate BCMs best accounts for measured brain-activity data. Here we describe a method that complements each BCM by a measurement model (MM), which simulates the way the brain-activity measurements reflect neuronal activity (e.g. local averaging in functional magnetic resonance imaging (fMRI) voxels or sparse sampling in array recordings). The resulting generative model (BCM-MM) produces simulated measurements. To avoid having to fit the MM to predict each individual measurement channel of the brain-activity data, we compare the measured and predicted data at the level of summary statistics. We describe a novel particular implementation of this approach, called probabilistic representational similarity analysis (pRSA) with MMs, which uses representational dissimilarity matrices (RDMs) as the summary statistics. We validate this method by simulations of fMRI measurements (locally averaging voxels) based on a deep convolutional neural network for visual object recognition. Results indicate that the way the measurements sample the activity patterns strongly affects the apparent representational dissimilarities. However, modelling of the measurement process can account for these effects, and different BCMs remain distinguishable even under substantial noise. The pRSA method enables us to perform Bayesian inference on the set of BCMs and to recognize the data-generating model in each case. This article is part of the themed issue ‘Interpreting BOLD: a dialogue between cognitive and cellular neuroscience’.

Bistability of somatic pattern memories: stochastic outcomes in bioelectric circuits underlying regeneration

2021 ◽  
Vol 376 (1821) ◽  
pp. 20190765 ◽  
Author(s):  
Giovanni Pezzulo ◽  
Joshua LaPalme ◽  
Fallon Durant ◽  
Michael Levin
Keyword(s):  
Large Scale ◽  
Computational Models ◽  
State Of The Art ◽  
Memory Representation ◽  
Theme Issue ◽  
Evolutionary Innovation ◽  
New Interpretation ◽  
The Brain

Nervous systems’ computational abilities are an evolutionary innovation, specializing and speed-optimizing ancient biophysical dynamics. Bioelectric signalling originated in cells' communication with the outside world and with each other, enabling cooperation towards adaptive construction and repair of multicellular bodies. Here, we review the emerging field of developmental bioelectricity, which links the field of basal cognition to state-of-the-art questions in regenerative medicine, synthetic bioengineering and even artificial intelligence. One of the predictions of this view is that regeneration and regulative development can restore correct large-scale anatomies from diverse starting states because, like the brain, they exploit bioelectric encoding of distributed goal states—in this case, pattern memories. We propose a new interpretation of recent stochastic regenerative phenotypes in planaria, by appealing to computational models of memory representation and processing in the brain. Moreover, we discuss novel findings showing that bioelectric changes induced in planaria can be stored in tissue for over a week, thus revealing that somatic bioelectric circuits in vivo can implement a long-term, re-writable memory medium. A consideration of the mechanisms, evolution and functionality of basal cognition makes novel predictions and provides an integrative perspective on the evolution, physiology and biomedicine of information processing in vivo . This article is part of the theme issue ‘Basal cognition: multicellularity, neurons and the cognitive lens’.

Neurocomics and Neuroimaging

2018 ◽  
Author(s):  
Jason Tougaw
Keyword(s):  
Brain Research ◽  
Direct Access ◽  
Brain Images ◽  
High Profile ◽  
Graphic Narratives ◽  
Novel Methods ◽  
The Aesthetic ◽  
Cultural Responses ◽  
The Brain ◽  
Neuroimaging Techniques

This chapter examines a small number of recent graphic brain narratives that experiment with novel methods of visualizing the brain—including David B.’s Epileptic, Ellen Forney’s Marbles, and Matteo Farinella and Hana Ros’s Neurocomic. Tougaw argues that these narratives both draw from and challenge cultural responses to high-profile neuroimaging techniques, including PET and fMRI. Graphic narratives are a subcultural genre celebrated for their rebellious aesthetics and emphasis on narratives that challenge mainstream social and political assumptions. Brain scanning technologies are highly specialized tools that have revolutionized brain research and gained considerable mainstream attention. The mainstreaming of these technologies oversimplifies the images they produce, creating a widely held sense that they offer direct access to the brains they visualize. By contrast, graphic narratives put heavy emphasis on the aesthetic process involved in their making of brain images. While careful not to minimize these differences, the chapter argues that key similarities between neurocomics and neuroimaging techniques can be a means for clarifying the roles played by the sciences and the humanities in the cultural laboratory of contemporary neuromania.

scholarly journals Attention controls multisensory perception via 2 distinct mechanisms at different levels of the cortical hierarchy

PLoS Biology ◽  
2021 ◽  
Vol 19 (11) ◽  
pp. e3001465
Author(s):  
Ambra Ferrari ◽  
Uta Noppeney
Keyword(s):  
Causal Inference ◽  
Causal Structure ◽  
Multisensory Perception ◽  
Spatial Representations ◽  
Perceptual Inference ◽  
Sensory Signals ◽  
Cortical Hierarchy ◽  
Parietal Cortices ◽  
The Brain ◽  
Different Levels

To form a percept of the multisensory world, the brain needs to integrate signals from common sources weighted by their reliabilities and segregate those from independent sources. Previously, we have shown that anterior parietal cortices combine sensory signals into representations that take into account the signals’ causal structure (i.e., common versus independent sources) and their sensory reliabilities as predicted by Bayesian causal inference. The current study asks to what extent and how attentional mechanisms can actively control how sensory signals are combined for perceptual inference. In a pre- and postcueing paradigm, we presented observers with audiovisual signals at variable spatial disparities. Observers were precued to attend to auditory or visual modalities prior to stimulus presentation and postcued to report their perceived auditory or visual location. Combining psychophysics, functional magnetic resonance imaging (fMRI), and Bayesian modelling, we demonstrate that the brain moulds multisensory inference via 2 distinct mechanisms. Prestimulus attention to vision enhances the reliability and influence of visual inputs on spatial representations in visual and posterior parietal cortices. Poststimulus report determines how parietal cortices flexibly combine sensory estimates into spatial representations consistent with Bayesian causal inference. Our results show that distinct neural mechanisms control how signals are combined for perceptual inference at different levels of the cortical hierarchy.

scholarly journals Understanding Mental Health and Cognitive Restructuring With Ecological Neuroscience

2021 ◽  
Vol 12 ◽  
Author(s):  
James Crum
Keyword(s):  
Mental Health ◽  
Near Infrared ◽  
Ecological Validity ◽  
Experimental Designs ◽  
Functional Near Infrared Spectroscopy ◽  
Practical Applications ◽  
Recent Developments ◽  
The Brain ◽  
Neuroimaging Techniques ◽  
Selection Of

Neuroimaging and neuropsychological methods have contributed much toward an understanding of the information processing systems of the human brain in the last few decades, but to what extent do cognitive neuroscientific findings represent and generalize to the inter- and intra-brain dynamics engaged in adapting to naturalistic situations? If it is not marked, and experimental designs lack ecological validity, then this stands to potentially impact the practical applications of a paradigm. In no other domain is this more important to acknowledge than in human clinical neuroimaging research, wherein reduced ecological validity could mean a loss in clinical utility. One way to improve the generalizability and representativeness of findings is to adopt a more “real-world” approach to the development and selection of experimental designs and neuroimaging techniques to investigate the clinically-relevant phenomena of interest. For example, some relatively recent developments to neuroimaging techniques such as functional near-infrared spectroscopy (fNIRS) make it possible to create experimental designs using naturalistic tasks that would otherwise not be possible within the confines of a conventional laboratory. Mental health, cognitive interventions, and the present challenges to investigating the brain during treatment are discussed, as well as how the ecological use of fNIRS might be helpful in bridging the explanatory gaps to understanding the cultivation of mental health.

scholarly journals Distributed Code for Semantic Relations Predicts Neural Similarity

2019 ◽  
Author(s):  
Jeffrey N. Chiang ◽  
Yujia Peng ◽  
Hongjing Lu ◽  
Keith J. Holyoak ◽  
Martin M. Monti
Keyword(s):  
Computational Models ◽  
Semantic Network ◽  
Building Blocks ◽  
Semantic Relations ◽  
Human Cognition ◽  
Sequential Presentation ◽  
Specific Content ◽  
Distributed Representations ◽  
Relational Similarity ◽  
The Brain

AbstractThe ability to generate and process semantic relations is central to many aspects of human cognition. Theorists have long debated whether such relations are coded as atomistic links in a semantic network, or as distributed patterns over some core set of abstract relations. The form and content of the conceptual and neural representations of semantic relations remains to be empirically established. The present study combined computational modeling and neuroimaging to investigate the representation and comparison of abstract semantic relations in the brain. By using sequential presentation of verbal analogies, we decoupled the neural activity associated with encoding the representation of the first-order semantic relation between words in a pair from that associated with the second-order comparison of two relations. We tested alternative computational models of relational similarity in order to distinguish between rival accounts of how semantic relations are coded and compared in the brain. Analyses of neural similarity patterns supported the hypothesis that semantic relations are coded, in the parietal cortex, as distributed representations over a pool of abstract relations specified in a theory-based taxonomy. These representations, in turn, provide the immediate inputs to the process of analogical comparison, which draws on a broad frontoparietal network. This study sheds light not only on the form of relation representations but also on their specific content.SignificanceRelations provide basic building blocks for language and thought. For the past half century, cognitive scientists exploring human semantic memory have sought to identify the code for relations. In a neuroimaging paradigm, we tested alternative computational models of relation processing that predict patterns of neural similarity during distinct phases of analogical reasoning. The findings allowed us to draw inferences not only about the form of relation representations, but also about their specific content. The core of these distributed representations is based on a relatively small number of abstract relation types specified in a theory-based taxonomy. This study helps to resolve a longstanding debate concerning the nature of the conceptual and neural code for semantic relations in the mind and brain.

Sign in / Sign up

Export Citation Format

Share Document