scholarly journals How multisensory neurons solve causal inference

2021 ◽  
Vol 118 (32) ◽  
pp. e2106235118
Author(s):  
Reuben Rideaux ◽  
Katherine R. Storrs ◽  
Guido Maiello ◽  
Andrew E. Welchman
Keyword(s):  
Motion Estimation ◽  
Causal Inference ◽  
Visual Motion ◽  
Temporal Area ◽  
Medial Superior Temporal ◽  
Electrophysiological Recordings ◽  
Visual Motion Cues ◽  
Integrated Or ◽  
Self Motion ◽  
The Brain

Sitting in a static railway carriage can produce illusory self-motion if the train on an adjoining track moves off. While our visual system registers motion, vestibular signals indicate that we are stationary. The brain is faced with a difficult challenge: is there a single cause of sensations (I am moving) or two causes (I am static, another train is moving)? If a single cause, integrating signals produces a more precise estimate of self-motion, but if not, one cue should be ignored. In many cases, this process of causal inference works without error, but how does the brain achieve it? Electrophysiological recordings show that the macaque medial superior temporal area contains many neurons that encode combinations of vestibular and visual motion cues. Some respond best to vestibular and visual motion in the same direction (“congruent” neurons), while others prefer opposing directions (“opposite” neurons). Congruent neurons could underlie cue integration, but the function of opposite neurons remains a puzzle. Here, we seek to explain this computational arrangement by training a neural network model to solve causal inference for motion estimation. Like biological systems, the model develops congruent and opposite units and recapitulates known behavioral and neurophysiological observations. We show that all units (both congruent and opposite) contribute to motion estimation. Importantly, however, it is the balance between their activity that distinguishes whether visual and vestibular cues should be integrated or separated. This explains the computational purpose of puzzling neural representations and shows how a relatively simple feedforward network can solve causal inference.

scholarly journals Processing of object motion and self-motion in the lateral subdivision of the medial superior temporal area in macaques

2019 ◽  
Vol 121 (4) ◽  
pp. 1207-1221 ◽  
Author(s):  
Ryo Sasaki ◽  
Dora E. Angelaki ◽  
Gregory C. DeAngelis
Keyword(s):  
Visual Cues ◽  
Moving Objects ◽  
Visual Motion ◽  
Object Motion ◽  
Image Motion ◽  
Motion Processing ◽  
Temporal Area ◽  
Medial Superior Temporal ◽  
Visual Motion Processing ◽  
Self Motion

Multiple areas of macaque cortex are involved in visual motion processing, but their relative functional roles remain unclear. The medial superior temporal (MST) area is typically divided into lateral (MSTl) and dorsal (MSTd) subdivisions that are thought to be involved in processing object motion and self-motion, respectively. Whereas MSTd has been studied extensively with regard to processing visual and nonvisual self-motion cues, little is known about self-motion signals in MSTl, especially nonvisual signals. Moreover, little is known about how self-motion and object motion signals interact in MSTl and how this differs from interactions in MSTd. We compared the visual and vestibular heading tuning of neurons in MSTl and MSTd using identical stimuli. Our findings reveal that both visual and vestibular heading signals are weaker in MSTl than in MSTd, suggesting that MSTl is less well suited to participate in self-motion perception than MSTd. We also tested neurons in both areas with a variety of combinations of object motion and self-motion. Our findings reveal that vestibular signals improve the separability of coding of heading and object direction in both areas, albeit more strongly in MSTd due to the greater strength of vestibular signals. Based on a marginalization technique, population decoding reveals that heading and object direction can be more effectively dissociated from MSTd responses than MSTl responses. Our findings help to clarify the respective contributions that MSTl and MSTd make to processing of object motion and self-motion, although our conclusions may be somewhat specific to the multipart moving objects that we employed. NEW & NOTEWORTHY Retinal image motion reflects contributions from both the observer’s self-motion and the movement of objects in the environment. The neural mechanisms by which the brain dissociates self-motion and object motion remain unclear. This study provides the first systematic examination of how the lateral subdivision of area MST (MSTl) contributes to dissociating object motion and self-motion. We also examine, for the first time, how MSTl neurons represent translational self-motion based on both vestibular and visual cues.

Pursuit and optokinetic deficits following chemical lesions of cortical areas MT and MST

1988 ◽  
Vol 60 (3) ◽  
pp. 940-965 ◽  
Author(s):  
M. R. Dursteler ◽  
R. H. Wurtz
Keyword(s):  
Eye Movements ◽  
Visual Field ◽  
Visual Motion ◽  
Ibotenic Acid ◽  
Motion Processing ◽  
Temporal Area ◽  
Target Speed ◽  
Pursuit Eye Movements ◽  
Medial Superior Temporal ◽  
Visual Motion Processing

1. Previous experiments have shown that punctate chemical lesions within the middle temporal area (MT) of the superior temporal sulcus (STS) produce deficits in the initiation and maintenance of pursuit eye movements (10, 34). The present experiments were designed to test the effect of such chemical lesions in an area within the STS to which MT projects, the medial superior temporal area (MST). 2. We injected ibotenic acid into localized regions of MST, and we observed two deficits in pursuit eye movements, a retinotopic deficit and a directional deficit. 3. The retinotopic deficit in pursuit initiation was characterized by the monkey's inability to match eye speed to target speed or to adjust the amplitude of the saccade made to acquire the target to compensate for target motion. This deficit was related to the initiation of pursuit to targets moving in any direction in the visual field contralateral to the side of the brain with the lesion. This deficit was similar to the deficit we found following damage to extrafoveal MT except that the affected area of the visual field frequently extended throughout the entire contralateral visual field tested. 4. The directional deficit in pursuit maintenance was characterized by a failure to match eye speed to target speed once the fovea had been brought near the moving target. This deficit occurred only when the target was moving toward the side of the lesion, regardless of whether the target began to move in the ipsilateral or contralateral visual field. There was no deficit in the amplitude of saccades made to acquire the target, or in the amplitude of the catch-up saccades made to compensate for the slowed pursuit. The directional deficit is similar to the one we described previously following chemical lesions of the foveal representation in the STS. 5. Retinotopic deficits resulted from any of our injections in MST. Directional deficits resulted from lesions limited to subregions within MST, particularly lesions that invaded the floor of the STS and the posterior bank of the STS just lateral to MT. Extensive damage to the densely myelinated area of the anterior bank or to the posterior parietal area on the dorsal lip of the anterior bank produced minimal directional deficits. 6. We conclude that damage to visual motion processing in MST underlies the retinotopic pursuit deficit just as it does in MT. MST appears to be a sequential step in visual motion processing that occurs before all of the visual motion information is transmitted to the brainstem areas related to pursuit.(ABSTRACT TRUNCATED AT 400 WORDS)

scholarly journals Causal inference accounts for heading perception in the presence of object motion

2019 ◽  
Vol 116 (18) ◽  
pp. 9060-9065 ◽  
Author(s):  
Kalpana Dokka ◽  
Hyeshin Park ◽  
Michael Jansen ◽  
Gregory C. DeAngelis ◽  
Dora E. Angelaki
Keyword(s):  
Causal Inference ◽  
Virtual World ◽  
Moving Objects ◽  
Object Motion ◽  
Image Motion ◽  
Inference Problem ◽  
Accuracy And Precision ◽  
The World ◽  
Self Motion ◽  
The Brain

The brain infers our spatial orientation and properties of the world from ambiguous and noisy sensory cues. Judging self-motion (heading) in the presence of independently moving objects poses a challenging inference problem because the image motion of an object could be attributed to movement of the object, self-motion, or some combination of the two. We test whether perception of heading and object motion follows predictions of a normative causal inference framework. In a dual-report task, subjects indicated whether an object appeared stationary or moving in the virtual world, while simultaneously judging their heading. Consistent with causal inference predictions, the proportion of object stationarity reports, as well as the accuracy and precision of heading judgments, depended on the speed of object motion. Critically, biases in perceived heading declined when the object was perceived to be moving in the world. Our findings suggest that the brain interprets object motion and self-motion using a causal inference framework.

Visual selectivity for heading in the macaque ventral intraparietal area

2014 ◽  
Vol 112 (10) ◽  
pp. 2470-2480 ◽  
Author(s):  
Andre Kaminiarz ◽  
Anja Schlack ◽  
Klaus-Peter Hoffmann ◽  
Markus Lappe ◽  
Frank Bremmer
Keyword(s):  
Eye Movements ◽  
Visual Information ◽  
Flow Fields ◽  
Image Motion ◽  
Temporal Area ◽  
Cell Responses ◽  
Medial Superior Temporal ◽  
Retinal Image Motion ◽  
Ventral Intraparietal Area ◽  
Self Motion

The patterns of optic flow seen during self-motion can be used to determine the direction of one's own heading. Tracking eye movements which typically occur during everyday life alter this task since they add further retinal image motion and (predictably) distort the retinal flow pattern. Humans employ both visual and nonvisual (extraretinal) information to solve a heading task in such case. Likewise, it has been shown that neurons in the monkey medial superior temporal area (area MST) use both signals during the processing of self-motion information. In this article we report that neurons in the macaque ventral intraparietal area (area VIP) use visual information derived from the distorted flow patterns to encode heading during (simulated) eye movements. We recorded responses of VIP neurons to simple radial flow fields and to distorted flow fields that simulated self-motion plus eye movements. In 59% of the cases, cell responses compensated for the distortion and kept the same heading selectivity irrespective of different simulated eye movements. In addition, response modulations during real compared with simulated eye movements were smaller, being consistent with reafferent signaling involved in the processing of the visual consequences of eye movements in area VIP. We conclude that the motion selectivities found in area VIP, like those in area MST, provide a way to successfully analyze and use flow fields during self-motion and simultaneous tracking movements.

scholarly journals To integrate or not to integrate: Temporal dynamics of hierarchical Bayesian Causal Inference

2018 ◽  
Author(s):  
Máté Aller ◽  
Uta Noppeney
Keyword(s):  
Causal Inference ◽  
Temporal Dynamics ◽  
Causal Structure ◽  
Visual Signal ◽  
Effective Interactions ◽  
Statistical Computation ◽  
Spatial Estimates ◽  
Hierarchical Nature ◽  
Integrated Or ◽  
The Brain

AbstractTo form a percept of the environment, the brain needs to solve the binding problem – inferring whether signals come from a common cause and be integrated, or come from independent causes and be segregated. Behaviourally, humans solve this problem near-optimally as predicted by Bayesian Causal Inference; but, the neural mechanisms remain unclear. Combining Bayesian modelling, electroencephalography (EEG), and multivariate decoding in an audiovisual spatial localization task, we show that the brain accomplishes Bayesian Causal Inference by dynamically encoding multiple spatial estimates. Initially, auditory and visual signal locations are estimated independently; next, an estimate is formed that combines information from vision and audition. Yet, it is only from 200 ms onwards that the brain integrates audiovisual signals weighted by their bottom-up sensory reliabilities and top-down task-relevance into spatial priority maps that guide behavioural responses. Critically, as predicted by Bayesian Causal Inference, these spatial priority maps take into account the brain’s uncertainty about the world’s causal structure and flexibly arbitrate between sensory integration and segregation. The dynamic evolution of perceptual estimates thus reflects the hierarchical nature of Bayesian Causal Inference, a statistical computation, crucial for effective interactions with the environment.

Optic Flow Signals in Extrastriate Area MST: Comparison of Perceptual and Neuronal Sensitivity

2004 ◽  
Vol 91 (3) ◽  
pp. 1314-1326 ◽  
Author(s):  
Hilary W. Heuer ◽  
Kenneth H. Britten
Keyword(s):  
Optic Flow ◽  
Visual Motion ◽  
Extrastriate Cortex ◽  
Sufficient Information ◽  
Temporal Area ◽  
Medial Superior Temporal ◽  
Wide Range ◽  
Behavioral Choices ◽  
Behavioral Sensitivity

The medial superior temporal area of extrastriate cortex (MST) contains signals selective for nonuniform patterns of motion often termed “optic flow.” The presence of such tuning, however, does not necessarily imply involvement in perception. To quantify the relationship between these selective neuronal signals and the perception of optic flow, we designed a discrimination task that allowed us to simultaneously record neuronal and behavioral sensitivities to near-threshold optic flow stimuli tailored to MST cells' preferences. In this two-alternative forced-choice task, we controlled the salience of globally opposite patterns (e.g., expansion and contraction) by varying the coherence of the motion. Using these stimuli, we could both relate the sensitivity of neuronal signals in MST to the animal's behavioral sensitivity and also measure trial-by-trial correlation between neuronal signals and behavioral choices. Neurons in MST showed a wide range of sensitivities to these complex motion stimuli. Many neurons had sensitivities equal or superior to the monkey's threshold. On the other hand, trial-by-trial correlation between neuronal discharge and choice (“choice probability”) was weak or nonexistent in our data. Together, these results lead us to conclude that MST contains sufficient information for threshold judgments of optic flow; however, the role of MST activity in optic flow discriminations may be less direct than in other visual motion tasks previously described by other laboratories.

scholarly journals The temporal dynamics of heading perception in the presence of moving objects

2016 ◽  
Vol 115 (1) ◽  
pp. 286-300 ◽  
Author(s):  
Oliver W. Layton ◽  
Brett R. Fajen
Keyword(s):  
Optic Flow ◽  
Time Course ◽  
Moving Objects ◽  
Temporal Dynamics ◽  
Temporal Evolution ◽  
Moving Object ◽  
Temporal Area ◽  
Medial Superior Temporal ◽  
Self Motion ◽  
Future Path

Many forms of locomotion rely on the ability to accurately perceive one's direction of locomotion (i.e., heading) based on optic flow. Although accurate in rigid environments, heading judgments may be biased when independently moving objects are present. The aim of this study was to systematically investigate the conditions in which moving objects influence heading perception, with a focus on the temporal dynamics and the mechanisms underlying this bias. Subjects viewed stimuli simulating linear self-motion in the presence of a moving object and judged their direction of heading. Experiments 1 and 2 revealed that heading perception is biased when the object crosses or almost crosses the observer's future path toward the end of the trial, but not when the object crosses earlier in the trial. Nonetheless, heading perception is not based entirely on the instantaneous optic flow toward the end of the trial. This was demonstrated in Experiment 3 by varying the portion of the earlier part of the trial leading up to the last frame that was presented to subjects. When the stimulus duration was long enough to include the part of the trial before the moving object crossed the observer's path, heading judgments were less biased. The findings suggest that heading perception is affected by the temporal evolution of optic flow. The time course of dorsal medial superior temporal area (MSTd) neuron responses may play a crucial role in perceiving heading in the presence of moving objects, a property not captured by many existing models.

scholarly journals Emergence of opposite neurons in a firing-rate model of multisensory integration

2019 ◽  
Author(s):  
Ho Yin Chau ◽  
Wen-Hao Zhang ◽  
Tai Sing Lee
Keyword(s):  
Multisensory Integration ◽  
Neural Circuit ◽  
Hebbian Learning ◽  
Medial Superior Temporal ◽  
Learning Rules ◽  
Vestibular Cues ◽  
The One ◽  
Von Mises ◽  
Integrated Or ◽  
Self Motion

ABSTRACTOpposite neurons, found in macaque dorsal medial superior temporal (MSTd) and ventral intraparietal (VIP) areas, combine visual and vestibular cues of self-motion in opposite ways. A neural circuit recently proposed utilizes opposite neurons to perform causal inference and decide whether the visual and vestibular cues in MSTd and VIP should be integrated or segregated. However, it is unclear how these opposite connections can be formed with biologically realistic learning rules. We propose a network model capable of learning these opposite neurons, using Hebbian and Anti-Hebbian learning rules. The learned neurons are topographically organized and have von Mises-shaped feedforward connections, with tuning properties characteristic of opposite neurons. Our purpose is two-fold: on the one hand, we provide a circuit-level mechanism that explains the properties and formation of opposite neurons; on the other hand, we present a way to extend current theories of multisensory integration to account for appropriate segregation of sensory cues.

scholarly journals Spiral motion selective neurons in area MSTd contribute to judgments of heading

2014 ◽  
Vol 111 (11) ◽  
pp. 2332-2342 ◽  
Author(s):  
Hong Xu ◽  
Pascal Wallisch ◽  
David C. Bradley
Keyword(s):  
Optic Flow ◽  
Discrimination Task ◽  
Flow Patterns ◽  
Choice Probability ◽  
Dorsal Part ◽  
Significant Trial ◽  
Temporal Area ◽  
Spiral Motion ◽  
Medial Superior Temporal ◽  
Self Motion

Self-motion generates patterns of optic flow on the retina. Neurons in the dorsal part of the medial superior temporal area (MSTd) are selective for these optic flow patterns. It has been shown that neurons in this area that are selective for expanding optic flow fields are involved in heading judgments. We wondered how subpopulations of MSTd neurons, those tuned for expansion, rotation or spiral motion, contribute to heading perception. To investigate this question, we recorded from neurons in area MSTd with diverse tuning properties, while the animals performed a heading-discrimination task. We found a significant trial-to-trial correlation (choice probability) between the MSTd neurons and the animals' decision. Neurons in different subpopulations did not differ significantly in terms of their choice probability. Instead, choice probability was strongly related to the sensitivity of the neuron in our sample, regardless of tuning preference. We conclude that a variety of subpopulations of MSTd neurons with different tuning properties contribute to heading judgments.

Sign in / Sign up

Export Citation Format

Share Document