scholarly journals Neural mechanisms for reward-modulated vector learning and navigation: from social insects to embodied agents

2016 ◽  
Author(s):  
Dennis Goldschmidt ◽  
Poramate Manoonpong ◽  
Sakyasingha Dasgupta
Keyword(s):  
Social Insects ◽  
Path Integration ◽  
Computational Models ◽  
Activity Patterns ◽  
Learning Rule ◽  
Neural Substrates ◽  
Neural Mechanisms ◽  
Circular Arrays ◽  
Current Location ◽  
Integration Mechanism

AbstractDespite their small size, insect brains are able to produce robust and efficient navigation in complex environments. Specifically in social insects, such as ants and bees, these navigational capabilities are guided by orientation directing vectors generated by a process called path integration. During this process, they integrate compass and odometric cues to estimate their current location as a vector, called home vector for guiding them back home on a straight path. They further acquire and retrieve path integration-based vector memories anchored globally to the nest or visual landmarks. Although existing computational models reproduced similar behaviors, they largely neglected evidence for possible neural substrates underlying the generated behavior. Therefore, we present here a model of neural mechanisms in a modular closed-loop control - enabling vector navigation in embodied agents. The model consists of a path integration mechanism, reward-modulated global and local vector learning, random search, and action selection. The path integration mechanism integrates compass and odometric cues to compute a vectorial representation of the agent’s current location as neural activity patterns in circular arrays. A reward-modulated learning rule enables the acquisition of vector memories by associating the local food reward with the path integration state. A motor output is computed based on the combination of vector memories and random exploration. In sim-ulation, we show that the neural mechanisms enable robust homing and localization, even in the presence of external sensory noise. The proposed learning rules lead to goal-directed navigation and route formation performed under realistic conditions. This provides an explanation for, how view-based navigational strategies are guided by path integration. Consequently, we provide a novel approach for vector learning and navigation in a simulated embodied agent linking behavioral observations to their possible underlying neural substrates.Author SummaryDesert ants survive under harsh conditions by foraging for food in temperatures over 60° C. In this extreme environment, they cannot, like other ants, use pheromones to track their long-distance journeys back to their nests. Instead they apply a computation called path integration, which involves integrating skylight compass and odometric stimuli to estimate its current position. Path integration is not only used to return safely to their nests, but also helps in learning so-called vector memories. Such memories are sufficient to produce goal-directed and landmark-guided navigation in social insects. How can small insect brains generate such complex behaviors? Computational models are often useful for studying behavior and their underlying control mechanisms. Here we present a novel computational framework for the acquisition and expression of vector memories based on path integration. It consists of multiple neural networks and a reward-based learning rule, where vectors are represented by the activity patterns of circular arrays. Our model not only reproduces goal-directed navigation and route formation in a simulated agent, but also offers predictions about neural implementations. Taken together, we believe that it demonstrates the first complete model of vector-guided navigation linking observed behaviors of navigating social insects to their possible underlying neural mechanisms.

scholarly journals Path integration in large-scale space and with novel geometries: Comparing Vector Addition and Encoding-Error Models

2019 ◽  
Author(s):  
S. K. Harootonian ◽  
R. C. Wilson ◽  
L. Hejtmánek ◽  
E. M. Ziskin ◽  
A. D. Ekstrom
Keyword(s):  
Path Integration ◽  
Large Scale ◽  
Computational Models ◽  
Systematic Errors ◽  
Scale Space ◽  
Simple Explanation ◽  
Vector Addition ◽  
Current Location ◽  
Spatial Geometry ◽  
Homing Vector

AbstractPath integration is thought to rely on vestibular and proprioceptive cues yet most studies in humans involve primarily visual input, providing limited insight into their contributions. We developed a paradigm involving walking in an omnidirectional treadmill in which participants were guided on two legs of a triangle and then found their back way to origin. In Experiment 1, we tested a range of different triangle types while keeping distance relatively constant to determine the influence of spatial geometry. Participants overshot the angle they needed to turn and undershot the distance they needed to walk, with no consistent effect of triangle type. In Experiment 2, we manipulated distance while keeping angle relatively constant to determine how path integration operated over both shorter and longer distances. Participants underestimated the distance they needed to walk to the origin, with error increasing as a function of the walked distance. To attempt to account for our findings, we developed computational models involving vector addition, the second of which included terms for the influence of past trials on the current one. We compared against a previously developed model of human path integration, the Encoding Error model. We found that the vector addition models captured the tendency of participants to under-encode guided legs of the triangles and an influence of past trials on current trials. Together, our findings expand our understanding of body-based contributions to human path integration, further suggesting the value of vector addition models in understanding these important components of human navigation.Author SummaryHow do we remember where we have been? One important mechanism for doing so is called path integration, which refers to the ability to track one’s position in space with only self-motion cues. By tracking the direction and distance we have walked, we can create a mental arrow from the current location to the origin, termed the homing vector. Previous studies have shown that the homing vector is subject to systematic distortions depending on previously experienced paths, yet what influences these patterns of errors, particularly in humans, remains uncertain. In this study, we compare two models of path integration based on participants walking two legs of a triangle without vision and then completing the third leg based on their estimate of the homing vector. We found no effect of triangle shape on systematic errors, while path length scaled the systematic errors logarithmically, similar to Weber-Fechner law. While we show that both models captured participant’s behavior, a model based on vector addition best captured the patterns of error in the homing vector. Our study therefore has important implications for how humans track their location, suggesting that vector-based models provide a reasonable and simple explanation for how we do so.

Osmosensitivity of rat third ventricle and interactions with angiotensin

1979 ◽  
Vol 236 (1) ◽  
pp. R75-R82 ◽  
Author(s):  
J. Buggy ◽  
W. E. Hoffman ◽  
M. I. Phillips ◽  
A. E. Fisher ◽  
A. K. Johnson
Keyword(s):  
Third Ventricle ◽  
Neural Substrates ◽  
Short Latency ◽  
Neural Mechanisms ◽  
Nacl Solution ◽  
Pressor Responses ◽  
Sucrose Solutions ◽  
Osmotic Stimulation ◽  
Behavioral Dissociation ◽  
Drinking Fluid

Injections of hyperosmotic solutions (1- to 5-microliter injections of NaCl or sucrose solutions ranging in osmolarity from 0.34 to 0.90 osmol/l) into the anteroventral third ventricle (AV3V) of rats resulted in short latency drinking antidiuretic, and pressor responses. AV3V injections or infusions of combined angiotensin-hyperosmotic NaCl solution did not result in drinking greater than the sum of drinking to angiotensin and hyperosmotic NaCl separately administered. Differences in water versus saline drinking fluid preferences provided a behavioral dissociation of angiotensin and hyperosmotic sensitive neural mechanisms. Comparison of AV3V and lateral preoptic injection sites provided an additional separation since angiotensin was equally effective at both sites whereas osmotic stimulation was more effective at the AV3V site. AV3V lesions have previously been reported to abolish drinking, antidiuretic, and pressor responses to angiotensin and hyperosmotic stimulation. The data reported here provide additional evidence that angiotensin and hyperosmotic stimuli may both act on tissue surrounding AV3V but suggest that the neural substrates for these stimuli are not identical.

scholarly journals Exploring the role of expectations and stimulus relevance on stimulus-specific neural representations and conscious report

2019 ◽  
Vol 2019 (1) ◽  
Author(s):  
Erik L Meijs ◽  
Pim Mostert ◽  
Heleen A Slagter ◽  
Floris P de Lange ◽  
Simon van Gaal
Keyword(s):  
Attentional Blink ◽  
Neural Activity ◽  
Subjective Experience ◽  
Activity Patterns ◽  
Neural Mechanisms ◽  
Top Down ◽  
Neural Representations ◽  
Dependent Signal ◽  
Task Irrelevant

Abstract Subjective experience can be influenced by top-down factors, such as expectations and stimulus relevance. Recently, it has been shown that expectations can enhance the likelihood that a stimulus is consciously reported, but the neural mechanisms supporting this enhancement are still unclear. We manipulated stimulus expectations within the attentional blink (AB) paradigm using letters and combined visual psychophysics with magnetoencephalographic (MEG) recordings to investigate whether prior expectations may enhance conscious access by sharpening stimulus-specific neural representations. We further explored how stimulus-specific neural activity patterns are affected by the factors expectation, stimulus relevance and conscious report. First, we show that valid expectations about the identity of an upcoming stimulus increase the likelihood that it is consciously reported. Second, using a series of multivariate decoding analyses, we show that the identity of letters presented in and out of the AB can be reliably decoded from MEG data. Third, we show that early sensory stimulus-specific neural representations are similar for reported and missed target letters in the AB task (active report required) and an oddball task in which the letter was clearly presented but its identity was task-irrelevant. However, later sustained and stable stimulus-specific representations were uniquely observed when target letters were consciously reported (decision-dependent signal). Fourth, we show that global pre-stimulus neural activity biased perceptual decisions for a ‘seen’ response. Fifth and last, no evidence was obtained for the sharpening of sensory representations by top-down expectations. We discuss these findings in light of emerging models of perception and conscious report highlighting the role of expectations and stimulus relevance.

scholarly journals Primary Cognitive Categories Are Determined by Their Invariances

2020 ◽  
Vol 11 ◽  
Author(s):  
Peter Gärdenfors
Keyword(s):  
Computational Models ◽  
External World ◽  
Neural Mechanisms ◽  
Sensory Signals ◽  
The World

The world as we perceive it is structured into objects, actions and places that form parts of events. In this article, my aim is to explain why these categories are cognitively primary. From an empiricist and evolutionary standpoint, it is argued that the reduction of the complexity of sensory signals is based on the brain's capacity to identify various types of invariances that are evolutionarily relevant for the activities of the organism. The first aim of the article is to explain why places, object and actions are primary cognitive categories in our constructions of the external world. It is shown that the invariances that determine these categories have their separate characteristics and that they are, by and large, independent of each other. This separation is supported by what is known about the neural mechanisms. The second aim is to show that the category of events can be analyzed as being constituted of the primary categories. The category of numbers is briefly discussed. Some implications for computational models of the categories are also presented.

scholarly journals An Experimental Swallow Evoked Potential Protocol to Investigate the Neural Substrates of Swallowing

OTO Open ◽  
2020 ◽  
Vol 4 (1) ◽  
pp. 2473974X2091354
Author(s):  
Ashley Kloepper ◽  
Joseph Arnold ◽  
Alexis Ruffolo ◽  
Brian Kinealy ◽  
Chandler Haxton ◽  
...  
Keyword(s):  
Evoked Potential ◽  
Neural Substrates ◽  
Neural Mechanisms ◽  
Experimental Protocol ◽  
Adult Rats ◽  
Invasive Approach ◽  
Sensory Evoked Potential ◽  
Age Related ◽  
Health And Disease ◽  
Sensory Evoked

Advancement in dysphagia intervention is hindered by our lack of understanding of the neural mechanisms of swallowing in health and disease. Evoking and understanding neural activity in response to normal and disordered swallowing is essential to bridge this knowledge gap. Building on sensory evoked potential methodology, we developed a minimally invasive approach to generate swallow evoked potentials (SwEPs) in response to repetitive swallowing induced by citric acid stimulation of the oropharynx in lightly anesthetized healthy adult rats. The SwEP waveform consisted of 8 replicable peaks within 10 milliseconds immediately preceding the onset of electromyographic swallowing activity. Methodology refinement is underway with healthy rats to establish normative SwEP waveform morphology before proceeding to models of advanced aging and age-related neurodegenerative diseases. Ultimately, we envision that this experimental protocol may unmask the pathologic neural substrates contributing to dysphagia to accelerate the discovery of targeted therapeutics.

scholarly journals Computational Models of Neuromodulation

1998 ◽  
Vol 10 (4) ◽  
pp. 771-805 ◽  
Author(s):  
Jean-Marc Fellous ◽  
Christiane Linster
Keyword(s):  
Sensory Processing ◽  
Information Integration ◽  
Computational Models ◽  
Neural Substrates ◽  
Pattern Generation ◽  
Neural Models ◽  
Functional Roles ◽  
Modeling Studies ◽  
And Control

Computational modeling of neural substrates provides an excellent theoretical framework for the understanding of the computational roles of neuromodulation. In this review, we illustrate, with a large number of modeling studies, the specific computations performed by neuromodulation in the context of various neural models of invertebrate and vertebrate preparations. We base our characterization of neuromodulations on their computational and functional roles rather than on anatomical or chemical criteria. We review the main framework in which neuromodulation has been studied theoretically (central pattern generation and oscillations, sensory processing, memory and information integration). Finally, we present a detailed mathematical overview of how neuromodulation has been implemented at the single cell and network levels in modeling studies. Overall, neuromodulation is found to increase and control computational complexity.

scholarly journals Beyond cones: an improved model of whisker bending based on measured mechanics and tapering

2016 ◽  
Vol 116 (2) ◽  
pp. 812-824 ◽  
Author(s):  
Samuel Andrew Hires ◽  
Adam Schuyler ◽  
Jonathan Sy ◽  
Vincent Huang ◽  
Isis Wyche ◽  
...  
Keyword(s):  
Neural Activity ◽  
Computational Models ◽  
Sensory Input ◽  
Activity Patterns ◽  
Object Localization ◽  
Homogeneous Cone ◽  
Key Drivers ◽  
Improved Model ◽  
Sense Of Touch ◽  
Object Interactions

The sense of touch is represented by neural activity patterns evoked by mechanosensory input forces. The rodent whisker system is exceptional for studying the neurophysiology of touch in part because these forces can be precisely computed from video of whisker deformation. We evaluate the accuracy of a standard model of whisker bending, which assumes quasi-static dynamics and a linearly tapered conical profile, using controlled whisker deflections. We find significant discrepancies between model and experiment: real whiskers bend more than predicted upon contact at locations in the middle of the whisker and less at distal locations. Thus whiskers behave as if their stiffness near the base and near the tip is larger than expected for a homogeneous cone. We assess whether contact direction, friction, inhomogeneous elasticity, whisker orientation, or nonconical shape could explain these deviations. We show that a thin-middle taper of mouse whisker shape accounts for the majority of this behavior. This taper is conserved across rows and columns of the whisker array. The taper has a large effect on the touch-evoked forces and the ease with which whiskers slip past objects, which are key drivers of neural activity in tactile object localization and identification. This holds for orientations with intrinsic whisker curvature pointed toward, away from, or down from objects, validating two-dimensional models of simple whisker-object interactions. The precision of computational models relating sensory input forces to neural activity patterns can be quantitatively enhanced by taking thin-middle taper into account with a simple corrective function that we provide.

scholarly journals Inhibition underlies fast undulatory locomotion in C. elegans

2020 ◽  
Author(s):  
Lan Deng ◽  
Jack Denham ◽  
Charu Arya ◽  
Omer Yuval ◽  
Netta Cohen ◽  
...  
Keyword(s):  
Computational Models ◽  
Body Wall ◽  
Activity Patterns ◽  
Wild Type ◽  
C Elegans ◽  
Body Wall Muscles ◽  
Undulatory Locomotion ◽  
Gaba Transmission ◽  
Cross Inhibition

AbstractInhibition plays important roles in modulating the neural activities of sensory and motor systems at different levels from synapses to brain regions. To achieve coordinated movement, motor systems produce alternating contraction of antagonist muscles, whether along the body axis or within and among limbs. In the nematode C. elegans, a small network involving excitatory cholinergic and inhibitory GABAergic motoneurons generates the dorsoventral alternation of body-wall muscles that supports undulatory locomotion. Inhibition has been suggested to be necessary for backward undulation because mutants that are defective in GABA transmission exhibit a shrinking phenotype in response to a harsh touch to the head, whereas wild-type animals produce a backward escape response. Here, we demonstrate that the shrinking phenotype is exhibited by wild-type as well as mutant animals in response to harsh touch to the head or tail, but only GABA transmission mutants show slow locomotion after stimulation. Impairment of GABA transmission, either genetically or optogenetically, induces lower undulation frequency and lower translocation speed during crawling and swimming in both directions. The activity patterns of GABAergic motoneurons are different during low and high undulation frequencies. During low undulation frequency, GABAergic VD and DD motoneurons show similar activity patterns, while during high undulation frequency, their activity alternates. The experimental results suggest at least three non-mutually exclusive roles for inhibition that could underlie fast undulatory locomotion in C. elegans, which we tested with computational models: cross-inhibition or disinhibition of body-wall muscles, or inhibitory reset.Significance StatementInhibition serves multiple roles in the generation, maintenance, and modulation of the locomotive program and supports the alternating activation of antagonistic muscles. When the locomotor frequency increases, more inhibition is required. To better understand the role of inhibition in locomotion, we used C. elegans as an animal model, and challenged a prevalent hypothesis that cross-inhibition supports the dorsoventral alternation. We find that inhibition is related to the speed rather than the direction of locomotion and demonstrate that inhibition is unnecessary for muscle alternation during slow undulation in either direction but crucial to sustain rapid dorsoventral alternation. We combined calcium imaging of motoneurons and muscle with computational models to test hypotheses for the role of inhibition in locomotion.

scholarly journals Temperature compensation in a small rhythmic circuit

eLife ◽  
2020 ◽  
Vol 9 ◽  
Author(s):  
Leandro M Alonso ◽  
Eve Marder
Keyword(s):  
Computational Models ◽  
Characteristic Temperature ◽  
Activity Patterns ◽  
Membrane Conductance ◽  
Ionic Channels ◽  
Pyloric Network ◽  
Temperature Ranges ◽  
High And Low Temperatures ◽  
Kinetics Of ◽  
A Current

Temperature affects the conductances and kinetics of the ionic channels that underlie neuronal activity. Each membrane conductance has a different characteristic temperature sensitivity, which raises the question of how neurons and neuronal circuits can operate robustly over wide temperature ranges. To address this, we employed computational models of the pyloric network of crabs and lobsters. We produced multiple different models that exhibit a triphasic pyloric rhythm over a range of temperatures and explored the dynamics of their currents and how they change with temperature. Temperature can produce smooth changes in the relative contributions of the currents to neural activity so that neurons and networks undergo graceful transitions in the mechanisms that give rise to their activity patterns. Moreover, responses of the models to deletions of a current can be different at high and low temperatures, indicating that even a well-defined genetic or pharmacological manipulation may produce qualitatively distinct effects depending on the temperature.

scholarly journals Feature-reweighted RSA: A method for improving the fit between computational models, brains, and behavior

2021 ◽  
Author(s):  
Philipp Kaniuth ◽  
Martin N. Hebart
Keyword(s):  
Model Selection ◽  
Computational Models ◽  
Model Comparison ◽  
Brain Activity ◽  
Activity Patterns ◽  
Behavioral Data ◽  
Novel Approach ◽  
Improve Model ◽  
And Behavior ◽  
Model Features

AbstractRepresentational Similarity Analysis (RSA) has emerged as a popular method for relating representational spaces from human brain activity, behavioral data, and computational models. RSA is based on the comparison of representational dissimilarity matrices (RDM), which characterize the pairwise dissimilarities of all conditions across all features (e.g. fMRI voxels or units of a model). However, classical RSA treats each feature as equally important. This ‘equal weights’ assumption contrasts with the flexibility of multivariate decoding, which reweights individual features for predicting a target variable. As a consequence, classical RSA may lead researchers to underestimate the correspondence between a model and a brain region and, for model comparison, it may lead to selecting the inferior model. While previous work has suggested that reweighting can improve model selection in RSA, it has remained unclear to what extent these results generalize across datasets and data modalities. To fill this gap, the aim of this work is twofold: First, utilizing a range of publicly available datasets and three popular deep neural networks (DNNs), we seek to broadly test feature-reweighted RSA (FR-RSA) applied to computational models and reveal the extent to which reweighting model features improves RDM correspondence and affects model selection. Second, we propose voxel-reweighted RSA, a novel use case of FR-RSA that reweights fMRI voxels, mirroring the rationale of multivariate decoding of optimally combining voxel activity patterns. We find that reweighting individual model units (1) markedly improves the fit between model RDMs and target RDMs derived from several fMRI and behavioral datasets and (2) affects model selection, highlighting the importance of considering FR-RSA. For voxel-reweighted RSA, improvements in RDM correspondence were even more pronounced, demonstrating the utility of this novel approach. We additionally demonstrate that classical noise ceilings can be exceeded when FR-RSA is applied and propose an updated approach for their computation. Taken together, our results broadly validate the use of FR-RSA for improving the fit between computational models, brain and behavioral data, possibly allowing us to better adjudicate between competing computational models. Further, our results suggest that FR-RSA applied to brain measurement channels could become an important new method to assess the match between representational spaces.

Sign in / Sign up

Export Citation Format

Share Document