scholarly journals Learning optimal decisions with confidence

2018 ◽  
Author(s):  
Jan Drugowitsch ◽  
André G. Mendonça ◽  
Zachary F. Mainen ◽  
Alexandre Pouget
Keyword(s):  
Decision Making ◽  
Time Pressure ◽  
A Priori ◽  
Learning Rule ◽  
Normative Theory ◽  
Perceptual Decision Making ◽  
Sensory Inputs ◽  
Optimal Decisions ◽  
Low Dimensional ◽  
The Brain

AbstractDiffusion decision models (DDMs) are immensely successful models for decision-making under uncertainty and time pressure. In the context of perceptual decision making, these models typically start with two input units, organized in a neuron-antineuron pair. In contrast, in the brain, sensory inputs are encoded through the activity of large neuronal populations. Moreover, while DDMs are wired by hand, the nervous system must learn the weights of the network through trial and error. There is currently no normative theory of learning in DDMs and therefore no theory of how decision makers could learn to make optimal decisions in this context. Here, we derive the first such rule for learning a near-optimal linear combination of DDM inputs based on trial-by-trial feedback. The rule is Bayesian in the sense that it learns not only the mean of the weights but also the uncertainty around this mean in the form of a covariance matrix. In this rule, the rate of learning is proportional (resp. inversely proportional) to confidence for incorrect (resp. correct) decisions. Furthermore, we show that, in volatile environments, the rule predicts a bias towards repeating the same choice after correct decisions, with a bias strength that is modulated by the previous choice’s difficulty. Finally, we extend our learning rule to cases for which one of the choices is more likely a priori, which provides new insights into how such biases modulate the mechanisms leading to optimal decisions in diffusion models.Significance StatementPopular models for the tradeoff between speed and accuracy of everyday decisions usually assume fixed, low-dimensional sensory inputs. In contrast, in the brain, these inputs are distributed across larger populations of neurons, and their interpretation needs to be learned from feedback. We ask how such learning could occur and demonstrate that efficient learning is significantly modulated by decision confidence. This modulation predicts a particular dependency pattern between consecutive choices, and provides new insight into how a priori biases for particular choices modulate the mechanisms leading to efficient decisions in these models.

scholarly journals Learning optimal decisions with confidence

2019 ◽  
Vol 116 (49) ◽  
pp. 24872-24880 ◽  
Author(s):  
Jan Drugowitsch ◽  
André G. Mendonça ◽  
Zachary F. Mainen ◽  
Alexandre Pouget
Keyword(s):  
Decision Making ◽  
Time Pressure ◽  
A Priori ◽  
Learning Rule ◽  
Normative Theory ◽  
Perceptual Decision Making ◽  
Neuronal Populations ◽  
Optimal Decisions ◽  
Optimal Linear ◽  
Rate Of Learning

Diffusion decision models (DDMs) are immensely successful models for decision making under uncertainty and time pressure. In the context of perceptual decision making, these models typically start with two input units, organized in a neuron–antineuron pair. In contrast, in the brain, sensory inputs are encoded through the activity of large neuronal populations. Moreover, while DDMs are wired by hand, the nervous system must learn the weights of the network through trial and error. There is currently no normative theory of learning in DDMs and therefore no theory of how decision makers could learn to make optimal decisions in this context. Here, we derive such a rule for learning a near-optimal linear combination of DDM inputs based on trial-by-trial feedback. The rule is Bayesian in the sense that it learns not only the mean of the weights but also the uncertainty around this mean in the form of a covariance matrix. In this rule, the rate of learning is proportional (respectively, inversely proportional) to confidence for incorrect (respectively, correct) decisions. Furthermore, we show that, in volatile environments, the rule predicts a bias toward repeating the same choice after correct decisions, with a bias strength that is modulated by the previous choice’s difficulty. Finally, we extend our learning rule to cases for which one of the choices is more likely a priori, which provides insights into how such biases modulate the mechanisms leading to optimal decisions in diffusion models.

Sensory evidence integration and action initiation occur in parallel during perceptual decisions

2020 ◽  
Author(s):  
Lluís Hernández-Navarro ◽  
Ainhoa Hermoso-Mendizabal ◽  
Daniel Duque ◽  
Alexandre Hyafil ◽  
Jaime de la Rocha
Keyword(s):  
Time Pressure ◽  
Reaction Times ◽  
Stimulus Presentation ◽  
Perceptual Decision Making ◽  
Evidence Accumulation ◽  
Action Preparation ◽  
Action Model ◽  
The Brain ◽  
Evidence Integration ◽  
Action Initiation

It is commonly assumed that, during perceptual decisions, the brain integrates stimulus evidence until reaching a decision, and then performs the response. There are conditions, however (e.g. time pressure), in which the initiation of the response must be prepared in anticipation of the stimulus presentation. It is therefore not clear when the timing and the choice of perceptual responses depend exclusively on evidence accumulation, or when preparatory motor signals may interfere with this process. Here, we find that, in a free reaction time auditory discrimination task in rats, the timing of fast responses does not depend on the stimulus, although the choices do, suggesting a decoupling of the mechanisms of action initiation and choice selection. This behavior is captured by a novel model, the Parallel Sensory Integration and Action Model (PSIAM), in which response execution is triggered whenever one of two processes, Action Initiation or Evidence Accumulation, reaches a bound, while choice category is always set by the latter. Based on this separation, the model accurately predicts the distribution of reaction times when the stimulus is omitted, advanced or delayed. Furthermore, we show that changes in Action Initiation mediates both post-error slowing and a gradual slowing of the responses within each session. Overall, these results extend the standard models of perceptual decision-making, and shed a new light on the interaction between action preparation and evidence accumulation.

scholarly journals Flexible categorization in perceptual decision making

2020 ◽  
Author(s):  
Genís Prat-Ortega ◽  
Klaus Wimmer ◽  
Alex Roxin ◽  
Jaime de la Rocha
Keyword(s):  
Decision Making ◽  
Large Body ◽  
Network Models ◽  
Perceptual Decision Making ◽  
Drift Diffusion ◽  
Perceptual Decision ◽  
Attractor Dynamics ◽  
Attractor Model ◽  
Winner Take All ◽  
The Brain

AbstractPerceptual decisions require the brain to make categorical choices based on accumulated sensory evidence. The underlying computations have been studied using either phenomenological drift diffusion models or neurobiological network models exhibiting winner-take-all attractor dynamics. Although both classes of models can account for a large body of experimental data, it remains unclear to what extent their dynamics are qualitatively equivalent. Here we show that, unlike the drift diffusion model, the attractor model can operate in different integration regimes: an increase in the stimulus fluctuations or the stimulus duration promotes transitions between decision-states leading to a crossover between weighting mostly early evidence (primacy regime) to weighting late evidence (recency regime). Between these two limiting cases, we found a novel regime, which we name flexible categorization, in which fluctuations are strong enough to reverse initial categorizations, but only if they are incorrect. This asymmetry in the reversing probability results in a non-monotonic psychometric curve, a novel and distinctive feature of the attractor model. Finally, we show psychophysical evidence for the crossover between integration regimes predicted by the attractor model and for the relevance of this new regime. Our findings point to correcting transitions as an important yet overlooked feature of perceptual decision making.

scholarly journals Distributed context-dependent choice information in mouse dorsal-parietal cortex

2021 ◽  
Author(s):  
Javier Orlandi ◽  
Mohammad Adbolrahmani ◽  
Ryo Aoki ◽  
Dmitry Lyamzin ◽  
Andrea Benucci
Keyword(s):  
Decision Making ◽  
Decision Variable ◽  
Visual Stream ◽  
Posterior Cortex ◽  
Neural Data ◽  
Context Dependent ◽  
Cortical Regions ◽  
Low Dimensional ◽  
Independent Decision ◽  
The Brain

Abstract Choice information appears in the brain as distributed signals with top-down and bottom-up components that together support decision-making computations. In sensory and associative cortical regions, the presence of choice signals, their strength, and area specificity are known to be elusive and changeable, limiting a cohesive understanding of their computational significance. In this study, examining the mesoscale activity in mouse posterior cortex during a complex visual discrimination task, we found that broadly distributed choice signals defined a decision variable in a low-dimensional embedding space of multi-area activations, particularly along the ventral visual stream. The subspace they defined was near-orthogonal to concurrently represented sensory and motor-related activations, and it was modulated by task difficulty and contextually by the animals’ attention state. To mechanistically relate choice representations to decision-making computations, we trained recurrent neural networks with the animals’ choices and found an equivalent decision variable whose context-dependent dynamics agreed with that of the neural data. In conclusion, our results demonstrated an independent decision variable broadly represented in the posterior cortex, controlled by task features and cognitive demands. Its dynamics reflected decision computations, possibly linked to context-dependent feedback signals used for probabilistic-inference computations in variable animal-environment interactions.

scholarly journals Strategy-dependent effects of working-memory limitations on human perceptual decision-making

2021 ◽  
Author(s):  
Kyra Schapiro ◽  
Kresimir Josic ◽  
Zachary Kilpatrick ◽  
Joshua I Gold
Keyword(s):  
Decision Making ◽  
Working Memory ◽  
Decision Variable ◽  
Perceptual Decision Making ◽  
Perceptual Decision ◽  
Human Psychophysics ◽  
Decision Variables ◽  
The Impact ◽  
The Brain ◽  
Over Time

Deliberative decisions based on an accumulation of evidence over time depend on working memory, and working memory has limitations, but how these limitations affect deliberative decision-making is not understood. We used human psychophysics to assess the impact of working-memory limitations on the fidelity of a continuous decision variable. Participants decided the average location of multiple visual targets. This computed, continuous decision variable degraded with time and capacity in a manner that depended critically on the strategy used to form the decision variable. This dependence reflected whether the decision variable was computed either: 1) immediately upon observing the evidence, and thus stored as a single value in memory; or 2) at the time of the report, and thus stored as multiple values in memory. These results provide important constraints on how the brain computes and maintains temporally dynamic decision variables.

scholarly journals A cerebellar role in evidence-guided decision-making

2018 ◽  
Author(s):  
Ben Deverett ◽  
Sue Ann Koay ◽  
Marlies Oostland ◽  
Samuel S.-H. Wang
Keyword(s):  
Decision Making ◽  
Perceptual Decision Making ◽  
Perceptual Decision ◽  
Two Photon ◽  
Related Information ◽  
Distributed Computations ◽  
Lateral Posterior ◽  
Crus I ◽  
The Brain ◽  
Decision Errors

To make successful evidence-based decisions, the brain must rapidly and accurately transform sensory inputs into specific goal-directed behaviors. Most experimental work on this subject has focused on forebrain mechanisms. Here we show that during perceptual decision-making over a period of seconds, decision-, sensory-, and error-related information converge on the lateral posterior cerebellum in crus I, a structure that communicates bidirectionally with numerous forebrain regions. We trained mice on a novel evidence-accumulation task and demonstrated that cerebellar inactivation reduces behavioral accuracy without impairing motor parameters of action. Using two-photon calcium imaging, we found that Purkinje cell somatic activity encoded choice- and evidence-related variables. Decision errors were represented by dendritic calcium spikes, which are known to drive plasticity. We propose that cerebellar circuitry may contribute to the set of distributed computations in the brain that support accurate perceptual decision-making.

scholarly journals Manifold-tiling Localized Receptive Fields are Optimal in Similarity-preserving Neural Networks

2018 ◽  
Author(s):  
Anirvan M. Sengupta ◽  
Mariano Tepper ◽  
Cengiz Pehlevan ◽  
Alexander Genkin ◽  
Dmitri B. Chklovskii
Keyword(s):  
Neural Networks ◽  
Receptive Fields ◽  
Small Neighborhood ◽  
Place Cells ◽  
Optimal Solutions ◽  
Stimulus Space ◽  
Sensory Inputs ◽  
Similarity Preserving ◽  
Low Dimensional ◽  
The Brain

AbstractMany neurons in the brain, such as place cells in the rodent hippocampus, have localized receptive fields, i.e., they respond to a small neighborhood of stimulus space. What is the functional significance of such representations and how can they arise? Here, we propose that localized receptive fields emerge in similarity-preserving networks of rectifying neurons that learn low-dimensional manifolds populated by sensory inputs. Numerical simulations of such networks on standard datasets yield manifold-tiling localized receptive fields. More generally, we show analytically that, for data lying on symmetric manifolds, optimal solutions of objectives, from which similarity-preserving networks are derived, have localized receptive fields. Therefore, nonnegative similarity-preserving mapping (NSM) implemented by neural networks can model representations of continuous manifolds in the brain.

scholarly journals Confidence-Controlled Hebbian Learning Efficiently Extracts Category Membership From Stimuli Encoded in View of a Categorization Task

2021 ◽  
pp. 1-33
Author(s):  
Kevin Berlemont ◽  
Jean-Pierre Nadal
Keyword(s):  
Decision Making ◽  
Hebbian Learning ◽  
Learning Rule ◽  
Descent Method ◽  
Category Membership ◽  
Gradient Descent Method ◽  
Perceptual Decision Making ◽  
Categorization Task ◽  
Attractor Network ◽  
Categorical Information

Abstract In experiments on perceptual decision making, individuals learn a categorization task through trial-and-error protocols. We explore the capacity of a decision-making attractor network to learn a categorization task through reward-based, Hebbian-type modifications of the weights incoming from the stimulus encoding layer. For the latter, we assume a standard layer of a large number of stimu lus-specific neurons. Within the general framework of Hebbian learning, we have hypothesized that the learning rate is modulated by the reward at each trial. Surprisingly, we find that when the coding layer has been optimized in view of the categorization task, such reward-modulated Hebbian learning (RMHL) fails to extract efficiently the category membership. In previous work, we showed that the attractor neural networks' nonlinear dynamics accounts for behavioral confidence in sequences of decision trials. Taking advantage of these findings, we propose that learning is controlled by confidence, as computed from the neural activity of the decision-making attractor network. Here we show that this confidence-controlled, reward-based Hebbian learning efficiently extracts categorical information from the optimized coding layer. The proposed learning rule is local and, in contrast to RMHL, does not require storing the average rewards obtained on previous trials. In addition, we find that the confidence-controlled learning rule achieves near-optimal performance. In accordance with this result, we show that the learning rule approximates a gradient descent method on a maximizing reward cost function.

Dynamical Foundations of the Neural Circuit for Bayesian Decision Making

2009 ◽  
Vol 102 (1) ◽  
pp. 1-6 ◽  
Author(s):  
Kenji Morita
Keyword(s):  
Decision Making ◽  
Bayesian Inference ◽  
Neural Circuits ◽  
Neural Circuit ◽  
Bayesian Decision ◽  
Biophysical Modeling ◽  
Perceptual Decision Making ◽  
Basic Ideas ◽  
Bayesian Decision Making ◽  
The Brain

On the basis of accumulating behavioral and neural evidences, it has recently been proposed that the brain neural circuits of humans and animals are equipped with several specific properties, which ensure that perceptual decision making implemented by the circuits can be nearly optimal in terms of Bayesian inference. Here, I introduce the basic ideas of such a proposal and discuss its implications from the standpoint of biophysical modeling developed in the framework of dynamical systems.

Sign in / Sign up

Export Citation Format

Share Document