scholarly journals Confidence-Controlled Hebbian Learning Efficiently Extracts Category Membership From Stimuli Encoded in View of a Categorization Task

2021 ◽  
pp. 1-33
Author(s):  
Kevin Berlemont ◽  
Jean-Pierre Nadal
Keyword(s):  
Decision Making ◽  
Hebbian Learning ◽  
Learning Rule ◽  
Descent Method ◽  
Category Membership ◽  
Gradient Descent Method ◽  
Perceptual Decision Making ◽  
Categorization Task ◽  
Attractor Network ◽  
Categorical Information

Abstract In experiments on perceptual decision making, individuals learn a categorization task through trial-and-error protocols. We explore the capacity of a decision-making attractor network to learn a categorization task through reward-based, Hebbian-type modifications of the weights incoming from the stimulus encoding layer. For the latter, we assume a standard layer of a large number of stimu lus-specific neurons. Within the general framework of Hebbian learning, we have hypothesized that the learning rate is modulated by the reward at each trial. Surprisingly, we find that when the coding layer has been optimized in view of the categorization task, such reward-modulated Hebbian learning (RMHL) fails to extract efficiently the category membership. In previous work, we showed that the attractor neural networks' nonlinear dynamics accounts for behavioral confidence in sequences of decision trials. Taking advantage of these findings, we propose that learning is controlled by confidence, as computed from the neural activity of the decision-making attractor network. Here we show that this confidence-controlled, reward-based Hebbian learning efficiently extracts categorical information from the optimized coding layer. The proposed learning rule is local and, in contrast to RMHL, does not require storing the average rewards obtained on previous trials. In addition, we find that the confidence-controlled learning rule achieves near-optimal performance. In accordance with this result, we show that the learning rule approximates a gradient descent method on a maximizing reward cost function.

scholarly journals Confidence-Controlled Hebbian Learning Efficiently Extracts Category Membership from Stimuli Encoded in View of a Categorization Task

2020 ◽  
Author(s):  
Kevin Berlemont ◽  
Jean-Pierre Nadal
Keyword(s):  
Decision Making ◽  
Hebbian Learning ◽  
Learning Rule ◽  
Category Membership ◽  
Trial And Error ◽  
Perceptual Decision Making ◽  
Categorization Task ◽  
Attractor Network ◽  
Categorical Information ◽  
Average Rewards

AbstractIn experiments on perceptual decision-making, individuals learn a categorization task through trial-and-error protocols. We explore the capacity of a decision-making attractor network to learn a categorization task through reward-based, Hebbian type, modifications of the weights incoming from the stimulus encoding layer. For the latter, we assume a standard layer of a large number of stimulus specific neurons. Within the general framework of Hebbian learning, authors have hypothesized that the learning rate is modulated by the reward at each trial. Surprisingly, we find that, when the coding layer has been optimized in view of the categorization task, such reward-modulated Hebbian learning (RMHL) fails to extract efficiently the category membership. In a previous work we showed that the attractor neural networks nonlinear dynamics accounts for behavioral confidence in sequences of decision trials. Taking advantage of these findings, we propose that learning is controlled by confidence, as computed from the neural activity of the decision-making attractor network. Here we show that this confidence-controlled, reward-based, Hebbian learning efficiently extracts categorical information from the optimized coding layer. The proposed learning rule is local, and, in contrast to RMHL, does not require to store the average rewards obtained on previous trials. In addition, we find that the confidence-controlled learning rule achieves near optimal performance.

Pupil-Linked Arousal Biases Evidence Accumulation Toward Desirable Percepts During Perceptual Decision-Making

2021 ◽  
Vol 32 (9) ◽  
pp. 1494-1509
Author(s):  
Yuan Chang Leong ◽  
Roma Dziembaj ◽  
Mark D’Esposito
Keyword(s):  
Decision Making ◽  
Computational Models ◽  
Perceptual Decision Making ◽  
Categorization Task ◽  
Perceptual Decision ◽  
Evidence Accumulation ◽  
Perceptual Judgments ◽  
Significant Events ◽  
Arousal System ◽  
Objective Representation

People’s perceptual reports are biased toward percepts they are motivated to see. The arousal system coordinates the body’s response to motivationally significant events and is well positioned to regulate motivational effects on perceptual judgments. However, it remains unclear whether arousal would enhance or reduce motivational biases. Here, we measured pupil dilation as a measure of arousal while participants ( N = 38) performed a visual categorization task. We used monetary bonuses to motivate participants to perceive one category over another. Even though the reward-maximizing strategy was to perform the task accurately, participants were more likely to report seeing the desirable category. Furthermore, higher arousal levels were associated with making motivationally biased responses. Analyses using computational models suggested that arousal enhanced motivational effects by biasing evidence accumulation in favor of desirable percepts. These results suggest that heightened arousal biases people toward what they want to see and away from an objective representation of the environment.

Reward-Modulated Hebbian Learning of Decision Making

2010 ◽  
Vol 22 (6) ◽  
pp. 1399-1444 ◽  
Author(s):  
Michael Pfeiffer ◽  
Bernhard Nessler ◽  
Rodney J. Douglas ◽  
Wolfgang Maass
Keyword(s):  
Decision Making ◽  
Hebbian Learning ◽  
Learning Rule ◽  
Final Decision ◽  
Bayesian Decision ◽  
Decision Stage ◽  
Fast Learning ◽  
Hebb Rule ◽  
Adaptive Processes ◽  
Perceptron Learning

We introduce a framework for decision making in which the learning of decision making is reduced to its simplest and biologically most plausible form: Hebbian learning on a linear neuron. We cast our Bayesian-Hebb learning rule as reinforcement learning in which certain decisions are rewarded and prove that each synaptic weight will on average converge exponentially fast to the log-odd of receiving a reward when its pre- and postsynaptic neurons are active. In our simple architecture, a particular action is selected from the set of candidate actions by a winner-take-all operation. The global reward assigned to this action then modulates the update of each synapse. Apart from this global reward signal, our reward-modulated Bayesian Hebb rule is a pure Hebb update that depends only on the coactivation of the pre- and postsynaptic neurons, not on the weighted sum of all presynaptic inputs to the postsynaptic neuron as in the perceptron learning rule or the Rescorla-Wagner rule. This simple approach to action-selection learning requires that information about sensory inputs be presented to the Bayesian decision stage in a suitably preprocessed form resulting from other adaptive processes (acting on a larger timescale) that detect salient dependencies among input features. Hence our proposed framework for fast learning of decisions also provides interesting new hypotheses regarding neural nodes and computational goals of cortical areas that provide input to the final decision stage.

scholarly journals Stable thalamocortical learning between medial-dorsal thalamus and cortical attractor networks captures cognitive flexibility

2021 ◽  
Author(s):  
Siwei Qiu
Keyword(s):  
Decision Making ◽  
Prefrontal Cortex ◽  
Distributed Computing ◽  
Neural Circuit ◽  
Hebbian Learning ◽  
Learning Rule ◽  
Computing System ◽  
Synaptic Scaling ◽  
Dorsal Thalamus ◽  
Medial Dorsal

AbstractPrimates and rodents are able to continually acquire, adapt, and transfer knowledge and skill, and lead to goal-directed behavior during their lifespan. For the case when context switches slowly, animals learn via slow processes. For the case when context switches rapidly, animals learn via fast processes. We build a biologically realistic model with modules similar to a distributed computing system. Specifically, we are emphasizing the role of thalamocortical learning on a slow time scale between the prefrontal cortex (PFC) and medial dorsal thalamus (MD). Previous work [1] has already shown experimental evidence supporting classification of cell ensembles in the medial dorsal thalamus, where each class encodes a different context. However, the mechanism by which such classification is learned is not clear. In this work, we show that such learning can be self-organizing in the manner of an automaton (a distributed computing system), via a combination of Hebbian learning and homeostatic synaptic scaling. We show that in the simple case of two contexts, the network with hierarchical structure can do context-based decision making and smooth switching between different contexts. Our learning rule creates synaptic competition [2] between the thalamic cells to create winner-take-all activity. Our theory shows that the capacity of such a learning process depends on the total number of task-related hidden variables, and such a capacity is limited by system size N. We also theoretically derived the effective functional connectivity as a function of an order parameter dependent on the thalamo-cortical coupling structure.Significance StatementAnimals need to adapt to dynamically changing environments and make decisions based on changing contexts. Here we propose a combination of neural circuit structure with learning mechanisms to account for such behaviors. Specifically, we built a reservoir computing network improved by a Hebbian learning rule together with a synaptic scaling learning mechanism between the prefrontal cortex and the medial-dorsal (MD) thalamus. This model shows that MD thalamus is crucial in such context-based decision making. I also make use of dynamical mean field theory to predict the effective neural circuit. Furthermore, theoretical analysis provides a prediction that the capacity of such a network increases with the network size and the total number of tasks-related latent variables.

scholarly journals Learning optimal decisions with confidence

2019 ◽  
Vol 116 (49) ◽  
pp. 24872-24880 ◽  
Author(s):  
Jan Drugowitsch ◽  
André G. Mendonça ◽  
Zachary F. Mainen ◽  
Alexandre Pouget
Keyword(s):  
Decision Making ◽  
Time Pressure ◽  
A Priori ◽  
Learning Rule ◽  
Normative Theory ◽  
Perceptual Decision Making ◽  
Neuronal Populations ◽  
Optimal Decisions ◽  
Optimal Linear ◽  
Rate Of Learning

Diffusion decision models (DDMs) are immensely successful models for decision making under uncertainty and time pressure. In the context of perceptual decision making, these models typically start with two input units, organized in a neuron–antineuron pair. In contrast, in the brain, sensory inputs are encoded through the activity of large neuronal populations. Moreover, while DDMs are wired by hand, the nervous system must learn the weights of the network through trial and error. There is currently no normative theory of learning in DDMs and therefore no theory of how decision makers could learn to make optimal decisions in this context. Here, we derive such a rule for learning a near-optimal linear combination of DDM inputs based on trial-by-trial feedback. The rule is Bayesian in the sense that it learns not only the mean of the weights but also the uncertainty around this mean in the form of a covariance matrix. In this rule, the rate of learning is proportional (respectively, inversely proportional) to confidence for incorrect (respectively, correct) decisions. Furthermore, we show that, in volatile environments, the rule predicts a bias toward repeating the same choice after correct decisions, with a bias strength that is modulated by the previous choice’s difficulty. Finally, we extend our learning rule to cases for which one of the choices is more likely a priori, which provides insights into how such biases modulate the mechanisms leading to optimal decisions in diffusion models.

scholarly journals Learning optimal decisions with confidence

2018 ◽  
Author(s):  
Jan Drugowitsch ◽  
André G. Mendonça ◽  
Zachary F. Mainen ◽  
Alexandre Pouget
Keyword(s):  
Decision Making ◽  
Time Pressure ◽  
A Priori ◽  
Learning Rule ◽  
Normative Theory ◽  
Perceptual Decision Making ◽  
Sensory Inputs ◽  
Optimal Decisions ◽  
Low Dimensional ◽  
The Brain

AbstractDiffusion decision models (DDMs) are immensely successful models for decision-making under uncertainty and time pressure. In the context of perceptual decision making, these models typically start with two input units, organized in a neuron-antineuron pair. In contrast, in the brain, sensory inputs are encoded through the activity of large neuronal populations. Moreover, while DDMs are wired by hand, the nervous system must learn the weights of the network through trial and error. There is currently no normative theory of learning in DDMs and therefore no theory of how decision makers could learn to make optimal decisions in this context. Here, we derive the first such rule for learning a near-optimal linear combination of DDM inputs based on trial-by-trial feedback. The rule is Bayesian in the sense that it learns not only the mean of the weights but also the uncertainty around this mean in the form of a covariance matrix. In this rule, the rate of learning is proportional (resp. inversely proportional) to confidence for incorrect (resp. correct) decisions. Furthermore, we show that, in volatile environments, the rule predicts a bias towards repeating the same choice after correct decisions, with a bias strength that is modulated by the previous choice’s difficulty. Finally, we extend our learning rule to cases for which one of the choices is more likely a priori, which provides new insights into how such biases modulate the mechanisms leading to optimal decisions in diffusion models.Significance StatementPopular models for the tradeoff between speed and accuracy of everyday decisions usually assume fixed, low-dimensional sensory inputs. In contrast, in the brain, these inputs are distributed across larger populations of neurons, and their interpretation needs to be learned from feedback. We ask how such learning could occur and demonstrate that efficient learning is significantly modulated by decision confidence. This modulation predicts a particular dependency pattern between consecutive choices, and provides new insight into how a priori biases for particular choices modulate the mechanisms leading to efficient decisions in these models.

Discarding optimality: Throwing out the baby with the bathwater?

2018 ◽  
Vol 41 ◽  
Author(s):  
Patrick Simen ◽  
Fuat Balcı
Keyword(s):  
Decision Making ◽  
Perceptual Decision Making ◽  
Reward Rate ◽  
Perceptual Decision ◽  
Standard Observer ◽  
Rate Maximization ◽  
Reward Rate Maximization ◽  
Descriptive Standard ◽  
Observer Model

AbstractRahnev & Denison (R&D) argue against normative theories and in favor of a more descriptive “standard observer model” of perceptual decision making. We agree with the authors in many respects, but we argue that optimality (specifically, reward-rate maximization) has proved demonstrably useful as a hypothesis, contrary to the authors’ claims.

LPCD framework: Analytical tool or psychological model?

2018 ◽  
Vol 41 ◽  
Author(s):  
David Danks
Keyword(s):  
Decision Making ◽  
Analytical Tool ◽  
Mathematical Framework ◽  
Bayesian Decision ◽  
Perceptual Decision Making ◽  
Psychological Model ◽  
Psychological Reality ◽  
Psychological Theories ◽  
Target Article ◽  
Bayesian Decision Making

AbstractThe target article uses a mathematical framework derived from Bayesian decision making to demonstrate suboptimal decision making but then attributes psychological reality to the framework components. Rahnev & Denison's (R&D) positive proposal thus risks ignoring plausible psychological theories that could implement complex perceptual decision making. We must be careful not to slide from success with an analytical tool to the reality of the tool components.

Sign in / Sign up

Export Citation Format

Share Document