scholarly journals Rules warp feature encoding in decision-making circuits

PLoS Biology ◽  
2020 ◽  
Vol 18 (11) ◽  
pp. e3000951 ◽  
Author(s):  
R. Becket Ebitz ◽  
Jiaxin Cindy Tu ◽  
Benjamin Y. Hayden
Keyword(s):  
Decision Making ◽  
Dorsal Striatum ◽  
Energetic Cost ◽  
Rule Based ◽  
Stimulus Response ◽  
Feature Encoding ◽  
Computational Simplicity ◽  
Physically Identical ◽  
The Brain ◽  
Current Rule

We have the capacity to follow arbitrary stimulus–response rules, meaning simple policies that guide our behavior. Rule identity is broadly encoded across decision-making circuits, but there are less data on how rules shape the computations that lead to choices. One idea is that rules could simplify these computations. When we follow a rule, there is no need to encode or compute information that is irrelevant to the current rule, which could reduce the metabolic or energetic demands of decision-making. However, it is not clear if the brain can actually take advantage of this computational simplicity. To test this idea, we recorded from neurons in 3 regions linked to decision-making, the orbitofrontal cortex (OFC), ventral striatum (VS), and dorsal striatum (DS), while macaques performed a rule-based decision-making task. Rule-based decisions were identified via modeling rules as the latent causes of decisions. This left us with a set of physically identical choices that maximized reward and information, but could not be explained by simple stimulus–response rules. Contrasting rule-based choices with these residual choices revealed that following rules (1) decreased the energetic cost of decision-making; and (2) expanded rule-relevant coding dimensions and compressed rule-irrelevant ones. Together, these results suggest that we use rules, in part, because they reduce the costs of decision-making through a distributed representational warping in decision-making circuits.

scholarly journals Rule adherence warps decision-making

2019 ◽  
Author(s):  
R. Becket Ebitz ◽  
Jiaxin Cindy Tu ◽  
Benjamin Y. Hayden
Keyword(s):  
Decision Making ◽  
Dorsal Striatum ◽  
Neuronal Firing ◽  
Energetic Cost ◽  
Processing Efficiency ◽  
Rule Based ◽  
Stimulus Response ◽  
Neural Representations ◽  
Sensorimotor Information ◽  
The Brain

ABSTRACTWe have the capacity to follow arbitrary stimulus-response rules, meaning policies that determine how we will behave across circumstances. Yet, it is not clear how rules guide sensorimotor decision-making in the brain. Here, we recorded from neurons in three regions linked to decision-making, the orbitofrontal cortex, ventral striatum, and dorsal striatum, while macaques performed a rule-based decision-making task. We found that different rules warped the neural representations of chosen options by expanding rule-relevant coding dimensions relative to rule-irrelevant ones. Some cognitive theories suggest that warping could increase processing efficiency by facilitating rule-relevant computations at the expense of irrelevant ones. To test this idea, we modeled rules as the latent causes of decisions and identified a set of “rule-free” choices that could not be explained by simple rules. Contrasting these with rule-based choices revealed that following rules decreased the energetic cost of decision-making while warping the representational geometry of choice.SIGNIFICANCE STATEMENTOne important part of our ability to adapt flexibly to the world around us is our ability to implement arbitrary stimulus-response mappings, known as “rules”. Many studies have shown that when we follow a rule, its identity is encoded in neuronal firing rates. However, it remains unclear how rules regulate behavior. Here, we report that rules warp the way that sensorimotor information is represented in decision-making circuits: enhancing information that is relevant to the current rule at the expense of information that is irrelevant. These results imply that rules are implemented as a kind of attentional gate on what information is available for decision-making.

scholarly journals Roles for the Dorsal Striatum in Aversive Behavior

2021 ◽  
Vol 15 ◽  
Author(s):  
Adrien T. Stanley ◽  
Pellegrino Lippiello ◽  
David Sulzer ◽  
Maria Concetta Miniaci
Keyword(s):  
Ventral Striatum ◽  
Dorsal Striatum ◽  
Inhibitory Avoidance ◽  
Cell Types ◽  
Aversive Learning ◽  
Stimulus Response ◽  
Aversive Behavior ◽  
Established Role ◽  
The Brain

The ability to identify and avoid environmental stimuli that signal danger is essential to survival. Our understanding of how the brain encodes aversive behaviors has been primarily focused on roles for the amygdala, hippocampus (HIPP), prefrontal cortex, ventral midbrain, and ventral striatum. Relatively little attention has been paid to contributions from the dorsal striatum (DS) to aversive learning, despite its well-established role in stimulus-response learning. Here, we review studies exploring the role of DS in aversive learning, including different roles for the dorsomedial and dorsolateral striatum in Pavlovian fear conditioning as well as innate and inhibitory avoidance (IA) behaviors. We outline how future investigation might determine specific contributions from DS subregions, cell types, and connections that contribute to aversive behavior.

scholarly journals The contribution of striatal pseudo-reward prediction errors to value-based decision-making

2017 ◽  
Author(s):  
Ernest Mas-Herrero ◽  
Guillaume Sescousse ◽  
Roshan Cools ◽  
Josep Marco-Pallarés
Keyword(s):  
Decision Making ◽  
Choice Behavior ◽  
Prediction Errors ◽  
Life Outcomes ◽  
Stimulus Response ◽  
Hierarchical Reinforcement Learning ◽  
Reward Prediction ◽  
Fmri Study ◽  
Reward Contingencies ◽  
The Brain

AbstractMost studies that have investigated the brain mechanisms underlying learning have focused on the ability to learn simple stimulus-response associations. However, in everyday life, outcomes are often obtained through complex behavioral patterns involving a series of actions. In such scenarios, parallel learning systems are important to reduce the complexity of the learning problem, as proposed in the framework of hierarchical reinforcement learning (HRL). One of the key features of HRL is the computation of pseudo-reward prediction errors (PRPEs) which allow the reinforcement of actions that led to a sub-goal before the final goal itself is achieved. Here we wanted to test the hypothesis that, despite not carrying any rewarding value per se, pseudo-rewards might generate a bias in choice behavior when reward contingencies are not well-known or uncertain. Second, we also hypothesized that this bias might be related to the strength of PRPE striatal representations. In order to test these ideas, we developed a novel decision-making paradigm to assess reward prediction errors (RPEs) and PRPEs in two studies (fMRI study: n = 20; behavioural study: n = 19). Our results show that overall participants developed a preference for the most pseudo-rewarding option throughout the task, even though it did not lead to more monetary rewards. fMRI analyses revealed that this preference was predicted by individual differences in the relative striatal sensitivity to PRPEs vs RPEs. Together, our results indicate that pseudo-rewards generate learning signals in the striatum and subsequently bias choice behavior despite their lack of association with actual reward.

scholarly journals Dorsal striatum mediates deliberate decision making, not late-stage, stimulus-response learning

2017 ◽  
Vol 38 (12) ◽  
pp. 6133-6156 ◽  
Author(s):  
Nole M. Hiebert ◽  
Adrian M. Owen ◽  
Ken N. Seergobin ◽  
Penny A. MacDonald
Keyword(s):  
Decision Making ◽  
Late Stage ◽  
Dorsal Striatum ◽  
Response Learning ◽  
Stimulus Response

Schauer`s Playing By the Rules: A Philosophical Examination of Rule~Based Decision-Making in Law and in Life

1994 ◽  
Vol 16 (1) ◽  
Author(s):  
Judith Wagner Decew
Keyword(s):  
Decision Making ◽  
Rule Based

scholarly journals Veto and Vacillation: A Neural Precursor of the Decision to Withhold Action

2014 ◽  
Vol 26 (2) ◽  
pp. 296-304 ◽  
Author(s):  
Erman Misirlisoy ◽  
Patrick Haggard
Keyword(s):  
Human Behavior ◽  
Temporal Pattern ◽  
Neural Precursor ◽  
Rule Based ◽  
Action Sequences ◽  
Readiness Potentials ◽  
Preparatory Activity ◽  
The Brain ◽  
Motor Areas

The capacity to inhibit a planned action gives human behavior its characteristic flexibility. How this mechanism operates and what factors influence a decision to act or not act remain relatively unexplored. We used EEG readiness potentials (RPs) to examine preparatory activity before each action of an ongoing sequence, in which one action was occasionally omitted. We compared RPs between sequences in which omissions were instructed by a rule (e.g., “omit every fourth action”) and sequences in which the participant themselves freely decided which action to omit. RP amplitude was reduced for actions that immediately preceded a voluntary omission but not a rule-based omission. We also used the regular temporal pattern of the action sequences to explore brain processes linked to omitting an action by time-locking EEG averages to the inferred time when an action would have occurred had it not been omitted. When omissions were instructed by a rule, there was a negative-going trend in the EEG, recalling the rising ramp of an RP. No such component was found for voluntary omissions. The results are consistent with a model in which spontaneously fluctuating activity in motor areas of the brain could bias “free” decisions to act or not.

Neuroscience: The New English Major?

2021 ◽  
pp. 107385842110039
Author(s):  
Kristin F. Phillips ◽  
Harald Sontheimer
Keyword(s):  
Decision Making ◽  
Graduate Education ◽  
Human Behavior ◽  
Undergraduate Curriculum ◽  
Virginia Tech ◽  
Bachelor's Degree ◽  
State University ◽  
Human Interactions ◽  
English Major ◽  
The Brain

Once strictly the domain of medical and graduate education, neuroscience has made its way into the undergraduate curriculum with over 230 colleges and universities now offering a bachelor’s degree in neuroscience. The disciplinary focus on the brain teaches students to apply science to the understanding of human behavior, human interactions, sensation, emotions, and decision making. In this article, we encourage new and existing undergraduate neuroscience programs to envision neuroscience as a broad discipline with the potential to develop competencies suitable for a variety of careers that reach well beyond research and medicine. This article describes our philosophy and illustrates a broad-based undergraduate degree in neuroscience implemented at a major state university, Virginia Tech. We highlight the fact that the research-centered Experimental Neuroscience major is least popular of our four distinct majors, which underscores our philosophy that undergraduate neuroscience can cater to a different audience than traditionally thought.

Transfer Reinforcement Learning for Autonomous Driving

2021 ◽  
Vol 31 (3) ◽  
pp. 1-26
Author(s):  
Aravind Balakrishnan ◽  
Jaeyoung Lee ◽  
Ashish Gaurav ◽  
Krzysztof Czarnecki ◽  
Sean Sedwards
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Transfer Problem ◽  
Autonomous Driving ◽  
High Fidelity ◽  
Rule Based ◽  
High Level ◽  
Real Vehicle

Reinforcement learning (RL) is an attractive way to implement high-level decision-making policies for autonomous driving, but learning directly from a real vehicle or a high-fidelity simulator is variously infeasible. We therefore consider the problem of transfer reinforcement learning and study how a policy learned in a simple environment using WiseMove can be transferred to our high-fidelity simulator, W ise M ove . WiseMove is a framework to study safety and other aspects of RL for autonomous driving. W ise M ove accurately reproduces the dynamics and software stack of our real vehicle. We find that the accurately modelled perception errors in W ise M ove contribute the most to the transfer problem. These errors, when even naively modelled in WiseMove , provide an RL policy that performs better in W ise M ove than a hand-crafted rule-based policy. Applying domain randomization to the environment in WiseMove yields an even better policy. The final RL policy reduces the failures due to perception errors from 10% to 2.75%. We also observe that the RL policy has significantly less reliance on velocity compared to the rule-based policy, having learned that its measurement is unreliable.

scholarly journals Consciousness, decision making, and volition: freedom beyond chance and necessity

2021 ◽  
Author(s):  
Hans Liljenström
Keyword(s):  
Decision Making ◽  
Free Will ◽  
Brain Activity ◽  
Brain Structures ◽  
Complex Process ◽  
Neural Information ◽  
Neural Information Processing ◽  
Stochastic Population Model ◽  
The Brain

AbstractWhat is the role of consciousness in volition and decision-making? Are our actions fully determined by brain activity preceding our decisions to act, or can consciousness instead affect the brain activity leading to action? This has been much debated in philosophy, but also in science since the famous experiments by Libet in the 1980s, where the current most common interpretation is that conscious free will is an illusion. It seems that the brain knows, up to several seconds in advance what “you” decide to do. These studies have, however, been criticized, and alternative interpretations of the experiments can be given, some of which are discussed in this paper. In an attempt to elucidate the processes involved in decision-making (DM), as an essential part of volition, we have developed a computational model of relevant brain structures and their neurodynamics. While DM is a complex process, we have particularly focused on the amygdala and orbitofrontal cortex (OFC) for its emotional, and the lateral prefrontal cortex (LPFC) for its cognitive aspects. In this paper, we present a stochastic population model representing the neural information processing of DM. Simulation results seem to confirm the notion that if decisions have to be made fast, emotional processes and aspects dominate, while rational processes are more time consuming and may result in a delayed decision. Finally, some limitations of current science and computational modeling will be discussed, hinting at a future development of science, where consciousness and free will may add to chance and necessity as explanation for what happens in the world.

Neural activity in the dorsal striatum during cognitive decision making

2010 ◽  
Vol 68 ◽  
pp. e299
Author(s):  
Satoshi Nonomura ◽  
Kazuyuki Samejima ◽  
Kenji Doya ◽  
Jun Tanji
Keyword(s):  
Decision Making ◽  
Neural Activity ◽  
Dorsal Striatum ◽  
Cognitive Decision
Sign in / Sign up

Export Citation Format

Share Document