Role of Prefrontal Cortex in Reinforcement Learning and Decision Making

Author(s):  
Daeyeol Lee ◽  
Soyoun Kim ◽  
Hyojung Seo
2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Luca F. Kaiser ◽  
Theo O. J. Gruendler ◽  
Oliver Speck ◽  
Lennart Luettgau ◽  
Gerhard Jocham

AbstractIn a dynamic world, it is essential to decide when to leave an exploited resource. Such patch-leaving decisions involve balancing the cost of moving against the gain expected from the alternative patch. This contrasts with value-guided decisions that typically involve maximizing reward by selecting the current best option. Patterns of neuronal activity pertaining to patch-leaving decisions have been reported in dorsal anterior cingulate cortex (dACC), whereas competition via mutual inhibition in ventromedial prefrontal cortex (vmPFC) is thought to underlie value-guided choice. Here, we show that the balance between cortical excitation and inhibition (E/I balance), measured by the ratio of GABA and glutamate concentrations, plays a dissociable role for the two kinds of decisions. Patch-leaving decision behaviour relates to E/I balance in dACC. In contrast, value-guided decision-making relates to E/I balance in vmPFC. These results support mechanistic accounts of value-guided choice and provide evidence for a role of dACC E/I balance in patch-leaving decisions.


2019 ◽  
Author(s):  
Bhargav Teja Nallapu ◽  
Frédéric Alexandre

AbstractIn the context of flexible and adaptive animal behavior, the orbitofrontal cortex (OFC) is found to be one of the crucial regions in the prefrontal cortex (PFC) influencing the downstream processes of decision-making and learning in the sub-cortical regions. Although OFC has been implicated to be important in a variety of related behavioral processes, the exact mechanisms are unclear, through which the OFC encodes or processes information related to decision-making and learning. Here, we propose a systems-level view of the OFC, positioning it at the nexus of sub-cortical systems and other prefrontal regions. Particularly we focus on one of the most recent implications of neuroscientific evidences regarding the OFC - possible functional dissociation between two of its sub-regions : lateral and medial. We present a system-level computational model of decision-making and learning involving the two sub-regions taking into account their individual roles as commonly implicated in neuroscientific studies. We emphasize on the role of the interactions between the sub-regions within the OFC as well as the role of other sub-cortical structures which form a network with them. We leverage well-known computational architecture of thalamo-cortical basal ganglia loops, accounting for recent experimental findings on monkeys with lateral and medial OFC lesions, performing a 3-arm bandit task. First we replicate the seemingly dissociate effects of lesions to lateral and medial OFC during decision-making as a function of value-difference of the presented options. Further we demonstrate and argue that such an effect is not necessarily due to the dissociate roles of both the subregions, but rather a result of complex temporal dynamics between the interacting networks in which they are involved.Author summaryWe first highlight the role of the Orbitofrontal Cortex (OFC) in value-based decision making and goal-directed behavior in primates. We establish the position of OFC at the intersection of cortical mechanisms and thalamo-basal ganglial circuits. In order to understand possible mechanisms through which the OFC exerts emotional control over behavior, among several other possibilities, we consider the case of dissociate roles of two of its topographical subregions - lateral and medial parts of OFC. We gather predominant roles of each of these sub-regions as suggested by numerous experimental evidences in the form of a system-level computational model that is based on existing neuronal architectures. We argue that besides possible dissociation, there could be possible interaction of these sub-regions within themselves and through other sub-cortical structures, in distinct mechanisms of choice and learning. The computational framework described accounts for experimental data and can be extended to more comprehensive detail of representations required to understand the processes of decision-making, learning and the role of OFC and subsequently the regions of prefrontal cortex in general.


2021 ◽  
Vol 15 ◽  
Author(s):  
Neil M. Dundon ◽  
Allison D. Shapiro ◽  
Viktoriya Babenko ◽  
Gold N. Okafor ◽  
Scott T. Grafton

Anxiety is characterized by low confidence in daily decisions, coupled with high levels of phenomenological stress. Ventromedial prefrontal cortex (vmPFC) plays an integral role in maladaptive anxious behaviors via decreased sensitivity to threatening vs. non-threatening stimuli (fear generalization). vmPFC is also a key node in approach-avoidance decision making requiring two-dimensional integration of rewards and costs. More recently, vmPFC has been implicated as a key cortical input to the sympathetic branch of the autonomic nervous system. However, little is known about the role of this brain region in mediating rapid stress responses elicited by changes in confidence during decision making. We used an approach-avoidance task to examine the relationship between sympathetically mediated cardiac stress responses, vmPFC activity and choice behavior over long and short time-scales. To do this, we collected concurrent fMRI, EKG and impedance cardiography recordings of sympathetic drive while participants made approach-avoidance decisions about monetary rewards paired with painful electric shock stimuli. We observe first that increased sympathetic drive (shorter pre-ejection period) in states lasting minutes are associated with choices involving reduced decision ambivalence. Thus, on this slow time scale, sympathetic drive serves as a proxy for “mobilization” whereby participants are more likely to show consistent value-action mapping. In parallel, imaging analyses reveal that on shorter time scales (estimated with a trial-to-trial GLM), increased vmPFC activity, particularly during low-ambivalence decisions, is associated with decreased sympathetic state. Our findings support a role of sympathetic drive in resolving decision ambivalence across long time horizons and suggest a potential role of vmPFC in modulating this response on a moment-to-moment basis.


2021 ◽  
Vol 14 (1) ◽  
pp. 17
Author(s):  
Shuailong Li ◽  
Wei Zhang ◽  
Yuquan Leng ◽  
Xiaohui Wang

Environmental information plays an important role in deep reinforcement learning (DRL). However, many algorithms do not pay much attention to environmental information. In multi-agent reinforcement learning decision-making, because agents need to make decisions combined with the information of other agents in the environment, this makes the environmental information more important. To prove the importance of environmental information, we added environmental information to the algorithm. We evaluated many algorithms on a challenging set of StarCraft II micromanagement tasks. Compared with the original algorithm, the standard deviation (except for the VDN algorithm) was smaller than that of the original algorithm, which shows that our algorithm has better stability. The average score of our algorithm was higher than that of the original algorithm (except for VDN and COMA), which shows that our work significantly outperforms existing multi-agent RL methods.


2008 ◽  
Vol 66 (2b) ◽  
pp. 436-443 ◽  
Author(s):  
Henrique Cerqueira Guimarães ◽  
Richard Levy ◽  
Antônio Lúcio Teixeira ◽  
Rogério Gomes Beato ◽  
Paulo Caramelli

Apathy is considered the most frequent neuropsychiatric disturbance in dementia and its outcome is generally deleterious. Apathy can be related to a dysfunction of the anatomical-system that supports the generation of voluntary actions, namely the prefrontal cortex and/or the prefrontal-subcortical circuits. In Alzheimer's disease, pathological and neuroimaging data indicate that apathy is likely due to a dysfunction of the medial prefrontal cortex. Accordingly, in this review article, we propose a pathophysiological model to explain apathetic behavior in Alzheimer's disease, combining data from neuroimaging, neuropathology and experimental research on the role of orbito-frontal cortex, anterior cingulate cortex, basal ganglia and dopamine in decision-making neurobiology.


2020 ◽  
Vol 45 (13) ◽  
pp. 2278-2288 ◽  
Author(s):  
C. A. Hales ◽  
J. M. Bartlett ◽  
R. Arban ◽  
B. Hengerer ◽  
E. S. J. Robinson

2020 ◽  
Author(s):  
Milena Rmus ◽  
Samuel McDougle ◽  
Anne Collins

Reinforcement learning (RL) models have advanced our understanding of how animals learn and make decisions, and how the brain supports some aspects of learning. However, the neural computations that are explained by RL algorithms fall short of explaining many sophisticated aspects of human decision making, including the generalization of learned information, one-shot learning, and the synthesis of task information in complex environments. Instead, these aspects of instrumental behavior are assumed to be supported by the brain’s executive functions (EF). We review recent findings that highlight the importance of EF in learning. Specifically, we advance the theory that EF sets the stage for canonical RL computations in the brain, providing inputs that broaden their flexibility and applicability. Our theory has important implications for how to interpret RL computations in the brain and behavior.


Sign in / Sign up

Export Citation Format

Share Document