Neural mechanisms of credit assignment for inferred relationships in a structured world

Animals have been proposed to abstract compact representations of a task's structure that could, in principle, support accelerated learning and flexible behavior. Whether and how such abstracted representations may be used to assign credit for inferred, but unobserved, relationships in structured environments are unknown. Here, we develop a novel hierarchical reversal-learning task and Bayesian learning model to assess the computational and neural mechanisms underlying how humans infer specific choice-outcome associations via structured knowledge. We find that the medial prefrontal cortex (mPFC) efficiently represents hierarchically related choice-outcome associations governed by the same latent cause, using a generalized code to assign credit for both experienced and inferred outcomes. Furthermore, mPFC and lateral orbital frontal cortex track the inferred current "position" within a latent association space that generalizes over stimuli. Collectively, these findings demonstrate the importance both of tracking the current position in an abstracted task space and efficient, generalizable representations in prefrontal cortex for supporting flexible learning and inference in structured environments.

Download Full-text

Rule Activation and Ventromedial Prefrontal Engagement Support Accurate Stopping in Self-Paced Learning

10.1101/169110 ◽

2017 ◽

Author(s):

Sean R. O’Bryan ◽

Eric Walden ◽

Michael J. Serra ◽

Tyler Davis

Keyword(s):

Prefrontal Cortex ◽

Information Search ◽

Relevant Information ◽

Learning Task ◽

Neural Mechanisms ◽

Choice Accuracy ◽

Effective Learning ◽

Response Profile ◽

Dorsolateral Prefrontal ◽

Trial Participants

ABSTRACTWhen weighing evidence for a decision, individuals are continually faced with the choice of whether to gather more information or act on what has already been learned. The present experiment employed a self-paced category learning task and fMRI to examine the neural mechanisms underlying stopping of information search and how they contribute to choice accuracy. Participants learned to classify triads of face, object, and scene cues into one of two categories using a rule based on one of the stimulus dimensions. After each trial, participants were given the option to explicitly solve the rule or continue learning. Representational similarity analysis (RSA) was used to examine activation of rule-relevant information on trials leading up to a decision to solve the rule. We found that activation of rule-relevant information increased leading up to participants’ stopping decisions. Stopping was associated with widespread activation that included medial prefrontal cortex and visual association areas. Engagement of ventromedial prefrontal cortex (vmPFC) was associated with accurate stopping, and activation in this region was functionally coupled with signal in dorsolateral prefrontal cortex (dlPFC). Results suggest that activating rule information when deciding whether to stop an information search increases choice accuracy, and that the response profile of vmPFC during such decisions may provide an index of effective learning.

Download Full-text

Faculty Opinions recommendation of Prefrontal cortex mediates extinction of responding by two distinct neural mechanisms in accumbens shell.

Faculty Opinions – Post-Publication Peer Review of the Biomedical Literature ◽

10.3410/f.14253973.15768078 ◽

2012 ◽

Author(s):

Gary Aston-Jones ◽

David Moorman

Keyword(s):

Prefrontal Cortex ◽

Neural Mechanisms

Download Full-text

Retrieval practice facilitates memory updating by enhancing and differentiating medial prefrontal cortex representations

eLife ◽

10.7554/elife.57023 ◽

2020 ◽

Vol 9 ◽

Cited By ~ 1

Author(s):

Zhifang Ye ◽

Liang Shi ◽

Anqi Li ◽

Chuansheng Chen ◽

Gui Xue

Keyword(s):

Prefrontal Cortex ◽

Medial Prefrontal Cortex ◽

Pattern Analysis ◽

Retrieval Practice ◽

Neural Mechanisms ◽

Neural Activation ◽

Final Test ◽

Memory Updating ◽

Practice Condition ◽

Memory Integration

Updating old memories with new, more current information is critical for human survival, yet the neural mechanisms for memory updating in general and the effect of retrieval practice in particular are poorly understood. Using a three-day A-B/A-C memory updating paradigm, we found that compared to restudy, retrieval practice could strengthen new A-C memories and reduce old A-B memory intrusion, but did not suppress A-B memories. Neural activation pattern analysis revealed that compared to restudy, retrieval practice led to stronger target representation in the medial prefrontal cortex (MPFC) during the final test. Critically, it was only under the retrieval practice condition that the MPFC showed strong and comparable competitor evidence for both correct and incorrect trials during final test, and that the MPFC target representation during updating was predictive of subsequent memory. These results suggest that retrieval practice is able to facilitate memory updating by strongly engaging MPFC mechanisms in memory integration, differentiation and consolidation.

Download Full-text

Novel Inferences in a Multidimensional Social Network Use a Grid-like Code

10.1101/2020.05.29.124651 ◽

2020 ◽

Author(s):

Seongmin A. Park ◽

Douglas S. Miller ◽

Erie D. Boorman

Keyword(s):

Decision Making ◽

Prefrontal Cortex ◽

Medial Prefrontal Cortex ◽

Entorhinal Cortex ◽

Brain Regions ◽

Cognitive Map ◽

General Mechanism ◽

Temporoparietal Junction ◽

Flexible Behavior ◽

Discrete Problems

ABSTRACTGeneralizing experiences to guide decision making in novel situations is a hallmark of flexible behavior. It has been hypothesized such flexibility depends on a cognitive map of an environment or task, but directly linking the two has proven elusive. Here, we find that discretely sampled abstract relationships between entities in an unseen two-dimensional (2-D) social hierarchy are reconstructed into a unitary 2-D cognitive map in the hippocampus and entorhinal cortex. We further show that humans utilize a grid-like code in several brain regions, including entorhinal cortex and medial prefrontal cortex, for inferred direct trajectories between entities in the reconstructed abstract space during discrete decisions. Moreover, these neural grid-like codes in the entorhinal cortex predict neural decision value computations in the medial prefrontal cortex and temporoparietal junction area during choice. Collectively, these findings show that grid-like codes are used by the human brain to infer novel solutions, even in abstract and discrete problems, and suggest a general mechanism underpinning flexible decision making and generalization.

Download Full-text

Neural arbitration between social and individual learning systems

eLife ◽

10.7554/elife.54051 ◽

2020 ◽

Vol 9 ◽

Author(s):

Andreea Oliviana Diaconescu ◽

Madeline Stecy ◽

Lars Kasper ◽

Christopher J Burke ◽

Zoltan Nagy ◽

...

Keyword(s):

Prefrontal Cortex ◽

Functional Neuroimaging ◽

Learning Task ◽

Brain Regions ◽

Individual Learning ◽

Learning Systems ◽

Learning System ◽

Relative Precision ◽

Decision Confidence ◽

Dorsolateral Prefrontal

Decision making requires integrating knowledge gathered from personal experiences with advice from others. The neural underpinnings of the process of arbitrating between information sources has not been fully elucidated. In this study, we formalized arbitration as the relative precision of predictions, afforded by each learning system, using hierarchical Bayesian modeling. In a probabilistic learning task, participants predicted the outcome of a lottery using recommendations from a more informed advisor and/or self-sampled outcomes. Decision confidence, as measured by the number of points participants wagered on their predictions, varied with our definition of arbitration as a ratio of precisions. Functional neuroimaging demonstrated that arbitration signals were independent of decision confidence and involved modality-specific brain regions. Arbitrating in favor of self-gathered information activated the dorsolateral prefrontal cortex and the midbrain, whereas arbitrating in favor of social information engaged the ventromedial prefrontal cortex and the amygdala. These findings indicate that relative precision captures arbitration between social and individual learning systems at both behavioral and neural levels.

Download Full-text

Processing of Contextual Information during an Implicit Probabilistic Sequence Learning Task: Left Ventrolateral Prefrontal Cortex Involvement

NeuroImage ◽

10.1016/s1053-8119(18)31716-6 ◽

1998 ◽

Vol 7 (4) ◽

pp. S883

Author(s):

P. Peigneux ◽

P. Maquet ◽

M. Van der Linden ◽

T. Meulemans ◽

C. Degueldre ◽

...

Keyword(s):

Prefrontal Cortex ◽

Sequence Learning ◽

Contextual Information ◽

Learning Task ◽

Ventrolateral Prefrontal Cortex

Download Full-text

Integration of Teaching Processes and Learning Assessment in the Prefrontal Cortex during a Video Game Teaching–learning Task

Frontiers in Psychology ◽

10.3389/fpsyg.2016.02052 ◽

2017 ◽

Vol 7 ◽

Cited By ~ 15

Author(s):

Naoyuki Takeuchi ◽

Takayuki Mori ◽

Yoshimi Suzukamo ◽

Shin-Ichi Izumi

Keyword(s):

Prefrontal Cortex ◽

Video Game ◽

Learning Task ◽

Learning Assessment ◽

Teaching Learning

Download Full-text

Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia

Neural Computation ◽

10.1162/089976606775093909 ◽

2006 ◽

Vol 18 (2) ◽

pp. 283-328 ◽

Cited By ~ 510

Author(s):

Randall C. O'Reilly ◽

Michael J. Frank

Keyword(s):

Working Memory ◽

Prefrontal Cortex ◽

Basal Ganglia ◽

Computational Model ◽

Computational Models ◽

Memory Task ◽

Learning Mechanisms ◽

Subcortical Structures ◽

Credit Assignment ◽

Gating Mechanism

The prefrontal cortex has long been thought to subserve both working memory (the holding of information online for processing) and executive functions (deciding how to manipulate working memory and perform processing). Although many computational models of working memory have been developed, the mechanistic basis of executive function remains elusive, often amounting to a homunculus. This article presents an attempt to deconstruct this homunculus through powerful learning mechanisms that allow a computational model of the prefrontal cortex to control both itself and other brain areas in a strategic, task-appropriate manner. These learning mechanisms are based on subcortical structures in the midbrain, basal ganglia, and amygdala, which together form an actor-critic architecture. The critic system learns which prefrontal representations are task relevant and trains the actor, which in turn provides a dynamic gating mechanism for controlling working memory updating. Computationally, the learning mechanism is designed to simultaneously solve the temporal and structural credit assignment problems. The model's performance compares favorably with standard backpropagation-based temporal learning mechanisms on the challenging 1-2-AX working memory task and other benchmark working memory tasks.

Download Full-text