Asymmetric learning facilitates human inference of transitive relations

Humans and other animals are capable of inferring never-experienced relations (e.g., A>C) from other relational observations (e.g., A>B and B>C). The processes behind such transitive inference are subject to intense research. Here, we demonstrate a new aspect of relational learning, building on previous evidence that transitive inference can be accomplished through simple reinforcement learning mechanisms. We show in simulations that inference of novel relations benefits from an asymmetric learning policy, where observers update only their belief about the winner (or loser) in a pair. Across 4 experiments (n=145), we find substantial empirical support for such asymmetries in inferential learning. The learning policy favoured by our simulations and experiments gives rise to a compression of values which is routinely observed in psychophysics and behavioural economics. In other words, a seemingly biased learning strategy that yields well-known cognitive distortions can be beneficial for transitive inferential judgments.

Download Full-text

Learning at Variable Attentional Load Requires Cooperation of Working Memory, Meta-learning and Attention-augmented Reinforcement Learning

Journal of Cognitive Neuroscience ◽

10.1162/jocn_a_01780 ◽

2021 ◽

pp. 1-29

Author(s):

Thilo Womelsdorf ◽

Marcus R. Watson ◽

Paul Tiesinga

Keyword(s):

Working Memory ◽

Reinforcement Learning ◽

Real World ◽

Learning Strategy ◽

Prediction Errors ◽

Learning Mechanisms ◽

Negative Prediction ◽

Attentional Load ◽

Meta Learning ◽

Efficient Learning

Abstract Flexible learning of changing reward contingencies can be realized with different strategies. A fast learning strategy involves using working memory of recently rewarded objects to guide choices. A slower learning strategy uses prediction errors to gradually update value expectations to improve choices. How the fast and slow strategies work together in scenarios with real-world stimulus complexity is not well known. Here, we aim to disentangle their relative contributions in rhesus monkeys while they learned the relevance of object features at variable attentional load. We found that learning behavior across six monkeys is consistently best predicted with a model combining (i) fast working memory and (ii) slower reinforcement learning from differently weighted positive and negative prediction errors as well as (iii) selective suppression of nonchosen feature values and (iv) a meta-learning mechanism that enhances exploration rates based on a memory trace of recent errors. The optimal model parameter settings suggest that these mechanisms cooperate differently at low and high attentional loads. Whereas working memory was essential for efficient learning at lower attentional loads, enhanced weighting of negative prediction errors and meta-learning were essential for efficient learning at higher attentional loads. Together, these findings pinpoint a canonical set of learning mechanisms and suggest how they may cooperate when subjects flexibly adjust to environments with variable real-world attentional demands.

Download Full-text

MEG signatures of long-term effects of agreement and disagreement with the majority

Scientific Reports ◽

10.1038/s41598-021-82670-x ◽

2021 ◽

Vol 11 (1) ◽

Author(s):

A. Gorin ◽

V. Klucharev ◽

A. Ossadtchi ◽

I. Zubarev ◽

V. Moiseeva ◽

...

Keyword(s):

Reinforcement Learning ◽

Social Influence ◽

Temporal Dynamics ◽

Peer Group ◽

Long Term Effects ◽

Learning Mechanisms ◽

Source Imaging ◽

The Face ◽

First Session

AbstractPeople often change their beliefs by succumbing to an opinion of others. Such changes are often referred to as effects of social influence. While some previous studies have focused on the reinforcement learning mechanisms of social influence or on its internalization, others have reported evidence of changes in sensory processing evoked by social influence of peer groups. In this study, we used magnetoencephalographic (MEG) source imaging to further investigate the long-term effects of agreement and disagreement with the peer group. The study was composed of two sessions. During the first session, participants rated the trustworthiness of faces and subsequently learned group rating of each face. In the first session, a neural marker of an immediate mismatch between individual and group opinions was found in the posterior cingulate cortex, an area involved in conflict-monitoring and reinforcement learning. To identify the neural correlates of the long-lasting effect of the group opinion, we analysed MEG activity while participants rated faces during the second session. We found MEG traces of past disagreement or agreement with the peers at the parietal cortices 230 ms after the face onset. The neural activity of the superior parietal lobule, intraparietal sulcus, and precuneus was significantly stronger when the participant’s rating had previously differed from the ratings of the peers. The early MEG correlates of disagreement with the majority were followed by activity in the orbitofrontal cortex 320 ms after the face onset. Altogether, the results reveal the temporal dynamics of the neural mechanism of long-term effects of disagreement with the peer group: early signatures of modified face processing were followed by later markers of long-term social influence on the valuation process at the ventromedial prefrontal cortex.

Download Full-text

A New Hybrid Deep Neural Architectural Search based Ensemble Reinforcement Learning Strategy for Wind Power Forecasting

IEEE Transactions on Industry Applications ◽

10.1109/tia.2021.3126272 ◽

2021 ◽

pp. 1-1

Author(s):

Seyed Mohammad Jafar Jalali ◽

Gerardo J. Osorio ◽

Sajad Ahmadian ◽

Mohamed Lotfi ◽

Vasco Campos ◽

...

Keyword(s):

Reinforcement Learning ◽

Wind Power ◽

Learning Strategy ◽

Wind Power Forecasting ◽

Power Forecasting

Download Full-text

Reinforcement Learning Strategy for Solving the Resource-Constrained Project Scheduling Problem by a Team of A-Teams

Intelligent Information and Database Systems - Lecture Notes in Computer Science ◽

10.1007/978-3-319-05458-2_21 ◽

2014 ◽

pp. 197-206 ◽

Cited By ~ 2

Author(s):

Piotr Jędrzejowicz ◽

Ewa Ratajczak-Ropel

Keyword(s):

Reinforcement Learning ◽

Project Scheduling ◽

Learning Strategy ◽

Scheduling Problem ◽

Resource Constrained ◽

Resource Constrained Project Scheduling ◽

Project Scheduling Problem

Download Full-text

Severe Sexual Abuse Reduces Frontoparietal Network Activity during Model-Based Reinforcement Learning Updates

10.31234/osf.io/fqv2s ◽

2019 ◽

Author(s):

Allison Letkiewicz ◽

Amy L. Cochran ◽

Josh M. Cisler

Keyword(s):

Sexual Abuse ◽

Reinforcement Learning ◽

Learning Styles ◽

Computational Models ◽

Learning Strategy ◽

Network Activity ◽

Model Based ◽

Model Free ◽

Frontoparietal Network ◽

Assaultive Trauma

Trauma and trauma-related disorders are characterized by altered learning styles. Two learning processes that have been delineated using computational modeling are model-free and model-based reinforcement learning (RL), characterized by trial and error and goal-driven, rule-based learning, respectively. Prior research suggests that model-free RL is disrupted among individuals with a history of assaultive trauma and may contribute to altered fear responding. Currently, it is unclear whether model-based RL, which involves building abstract and nuanced representations of stimulus-outcome relationships to prospectively predict action-related outcomes, is also impaired among individuals who have experienced trauma. The present study sought to test the hypothesis of impaired model-based RL among adolescent females exposed to assaultive trauma. Participants (n=60) completed a three-arm bandit RL task during fMRI acquisition. Two computational models compared the degree to which each participant’s task behavior fit the use of a model-free versus model-based RL strategy. Overall, a greater portion of participants’ behavior was better captured by the model-based than model-free RL model. Although assaultive trauma did not predict learning strategy use, greater sexual abuse severity predicted less use of model-based compared to model-free RL. Additionally, severe sexual abuse predicted less left frontoparietal network encoding of model-based RL updates, which was not accounted for by PTSD. Given the significant impact that sexual trauma has on mental health and other aspects of functioning, it is plausible that altered model-based RL is an important route through which clinical impairment emerges.

Download Full-text