Metamers of Bayesian computation

Mapping Intimacies ◽

10.1101/2020.08.11.246355 ◽

2020 ◽

Author(s):

Hansem Sohn ◽

Mehrdad Jazayeri

Keyword(s):

Internal Models ◽

Cost Functions ◽

Trial And Error ◽

Bayesian Decision ◽

Visuomotor Rotation ◽

Model Based ◽

Model Free ◽

Sensory Experiences ◽

Timing Task ◽

Decision Outcomes

AbstractThere are two sharply debated views on how humans make decisions under uncertainty. Bayesian decision theory posits that humans optimize their behavior by establishing and integrating internal models of past sensory experiences (priors) and decision outcomes (cost functions). An alternative model-free hypothesis posits that decisions are optimized through trial and error without explicit internal models for priors and cost functions. To distinguish between these possibilities, we introduce a novel paradigm that probes sensitivity of humans to transitions between prior-cost pairs that demand the same optimal policy (metamers) but distinct internal models. We demonstrate the utility of our approach in two experiments that were classically explained by model-based Bayesian theory. Our approach validates the model-based strategy in an interval timing task but not in a visuomotor rotation task. More generally, our work provides a domain-general approach for testing the circumstances under which humans implement model-based Bayesian computations.

Download Full-text

Validating model-based Bayesian integration using prior–cost metamers

Proceedings of the National Academy of Sciences ◽

10.1073/pnas.2021531118 ◽

2021 ◽

Vol 118 (25) ◽

pp. e2021531118

Author(s):

Hansem Sohn ◽

Mehrdad Jazayeri

Keyword(s):

Bayesian Learning ◽

Learning Strategy ◽

Alternative Hypothesis ◽

Internal Models ◽

Cost Functions ◽

Bayesian Decision ◽

Visuomotor Rotation ◽

Model Based ◽

Sensory Experiences ◽

Timing Task

There are two competing views on how humans make decisions under uncertainty. Bayesian decision theory posits that humans optimize their behavior by establishing and integrating internal models of past sensory experiences (priors) and decision outcomes (cost functions). An alternative hypothesis posits that decisions are optimized through trial and error without explicit internal models for priors and cost functions. To distinguish between these possibilities, we introduce a paradigm that probes the sensitivity of humans to transitions between prior–cost pairs that demand the same optimal policy (metamers) but distinct internal models. We demonstrate the utility of our approach in two experiments that were classically explained by Bayesian theory. Our approach validates the Bayesian learning strategy in an interval timing task but not in a visuomotor rotation task. More generally, our work provides a domain-general approach for testing the circumstances under which humans explicitly implement model-based Bayesian computations.

Download Full-text

Representation, abstraction, and simple-minded sophisticates

Behavioral and Brain Sciences ◽

10.1017/s0140525x19002942 ◽

2020 ◽

Vol 43 ◽

Author(s):

Peter Dayan

Keyword(s):

Decision Theory ◽

Predictive Coding ◽

Bayesian Decision Theory ◽

Bayesian Decision ◽

Model Based ◽

Model Free

Abstract Bayesian decision theory provides a simple formal elucidation of some of the ways that representation and representational abstraction are involved with, and exploit, both prediction and its rather distant cousin, predictive coding. Both model-free and model-based methods are involved.

Download Full-text

Vicarious trial-and-error is associated with the use of place-strategies in human virtual navigation

10.31234/osf.io/f8rhn ◽

2020 ◽

Author(s):

Neil Schmitzer-Torbert

Keyword(s):

Trial And Error ◽

Human Navigation ◽

Response Strategies ◽

Model Based ◽

Model Free ◽

Decision Points ◽

Behavioral Index ◽

Vicarious Trial And Error ◽

Contrast Response ◽

Navigation Strategies

Studies of decision-making in rodents have demonstrated that vicarious trial-and-error (VTE) is an important behavioral index of deliberation, when animals search through and evaluate the available options before making a decision. In rodents, VTE is enhanced during the use of hippocampally-dependent place strategies, which may represent a type of model-based behavior. While some evidence exists for VTE-like behaviors in humans during navigation, it is unknown if VTE in humans is specifically associated place-strategies, as would be predicted for model-based behaviors. To address this gap, humans were tested in navigation tasks in symmetrical environments, which allowed for the use of probe trials to assess navigation strategies (place or response) or impose them directly. The use of place strategies (on probe trials and place-training) was associated with increases in measures of VTE (reorientations and pausing) especially at high-cost decision points, similar to results from rodent studies. In contrast, response-strategies were associated with the development of efficient, stereotyped trajectories (consistent with model-free learning). These results support the identification of place- and response-strategies in human navigation with model-based and model-free learning, respectively, and demonstrate that VTE is specifically related to the use of place-strategies.

Download Full-text

The influence of internal models on feedback-related brain activity

Cognitive Affective & Behavioral Neuroscience ◽

10.3758/s13415-020-00820-6 ◽

2020 ◽

Vol 20 (5) ◽

pp. 1070-1089

Author(s):

Franz Wurm ◽

Benjamin Ernst ◽

Marco Steinhauser

Keyword(s):

Internal Model ◽

Internal Representation ◽

Internal Models ◽

Decision Task ◽

Prediction Errors ◽

Feedback Processing ◽

Model Based ◽

Model Free ◽

Reward Prediction ◽

Environmental Structure

Abstract Decision making relies on the interplay between two distinct learning mechanisms, namely habitual model-free learning and goal-directed model-based learning. Recent literature suggests that this interplay is significantly shaped by the environmental structure as represented by an internal model. We employed a modified two-stage but one-decision Markov decision task to investigate how two internal models differing in the predictability of stage transitions influence the neural correlates of feedback processing. Our results demonstrate that fronto-central theta and the feedback-related negativity (FRN), two correlates of reward prediction errors in the medial frontal cortex, are independent of the internal representations of the environmental structure. In contrast, centro-parietal delta and the P3, two correlates possibly reflecting feedback evaluation in working memory, were highly susceptible to the underlying internal model. Model-based analyses of single-trial activity showed a comparable pattern, indicating that while the computation of unsigned reward prediction errors is represented by theta and the FRN irrespective of the internal models, the P3 adapts to the internal representation of an environment. Our findings further substantiate the assumption that the feedback-locked components under investigation reflect distinct mechanisms of feedback processing and that different internal models selectively influence these mechanisms.

Download Full-text

Shaping Model-Free Reinforcement-Learning with Model-Based Pseudorewards

10.32470/ccn.2018.1191-0 ◽

2018 ◽

Author(s):

Paul Krueger ◽

Thomas Griffiths

Keyword(s):

Reinforcement Learning ◽

Model Based ◽

Model Free

Download Full-text

Model-Based and Model-Free Social Cognition

10.31234/osf.io/ue6j2 ◽

2019 ◽

Author(s):

Leor M Hackel ◽

Jeffrey Jordan Berg ◽

Björn Lindström ◽

David Amodio

Keyword(s):

Reinforcement Learning ◽

Social Cognition ◽

Learning Strategies ◽

Memory Systems ◽

Learning Task ◽

Financial Advisors ◽

Model Based ◽

Model Free ◽

Systems Model ◽

Task Assessment

Do habits play a role in our social impressions? To investigate the contribution of habits to the formation of social attitudes, we examined the roles of model-free and model-based reinforcement learning in social interactions—computations linked in past work to habit and planning, respectively. Participants in this study learned about novel individuals in a sequential reinforcement learning paradigm, choosing financial advisors who led them to high- or low-paying stocks. Results indicated that participants relied on both model-based and model-free learning, such that each independently predicted choice during the learning task and self-reported liking in a post-task assessment. Specifically, participants liked advisors who could provide large future rewards as well as advisors who had provided them with large rewards in the past. Moreover, participants varied in their use of model-based and model-free learning strategies, and this individual difference influenced the way in which learning related to self-reported attitudes: among participants who relied more on model-free learning, model-free social learning related more to post-task attitudes. We discuss implications for attitudes, trait impressions, and social behavior, as well as the role of habits in a memory systems model of social cognition.

Download Full-text