scholarly journals Model-based prioritization for acquiring protection

2021 ◽  
Author(s):  
Sarah M. Tashjian ◽  
Toby Wise ◽  
dean mobbs
Keyword(s):  
Reinforcement Learning ◽  
Computational Modeling ◽  
Learning Rate ◽  
Behavioral Task ◽  
Decision Strategies ◽  
Total N ◽  
Model Based Control ◽  
Model Based ◽  
Computational Architecture ◽  
Reward And Punishment

Protection, or the mitigation of harm, often involves the capacity to prospectively plan the actions needed to combat a threat. The computational architecture of decisions involving protection remains unclear, as well as whether these decisions differ from other positive prospective actions. Here we examine effects of valence and context by comparing protection to reward, which occurs in a different context but is also positively valenced, and punishment, which also occurs in an aversive context but differs in valence. We applied computational modeling across three independent studies (Total N=600) using five iterations of a ‘two-step’ behavioral task to examine model-based reinforcement learning for protection, reward, and punishment in humans. Decisions motivated by acquiring safety via protection evoked a higher degree of model-based control than acquiring reward and avoiding punishment, with no significant differences in learning rate. The context-valence asymmetry characteristic of protection increased deployment of flexible decision strategies, suggesting model-based control depends on the context in which outcomes are encountered as well as the valence of the outcome.

scholarly journals Revealing Neurocomputational Mechanisms of Reinforcement Learning and Decision-Making With the hBayesDM Package

2017 ◽  
Vol 1 ◽  
pp. 24-57 ◽  
Author(s):  
Woo-Young Ahn ◽  
Nathaniel Haines ◽  
Lei Zhang
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Computational Modeling ◽  
Bayesian Modeling ◽  
Hierarchical Bayesian ◽  
Hierarchical Bayesian Modeling ◽  
Model Based ◽  
Modeling Approaches ◽  
Wide Range ◽  
User Friendly

Reinforcement learning and decision-making (RLDM) provide a quantitative framework and computational theories with which we can disentangle psychiatric conditions into the basic dimensions of neurocognitive functioning. RLDM offer a novel approach to assessing and potentially diagnosing psychiatric patients, and there is growing enthusiasm for both RLDM and computational psychiatry among clinical researchers. Such a framework can also provide insights into the brain substrates of particular RLDM processes, as exemplified by model-based analysis of data from functional magnetic resonance imaging (fMRI) or electroencephalography (EEG). However, researchers often find the approach too technical and have difficulty adopting it for their research. Thus, a critical need remains to develop a user-friendly tool for the wide dissemination of computational psychiatric methods. We introduce an R package called hBayesDM (hierarchical Bayesian modeling of Decision-Making tasks), which offers computational modeling of an array of RLDM tasks and social exchange games. The hBayesDM package offers state-of-the-art hierarchical Bayesian modeling, in which both individual and group parameters (i.e., posterior distributions) are estimated simultaneously in a mutually constraining fashion. At the same time, the package is extremely user-friendly: users can perform computational modeling, output visualization, and Bayesian model comparisons, each with a single line of coding. Users can also extract the trial-by-trial latent variables (e.g., prediction errors) required for model-based fMRI/EEG. With the hBayesDM package, we anticipate that anyone with minimal knowledge of programming can take advantage of cutting-edge computational-modeling approaches to investigate the underlying processes of and interactions between multiple decision-making (e.g., goal-directed, habitual, and Pavlovian) systems. In this way, we expect that the hBayesDM package will contribute to the dissemination of advanced modeling approaches and enable a wide range of researchers to easily perform computational psychiatric research within different populations.

scholarly journals A Day-to-Day Route Choice Model Based on Reinforcement Learning

2014 ◽  
Vol 2014 ◽  
pp. 1-19 ◽  
Author(s):  
Fangfang Wei ◽  
Shoufeng Ma ◽  
Ning Jia
Keyword(s):  
Reinforcement Learning ◽  
Network Flow ◽  
Choice Model ◽  
Route Choice ◽  
User Equilibrium ◽  
Learning Rate ◽  
Traffic Dynamics ◽  
Link Capacity ◽  
Model Based ◽  
Capacity Degradation

Day-to-day traffic dynamics are generated by individual traveler’s route choice and route adjustment behaviors, which are appropriate to be researched by using agent-based model and learning theory. In this paper, we propose a day-to-day route choice model based on reinforcement learning and multiagent simulation. Travelers’ memory, learning rate, and experience cognition are taken into account. Then the model is verified and analyzed. Results show that the network flow can converge to user equilibrium (UE) if travelers can remember all the travel time they have experienced, but which is not necessarily the case under limited memory; learning rate can strengthen the flow fluctuation, but memory leads to the contrary side; moreover, high learning rate results in the cyclical oscillation during the process of flow evolution. Finally, both the scenarios of link capacity degradation and random link capacity are used to illustrate the model’s applications. Analyses and applications of our model demonstrate the model is reasonable and useful for studying the day-to-day traffic dynamics.

Altered Reinforcement Learning From Reward and Punishment in Anorexia Nervosa: Evidence From Computational Modeling

2020 ◽  
Vol 87 (9) ◽  
pp. S141
Author(s):  
Christina Wierenga ◽  
Erin Reilly ◽  
Amanda Bischoff-Grethe ◽  
Walter Kaye ◽  
Gregory Brown
Keyword(s):  
Anorexia Nervosa ◽  
Reinforcement Learning ◽  
Computational Modeling ◽  
Reward And Punishment

scholarly journals Revealing neuro-computational mechanisms of reinforcement learning and decision-making with the hBayesDM package

2016 ◽  
Author(s):  
Woo-Young Ahn ◽  
Nathaniel Haines ◽  
Lei Zhang
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Computational Modeling ◽  
Social Exchange ◽  
Latent Variables ◽  
Psychiatric Research ◽  
Model Based ◽  
Modeling Approaches ◽  
Wide Range ◽  
User Friendly

AbstractReinforcement learning and decision-making (RLDM) provide a quantitative framework and computational theories, with which we can disentangle psychiatric conditions into basic dimensions of neurocognitive functioning. RLDM offer a novel approach to assess and potentially diagnose psychiatric patients, and there is growing enthusiasm on RLDM and Computational Psychiatry among clinical researchers. Such a framework can also provide insights into the brain substrates of particular RLDM processes as exemplified by model-based functional magnetic resonance imaging (fMRI) or electroencephalogram (EEG). However, many researchers often find the approach too technical and have difficulty adopting it for their research. Thus, there remains a critical need to develop a user-friendly tool for the wide dissemination of computational psychiatric methods. We introduce an R package called hBayesDM (hierarchical Bayesian modeling of Decision-Making tasks), which offers computational modeling on an array of RLDM tasks and social exchange games. The hBayesDM package offers state-of-the-art hierarchical Bayesian modeling, where both individual and group parameters (i.e., posterior distributions) are estimated simultaneously in a mutually constraining fashion. At the same time, it is extremely user-friendly: users can perform computational modeling, output visualization, and Bayesian model comparisons–each with a single line of coding. Users can also extract trial-by-trial latent variables (e.g., prediction errors) required for model-based fMRI/EEG. With the hBayesDM package, we anticipate that anyone with minimal knowledge of programming can take advantage of cutting-edge computational modeling approaches and investigate the underlying processes of and interactions between multiple decision-making (e.g., goal-directed, habitual, and Pavlovian) systems. In this way, it is our expectation that the hBayesDM package will contribute to the dissemination of advanced modeling approaches and enable a wide range of researchers to easily perform computational psychiatric research within their populations.

scholarly journals Risk-Aware Model-Based Control

2021 ◽  
Vol 8 ◽  
Author(s):  
Chen Yu ◽  
Andre Rosendo
Keyword(s):  
Reinforcement Learning ◽  
Value At Risk ◽  
Learning Algorithm ◽  
State Of The Art ◽  
Training Data ◽  
Conditional Value At Risk ◽  
High Dimensional ◽  
Model Based Control ◽  
Model Based ◽  
Model Free

Model-Based Reinforcement Learning (MBRL) algorithms have been shown to have an advantage on data-efficiency, but often overshadowed by state-of-the-art model-free methods in performance, especially when facing high-dimensional and complex problems. In this work, a novel MBRL method is proposed, called Risk-Aware Model-Based Control (RAMCO). It combines uncertainty-aware deep dynamics models and the risk assessment technique Conditional Value at Risk (CVaR). This mechanism is appropriate for real-world application since it takes epistemic risk into consideration. In addition, we use a model-free solver to produce warm-up training data, and this setting improves the performance in low-dimensional environments and covers the shortage of MBRL’s nature in the high-dimensional scenarios. In comparison with other state-of-the-art reinforcement learning algorithms, we show that it produces superior results on a walking robot model. We also evaluate the method with an Eidos environment, which is a novel experimental method with multi-dimensional randomly initialized deep neural networks to measure the performance of any reinforcement learning algorithm, and the advantages of RAMCO are highlighted.

scholarly journals Stronger Prejudices Are Associated With Decreased Model-Based Control

2022 ◽  
Vol 12 ◽  
Author(s):  
Miriam Sebold ◽  
Hao Chen ◽  
Aleyna Önal ◽  
Sören Kuitunen-Paul ◽  
Negin Mojtahedzadeh ◽  
...  
Keyword(s):  
Decision Making ◽  
Individual Differences ◽  
Computational Modeling ◽  
Reaction Times ◽  
Model Based Control ◽  
Model Based ◽  
Relative Dominance ◽  
Model Free ◽  
Decision Making Processes

Background: Prejudices against minorities can be understood as habitually negative evaluations that are kept in spite of evidence to the contrary. Therefore, individuals with strong prejudices might be dominated by habitual or “automatic” reactions at the expense of more controlled reactions. Computational theories suggest individual differences in the balance between habitual/model-free and deliberative/model-based decision-making.Methods: 127 subjects performed the two Step task and completed the blatant and subtle prejudice scale.Results: By using analyses of choices and reaction times in combination with computational modeling, subjects with stronger blatant prejudices showed a shift away from model-based control. There was no association between these decision-making processes and subtle prejudices.Conclusion: These results support the idea that blatant prejudices toward minorities are related to a relative dominance of habitual decision-making. This finding has important implications for developing interventions that target to change prejudices across societies.

scholarly journals Reward and punishment reversal learning in major depressive disorder

2020 ◽  
Author(s):  
Dahlia Mukherjee ◽  
Alexandre Leo Stephen Filipowicz ◽  
Khoi D. Vo ◽  
Theodore Sattherwaite ◽  
Joe Kable
Keyword(s):  
Major Depressive Disorder ◽  
Reinforcement Learning ◽  
Depressive Disorder ◽  
Computational Modeling ◽  
Reversal Learning ◽  
Performance Metrics ◽  
Learning Task ◽  
Major Depressive ◽  
Learning Rates ◽  
Reward And Punishment

Depression has been associated with impaired reward and punishment processing, but the specific nature of these deficits is less understood and still widely debated. We analyzed reinforcement-based decision-making in individuals diagnosed with major depressive disorder (MDD) to identify the specific decision mechanisms contributing to poorer performance. Individuals with MDD (n = 64) and matched healthy controls (n = 64) performed a probabilistic reversal learning task in which they used feedback to identify which of two stimuli had the highest probability of reward (reward condition) or lowest probability of punishment (punishment condition). Learning differences were characterized using a hierarchical Bayesian reinforcement learning model. While both groups showed reinforcement learning-like behavior, depressed individuals made fewer optimal choices and adjusted more slowly to reversals in both the reward and punishment conditions. Our computational modeling analysis found that depressed individuals showed lower learning rates and, to a lesser extent, lower value sensitivity in both the reward and punishment conditions. Learning rates also predicted depression more accurately than simple performance metrics. These results demonstrate that depression is characterized by a hyposensitivity to positive outcomes, which influences the rate at which depressed individuals learn from feedback, but not a hypersensitivity to negative outcomes as has previously been suggested. Additionally, we demonstrate that computational modeling provides a more precise characterization of the dynamics contributing to these learning deficits, and offers stronger insights into the mechanistic processes affected by depression.

Sign in / Sign up

Export Citation Format

Share Document