scholarly journals Gambling environment exposure increases temporal discounting but improves model-based control in regular slot-machine gamblers.

2021 ◽  
Author(s):  
Ben Jonathan Wagner ◽  
David Mathar ◽  
Jan Peters
Keyword(s):  
Reinforcement Learning ◽  
Gambling Disorder ◽  
Cognitive Distortions ◽  
Real Life ◽  
Temporal Discounting ◽  
Slot Machine ◽  
Model Based ◽  
Environment Exposure ◽  
Hypothesis Model ◽  
The Impact

Gambling disorder is a behavioral addiction that negatively impacts personal finances, work, relationships and mental health. In this pre-registered study (https://osf.io/5ptz9/) we investigated the impact of real-life gambling environments on two computational markers of addiction, temporal discounting and model-based reinforcement learning. Regular gamblers (n = 30, DSM-5 score range 3-9) performed both tasks in a neutral (cafe) and a gambling-related environment (slot-machine venue) in counterbalanced order. Data were modeled using drift diffusion models for temporal discounting and reinforcement learning via hierarchical Bayesian estimation. Replicating previous findings, gamblers discounted rewards more steeply in the gambling-related context. This effect was positively correlated with gambling related cognitive distortions (pre-registered analysis). In contrast to our pre-registered hypothesis, model-based reinforcement learning was improved in the gambling context. Gambling disorder is associated with increased temporal discounting and reduced model-based learning. Here we show that these effects are modulated in opposite ways by real-life gambling cue exposure. Results challenge aspects of habit theories of addiction, and reveal that laboratory-based computational markers of psychopathology are under substantial contextual control.

scholarly journals Increased temporal discounting and reduced model-based control in problem gambling are not substantially modulated by exposure to virtual gambling environments.

2021 ◽  
Author(s):  
Luca Rene Bruder ◽  
Ben Wagner ◽  
David Mathar ◽  
Jan Peters
Keyword(s):  
Pathological Gambling ◽  
Electrodermal Activity ◽  
Temporal Discounting ◽  
Sports Betting ◽  
Reduced Model ◽  
Sequential Decision ◽  
Model Based Control ◽  
Model Based ◽  
Model Free ◽  
Environment Exposure

High-performance virtual reality (VR) technology has opened new possibilities for the examination of the reactivity towards addiction-related cues (cue-reactivity) in addiction. In this preregistered study (https://osf.io/4mrta), we investigated the subjective, physiological, and behavioral effects of gambling-related VR environment exposure in participants reporting frequent or pathological gambling (n=31) as well as non-gambling controls (n=29). On two separate days, participants explored two rich and navigable VR-environments (neutral: cafe vs. gambling-related: casino/sports-betting facility), while electrodermal activity and heart rate were continuously measured using remote sensors. Within VR, participants performed a temporal discounting task and a sequential decision-making task designed to assess model-based and model-free contributions to behavior. Replicating previous findings, we found strong evidence for increased temporal discounting and reduced model-based control in participants reporting frequent or pathological gambling. Although VR gambling environment exposure increased subjective craving, there was if anything inconclusive evidence for further behavioral or physiological effects. Instead, VR exposure substantially increased physiological arousal (electrodermal activity), across groups and conditions. VR is a promising tool for the investigation of context effects in addiction, but some caution is warranted since effects of real gambling environments might not generally replicate in VR. Future studies should delineate how factors such as cognitive load and ecological validity could be balanced to create a more naturalistic VR experience.

scholarly journals Diffusion modeling reveals reinforcement learning impairments in gambling disorder that are linked to attenuated ventromedial prefrontal cortex value representations

2020 ◽  
Author(s):  
Antonius Wiehler ◽  
Jan Peters
Keyword(s):  
Prefrontal Cortex ◽  
Reinforcement Learning ◽  
Gambling Disorder ◽  
Ventromedial Prefrontal Cortex ◽  
Response Threshold ◽  
Boundary Separation ◽  
Learning Impairments ◽  
Model Based ◽  
Learning Rates ◽  
Over Time

AbstractGambling disorder is associated with deficits in classical feedback-based learning tasks, but the computational mechanisms underlying such learning impairments are still poorly understood. Here, we examined this question using a combination of computational modeling and functional resonance imaging (fMRI) in gambling disorder participants (n=23) and matched controls (n=19). Participants performed a simple reinforcement learning task with two pairs of stimuli (80% vs. 20% reinforcement rates per pair). As predicted, gamblers made significantly fewer selections of the optimal stimulus, while overall response times (RTs) were not significantly different between groups. We then used comprehensive modeling using reinforcement learning drift diffusion models (RLDDMs) in combination with hierarchical Bayesian parameter estimation to shed light on the computational underpinnings of this performance impairment. In both groups, an RLDDM in which both non-decision time and response threshold (boundary separation) changed over the course of the experiment accounted for the data best. The model showed good parameter recovery, and posterior predictive checks revealed that in both groups, the model reproduced the evolution of both accuracy and RTs over time. Examination of the group-wise posterior distributions revealed that the learning impairment in gamblers was attributable to both reduced learning rates and a more rapid reduction in boundary separation over time, compared to controls. Furthermore, gamblers also showed substantially shorter non-decision times. Model-based imaging analyses then revealed that value representations in gamblers in the ventromedial prefrontal cortex were attenuated compared to controls, and these effects were partly associated with model-based learning rates. Exploratory analyses revealed that a more anterior ventromedial prefrontal cortex cluster showed attenuations in value representations in proportion to gambling disorder severity in gamblers. Taken together, our findings reveal computational mechanisms underlying reinforcement learning impairments in gambling disorder, and confirm the ventromedial prefrontal cortex and as a critical neural hub in this disorder.

scholarly journals Reliability assessment of temporal discounting measures in virtual reality environments

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Luca R. Bruder ◽  
Lisa Scharer ◽  
Jan Peters
Keyword(s):  
Virtual Reality ◽  
Gambling Disorder ◽  
High Performance ◽  
Context Effects ◽  
Electrodermal Activity ◽  
Cue Reactivity ◽  
Real Life ◽  
Temporal Discounting ◽  
Sports Betting ◽  
Model Parameters

AbstractIn recent years the emergence of high-performance virtual reality (VR) technology has opened up new possibilities for the examination of context effects in psychological studies. The opportunity to create ecologically valid stimulation in a highly controlled lab environment is especially relevant for studies of psychiatric disorders, where it can be problematic to confront participants with certain stimuli in real life. However, before VR can be confidently applied widely it is important to establish that commonly used behavioral tasks generate reliable data within a VR surrounding. One field of research that could benefit greatly from VR-applications are studies assessing the reactivity to addiction related cues (cue-reactivity) in participants suffering from gambling disorder. Here we tested the reliability of a commonly used temporal discounting task in a novel VR set-up designed for the concurrent assessment of behavioral and psychophysiological cue-reactivity in gambling disorder. On 2 days, thirty-four healthy non-gambling participants explored two rich and navigable VR-environments (neutral: café vs. gambling-related: casino and sports-betting facility), while their electrodermal activity was measured using remote sensors. In addition, participants completed the temporal discounting task implemented in each VR environment. On a third day, participants performed the task in a standard lab testing context. We then used comprehensive computational modeling using both standard softmax and drift diffusion model (DDM) choice rules to assess the reliability of discounting model parameters assessed in VR. Test–retest reliability estimates were good to excellent for the discount rate log(k), whereas they were poor to moderate for additional DDM parameters. Differences in model parameters between standard lab testing and VR, reflecting reactivity to the different environments, were mostly numerically small and of inconclusive directionality. Finally, while exposure to VR generally increased tonic skin conductance, this effect was not modulated by the neutral versus gambling-related VR-environment. Taken together this proof-of-concept study in non-gambling participants demonstrates that temporal discounting measures obtained in VR are reliable, suggesting that VR is a promising tool for applications in computational psychiatry, including studies on cue-reactivity in addiction.

scholarly journals Reduced model-based decision-making in gambling disorder

2019 ◽  
Vol 9 (1) ◽  
Author(s):  
Florent Wyckmans ◽  
A. Ross Otto ◽  
Miriam Sebold ◽  
Nathaniel Daw ◽  
Antoine Bechara ◽  
...  
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Gambling Disorder ◽  
Reaction Times ◽  
Learning System ◽  
Behavioral Learning ◽  
Model Based ◽  
Model Free ◽  
Effective Interventions ◽  
Compulsive Behaviors

AbstractCompulsive behaviors (e.g., addiction) can be viewed as an aberrant decision process where inflexible reactions automatically evoked by stimuli (habit) take control over decision making to the detriment of a more flexible (goal-oriented) behavioral learning system. These behaviors are thought to arise from learning algorithms known as “model-based” and “model-free” reinforcement learning. Gambling disorder, a form of addiction without the confound of neurotoxic effects of drugs, showed impaired goal-directed control but the way in which problem gamblers (PG) orchestrate model-based and model-free strategies has not been evaluated. Forty-nine PG and 33 healthy participants (CP) completed a two-step sequential choice task for which model-based and model-free learning have distinct and identifiable trial-by-trial learning signatures. The influence of common psychopathological comorbidities on those two forms of learning were investigated. PG showed impaired model-based learning, particularly after unrewarded outcomes. In addition, PG exhibited faster reaction times than CP following unrewarded decisions. Troubled mood, higher impulsivity (i.e., positive and negative urgency) and current and chronic stress reported via questionnaires did not account for those results. These findings demonstrate specific reinforcement learning and decision-making deficits in behavioral addiction that advances our understanding and may be important dimensions for designing effective interventions.

Information Feedback Appraoch for the Simulation of Service Quality in the Inter-Object Communications

2008 ◽  
pp. 175-198
Author(s):  
R. Manjunath
Keyword(s):  
Neural Networks ◽  
Real Life ◽  
System Level ◽  
New Class ◽  
Model Based ◽  
Multiple Flows ◽  
Life Problems ◽  
Reference Design ◽  
The Impact ◽  
Level Performance

Simulation of a system with limited data is challenging. It calls for a certain degree of intelligence built into the system. This chapter provides a new model-based simulation methodology that may be customized and used in the simulation of a wide variety of problems involving multiple source-destination flows with intermediate agents. It explains the model based on a new class of neural networks called differentially fed artificial neural networks and the system level performance of the same. Next, as an example, the impact of system level differential feedback on multiple flows and the application of the concept are presented, followed by the simulation results. The author hopes that a variety of real life problems that involve multiple flows may be mapped onto this simulation model and optimal performance may be obtained. The model serves as a reference design that may be fine-tuned based on the application.

The Impact of Angiotensin-II Receptor Blockers on Cardiovascular Events in Hypertensive Patients – Evidence from Real-life Databases

2010 ◽  
Vol 6 (3) ◽  
pp. 33
Author(s):  
Robert J Petrella ◽  
Keyword(s):  
Angiotensin Ii ◽  
Antihypertensive Drugs ◽  
Real Life ◽  
Angiotensin Ii Receptor Blockers ◽  
Angiotensin Ii Receptor ◽  
Real World Setting ◽  
Actual Experience ◽  
Receptor Blockers ◽  
Randomised Controlled ◽  
The Impact

It is widely recognised that hypertension is a major risk factor for the development of future cardiovascular (CV) events, which in turn are a major cause of morbidity and mortality. Blood pressure (BP) control with antihypertensive drugs has been shown to reduce the risk of CV events. Angiotensin-II receptor blockers (ARBs) are one such class of antihypertensive drugs and randomised controlled trials (RCTs) have shown ARB-based therapies to have effective BP-lowering properties. However, data obtained under these tightly controlled settings do not necessarily reflect actual experience in clinical practice. Real-life databases may offer alternative information that reflects an uncontrolled real-world setting and complements and expands on the findings of clinical trials. Recent analyses of practice-based real-life databases have shown ARB-based therapies to be associated with better persistence and adherence rates and with superior BP control than non-ARB-based therapies. Analyses of real-life databases also suggest that ARB-based therapies may be associated with a lower risk of CV events than other antihypertensive-drug-based therapies.

High dietary restraint improves performance on a food-motivated probabilistic selection task

2019 ◽  
Author(s):  
Jennifer R Sadler ◽  
Grace Elisabeth Shearrer ◽  
Nichollette Acosta ◽  
Kyle Stanley Burger
Keyword(s):  
Reinforcement Learning ◽  
Instrumental Conditioning ◽  
Food Reinforcement ◽  
Dietary Restraint ◽  
Memory Task ◽  
Brain Response ◽  
Anthropometric Measures ◽  
Instrumental Task ◽  
The Impact ◽  
Reward And Punishment

BACKGROUND: Dietary restraint represents an individual’s intent to limit their food intake and has been associated with impaired passive food reinforcement learning. However, the impact of dietary restraint on an active, response dependent learning is poorly understood. In this study, we tested the relationship between dietary restraint and food reinforcement learning using an active, instrumental conditioning task. METHODS: A sample of ninety adults completed a response-dependent instrumental conditioning task with reward and punishment using sweet and bitter tastes. Brain response via functional MRI was measured during the task. Participants also completed anthropometric measures, reward/motivation related questionnaires, and a working memory task. Dietary restraint was assessed via the Dutch Restrained Eating Scale. RESULTS: Two groups were selected from the sample: high restraint (n=29, score >2.5) and low restraint (n=30; score <1.85). High restraint was associated with significantly higher BMI (p=0.003) and lower N-back accuracy (p=0.045). The high restraint group also was marginally better at the instrumental conditioning task (p=0.066, r=0.37). High restraint was also associated with significantly greater brain response in the intracalcarine cortex (MNI: 15, -69, 12; k=35, pfwe< 0.05) to bitter taste, compared to neutral taste.CONCLUSIONS: High restraint was associated with improved performance on an instrumental task testing how individuals learn from reward and punishment. This may be mediated by greater brain response in the primary visual cortex, which has been associated with mental representation. Results suggest that dietary restraint does not impair response-dependent reinforcement learning.

Model-Based and Model-Free Social Cognition

2019 ◽  
Author(s):  
Leor M Hackel ◽  
Jeffrey Jordan Berg ◽  
Björn Lindström ◽  
David Amodio
Keyword(s):  
Reinforcement Learning ◽  
Social Cognition ◽  
Learning Strategies ◽  
Memory Systems ◽  
Learning Task ◽  
Financial Advisors ◽  
Model Based ◽  
Model Free ◽  
Systems Model ◽  
Task Assessment

Do habits play a role in our social impressions? To investigate the contribution of habits to the formation of social attitudes, we examined the roles of model-free and model-based reinforcement learning in social interactions—computations linked in past work to habit and planning, respectively. Participants in this study learned about novel individuals in a sequential reinforcement learning paradigm, choosing financial advisors who led them to high- or low-paying stocks. Results indicated that participants relied on both model-based and model-free learning, such that each independently predicted choice during the learning task and self-reported liking in a post-task assessment. Specifically, participants liked advisors who could provide large future rewards as well as advisors who had provided them with large rewards in the past. Moreover, participants varied in their use of model-based and model-free learning strategies, and this individual difference influenced the way in which learning related to self-reported attitudes: among participants who relied more on model-free learning, model-free social learning related more to post-task attitudes. We discuss implications for attitudes, trait impressions, and social behavior, as well as the role of habits in a memory systems model of social cognition.

Sign in / Sign up

Export Citation Format

Share Document