scholarly journals Increased temporal discounting and reduced model-based control in problem gambling are not substantially modulated by exposure to virtual gambling environments.

2021 ◽  
Author(s):  
Luca Rene Bruder ◽  
Ben Wagner ◽  
David Mathar ◽  
Jan Peters
Keyword(s):  
Pathological Gambling ◽  
Electrodermal Activity ◽  
Temporal Discounting ◽  
Sports Betting ◽  
Reduced Model ◽  
Sequential Decision ◽  
Model Based Control ◽  
Model Based ◽  
Model Free ◽  
Environment Exposure

High-performance virtual reality (VR) technology has opened new possibilities for the examination of the reactivity towards addiction-related cues (cue-reactivity) in addiction. In this preregistered study (https://osf.io/4mrta), we investigated the subjective, physiological, and behavioral effects of gambling-related VR environment exposure in participants reporting frequent or pathological gambling (n=31) as well as non-gambling controls (n=29). On two separate days, participants explored two rich and navigable VR-environments (neutral: cafe vs. gambling-related: casino/sports-betting facility), while electrodermal activity and heart rate were continuously measured using remote sensors. Within VR, participants performed a temporal discounting task and a sequential decision-making task designed to assess model-based and model-free contributions to behavior. Replicating previous findings, we found strong evidence for increased temporal discounting and reduced model-based control in participants reporting frequent or pathological gambling. Although VR gambling environment exposure increased subjective craving, there was if anything inconclusive evidence for further behavioral or physiological effects. Instead, VR exposure substantially increased physiological arousal (electrodermal activity), across groups and conditions. VR is a promising tool for the investigation of context effects in addiction, but some caution is warranted since effects of real gambling environments might not generally replicate in VR. Future studies should delineate how factors such as cognitive load and ecological validity could be balanced to create a more naturalistic VR experience.

scholarly journals Metacontrol of decision-making strategies in human aging

2019 ◽  
Author(s):  
Florian Bolenz ◽  
Wouter Kool ◽  
Andrea M.F. Reiter ◽  
Ben Eppinger
Keyword(s):  
Older Adults ◽  
Decision Making ◽  
Cognitive Abilities ◽  
Reduced Model ◽  
Task Structure ◽  
Costs And Benefits ◽  
Sequential Decision ◽  
Model Based Control ◽  
Model Based ◽  
Model Free

When making decisions, humans employ different strategies which are commonly formalized as model-free and model-based reinforcement learning. While previous research has reported reduced model-based control with aging, it remains unclear whether this is due to limited cognitive capacities or a reduced willingness to engage in an effortful strategy. Moreover, it is not clear how aging affects the metacontrol of decision making, i.e. the dynamic adaptation of decision-making strategies to varying situational demands. To this end, we tested younger and older adults in a sequential decision-making task that dissociates model-free and model-based control. In contrast to previous research, in this study we applied a task in which model-based control led to higher payoffs in terms of monetary reward. Moreover, we manipulated the costs and benefits associated with model-based control by varying reward magnitude as well as the stability of the task structure. Compared to younger adults, older adults showed reduced reliance on model-based decision making and less adaptation of decision-making strategies to varying costs and benefits of model-based control. Our findings suggest that aging affects the dynamic metacontrol of decision-making strategies and that reduced model-based control in older adults is due to limited cognitive abilities to represent the structure of the task.

scholarly journals Ventral striatal dopamine reflects behavioral and neural signatures of model-based control during sequential decision making

2015 ◽  
Vol 112 (5) ◽  
pp. 1595-1600 ◽  
Author(s):  
Lorenz Deserno ◽  
Quentin J. M. Huys ◽  
Rebecca Boehme ◽  
Ralph Buchert ◽  
Hans-Jochen Heinze ◽  
...  
Keyword(s):  
Ventral Striatum ◽  
Behavioral Control ◽  
Striatal Dopamine ◽  
Quantitative Relation ◽  
Sequential Decision ◽  
Model Based Control ◽  
Model Based ◽  
Model Free ◽  
Neuropsychiatric Diseases ◽  
Neural Signatures

Dual system theories suggest that behavioral control is parsed between a deliberative “model-based” and a more reflexive “model-free” system. A balance of control exerted by these systems is thought to be related to dopamine neurotransmission. However, in the absence of direct measures of human dopamine, it remains unknown whether this reflects a quantitative relation with dopamine either in the striatum or other brain areas. Using a sequential decision task performed during functional magnetic resonance imaging, combined with striatal measures of dopamine using [18F]DOPA positron emission tomography, we show that higher presynaptic ventral striatal dopamine levels were associated with a behavioral bias toward more model-based control. Higher presynaptic dopamine in ventral striatum was associated with greater coding of model-based signatures in lateral prefrontal cortex and diminished coding of model-free prediction errors in ventral striatum. Thus, interindividual variability in ventral striatal presynaptic dopamine reflects a balance in the behavioral expression and the neural signatures of model-free and model-based control. Our data provide a novel perspective on how alterations in presynaptic dopamine levels might be accompanied by a disruption of behavioral control as observed in aging or neuropsychiatric diseases such as schizophrenia and addiction.

scholarly journals Metacontrol of decision-making strategies in human aging

eLife ◽  
2019 ◽  
Vol 8 ◽  
Author(s):  
Florian Bolenz ◽  
Wouter Kool ◽  
Andrea MF Reiter ◽  
Ben Eppinger
Keyword(s):  
Older Adults ◽  
Decision Making ◽  
Cognitive Abilities ◽  
Cross Sectional Study ◽  
Reduced Model ◽  
Task Structure ◽  
Sequential Decision ◽  
Cross Sectional ◽  
Model Based ◽  
Model Free

Humans employ different strategies when making decisions. Previous research has reported reduced reliance on model-based strategies with aging, but it remains unclear whether this is due to cognitive or motivational factors. Moreover, it is not clear how aging affects the metacontrol of decision making, that is the dynamic adaptation of decision-making strategies to varying situational demands. In this cross-sectional study, we tested younger and older adults in a sequential decision-making task that dissociates model-free and model-based strategies. In contrast to previous research, model-based strategies led to higher payoffs. Moreover, we manipulated the costs and benefits of model-based strategies by varying reward magnitude and the stability of the task structure. Compared to younger adults, older adults showed reduced model-based decision making and less adaptation of decision-making strategies. Our findings suggest that aging affects the metacontrol of decision-making strategies and that reduced model-based strategies in older adults are due to limited cognitive abilities.

scholarly journals Hand interception of occluded motion in humans: a test of model-based vs. on-line control

2015 ◽  
Vol 114 (3) ◽  
pp. 1577-1592 ◽  
Author(s):  
Barbara La Scaleia ◽  
Myrka Zago ◽  
Francesco Lacquaniti
Keyword(s):  
Visual Information ◽  
Target Motion ◽  
Physical Models ◽  
Model Based Control ◽  
Projectile Motion ◽  
Starting Position ◽  
Model Based ◽  
Model Free ◽  
Ball Velocity ◽  
On Line

Two control schemes have been hypothesized for the manual interception of fast visual targets. In the model-free on-line control, extrapolation of target motion is based on continuous visual information, without resorting to physical models. In the model-based control, instead, a prior model of target motion predicts the future spatiotemporal trajectory. To distinguish between the two hypotheses in the case of projectile motion, we asked participants to hit a ball that rolled down an incline at 0.2 g and then fell in air at 1 g along a parabola. By varying starting position, ball velocity and trajectory differed between trials. Motion on the incline was always visible, whereas parabolic motion was either visible or occluded. We found that participants were equally successful at hitting the falling ball in both visible and occluded conditions. Moreover, in different trials the intersection points were distributed along the parabolic trajectories of the ball, indicating that subjects were able to extrapolate an extended segment of the target trajectory. Remarkably, this trend was observed even at the very first repetition of movements. These results are consistent with the hypothesis of model-based control, but not with on-line control. Indeed, ball path and speed during the occlusion could not be extrapolated solely from the kinematic information obtained during the preceding visible phase. The only way to extrapolate ball motion correctly during the occlusion was to assume that the ball would fall under gravity and air drag when hidden from view. Such an assumption had to be derived from prior experience.

Nonlinear Model Reduction for Automotive System Descriptions

2001 ◽  
Author(s):  
Zeyu Liu ◽  
John Wagner
Keyword(s):  
Dynamic System ◽  
Model Reduction ◽  
Nonlinear Model ◽  
Order System ◽  
Control Algorithms ◽  
Model Based Control ◽  
Reduction Strategy ◽  
Nonlinear Model Reduction ◽  
Model Based ◽  
Model Free

Abstract The mathematical modeling of dynamic systems is an important task in the design, analysis, and implementation of advanced automotive control systems. Although most vehicle control algorithms tend to use model-free calibration architectures, a need exists to migrate to model-based control algorithms which offer greater operating performance. However, in many instances, the analytical descriptions are too complex for real-time powertrain and chassis model-based control algorithms. Therefore, model reduction strategies may be applied to transform the original model into a simplified lower-order form while preserving the dynamic characteristics of the original high-order system. In this paper, an empirical gramian balanced nonlinear model reduction strategy is examined for the simplification process of dynamic system descriptions. The empirical gramians may be computed using either experimental or simulation data. These gramians are then balanced and unimportant system dynamics truncated. For comparison purposes, a Taylor Series linearization will also be introduced to linearize the original nonlinear system about an equilibrium operating point and then a balanced realization linear reduction strategy will be applied. To demonstrate the functionality of each model reduction strategy, two nonlinear dynamic system models are investigated and respective transient performances compared.

scholarly journals No substantial change in the balance between model-free and model-based control via training on the two-step task

2019 ◽  
Vol 15 (11) ◽  
pp. e1007443
Author(s):  
Elmar D. Grosskurth ◽  
Dominik R. Bach ◽  
Marcos Economides ◽  
Quentin J. M. Huys ◽  
Lisa Holper
Keyword(s):  
Model Based Control ◽  
Model Based ◽  
Model Free ◽  
Substantial Change

scholarly journals Έλεγχος εξασφάλισης κύλισης σε σφαιρικά ρομποτικά ακροδάκτυλα

2016 ◽  
Author(s):  
Λεωνίδας Δρούκας
Keyword(s):  
Performance Control ◽  
Model Based Control ◽  
Prescribed Performance ◽  
Model Based ◽  
Model Free ◽  
Prescribed Performance Control ◽  
Model Free Control ◽  
Control Methodology

Όπως στην περίπτωση του ανθρώπου έτσι και στην περίπτωση των δακτύλων ενός ρομποτικού χεριού, η κίνηση μέσω κύλισης των ακροδακτύλων είναι καθοριστική για την επίτευξη της ευσταθούς σύλληψης και του ομαλού χειρισμού ενός αντικειμένου από το ρομποτικό χέρι. Σε αντίθεση με μια κίνηση ολίσθησης πάνω στην επιφάνεια με την οποία έρχεται σε επαφή το ρομποτικό δάκτυλο, η κύλιση του ακροδακτύλου βοηθάει στην ακριβέστερη τοποθέτησή του πάνω στην επιφάνεια αυτή, συμβάλλοντας έτσι στον ομαλότερο συνολικά χειρισμό του εκάστοτε αντικειμένου όπως για παράδειγμα η μετακίνηση ή η περιστροφή του. Στην υπάρχουσα βιβλιογραφία, οι περισσότεροι ελεγκτικοί νόμοι που έχουν προταθεί σχεδιάζονται με βάση ένα εξιδανικευμένο μοντέλο του συστήματος ρομποτικού δακτύλου - επιφάνειας επαφής. Το μοντέλο αυτό περιλαμβάνει ενσωματωμένους τους δεσμούς κύλισης του ακροδακτύλου, με αποτέλεσμα η κίνηση κύλισης να θεωρείται επί της ουσίας ως κάτι δεδομένο και εκ των προτέρων εξασφαλισμένο. Στην πραγματικότητα φυσικά το παραπάνω δεν ισχύει, καθώς οι εκάστοτε υπάρχουσες συνθήκες τριβής που εξαρτώνται από τα υλικά του ακροδακτύλου και της επιφάνειας μπορεί να ευνοούν λίγο ή και καθόλου την κύλιση του πρώτου πάνω στην δεύτερη. Έτσι στις περιπτώσεις αυτές, οι παραπάνω ελεγκτικές μεθοδολογίες είναι πολύ πιθανό να οδηγήσουν το ακροδάκτυλο σε μια κίνηση που θα αποτελείται από συνδυασμένη κύλιση και ολίσθηση ή ακόμη και μόνον από ολίσθηση. Στην διδακτορική αυτή διατριβή, η εξασφάλιση της κύλισης του ακροδακτύλου πάνω σε μια επιφάνεια επαφής δεν θεωρείται δεδομένη από πριν, αλλά λαμβάνεται υπόψιν ως ένας επιπλέον στόχος ελέγχου. Οι ελεγκτές που προτείνονται επιτυγχάνουν την διασφάλιση της κύλισης του σφαιρικού άκρου ενός ρομποτικού δακτύλου σε μια οποιαδήποτε επιφάνεια με άγνωστες ή και μη ευνοϊκές συνθήκες τριβής, συνδυάζοντας παράλληλα την επίτευξη του ευρέως διαδεδομένου στη ρομποτική ελέγχου της θέσης του ακροδακτύλου και του μέτρου της ασκούμενης από αυτό κάθετης δύναμης. Σχεδιάζονται ελεγκτές οι οποίοι είτε βασίζονται στην γνώση του μοντέλου του ρομποτικού δακτύλου (model based control), είτε δεν απαιτούν καμία τέτοια πληροφορία (model free control). Για την σχεδίαση των τελευταίων χρησιμοποιείται η Μέθοδος Ελέγχου Προδιαγεγραμμένης Επίδοσης (Prescribed Performance Control Methodology). Για όλους τους προτεινόμενους ελεγκτές, παρουσιάζεται η θεωρητική σχεδίαση και η προσομοιακή τους αξιολόγηση που αποδεικνύουν την εξασφάλιση του βασικότερου ελεγκτικού μας στόχου, δηλαδή της κύλισης του ρομποτικού ακροδακτύλου πάνω στην επιφάνεια επαφής, ενώ σε ορισμένες περιπτώσεις παρουσιάζονται επιπροσθέτως και πειραματικά αποτελέσματα που ενισχύουν ακόμα περισσότερο την αποτελεσματικότητα των αντίστοιχων προτεινόμενων ελεγκτών. Τέλος, η εξασφάλιση της κύλισης του ακροδακτύλου μέσω ενός από τους προτεινόμενους ελεγκτές αξιοποιείται για την εφαρμογή κατάλληλης εφαπτομενικής δύναμης ώστε να επιτευχθεί ο επιθυμητός χειρισμός χωρίς σύλληψη ενός αντικειμένου από ένα ρομποτικό δάκτυλο. Ειδικότερα, παρουσιάζεται μέσω προσομοιώσεων η επιτυχής μετακίνηση ή/και περιστροφή ενός επίπεδου και ενός κυλινδρικού αντικειμένου μέσω χειρισμού χωρίς σύλληψη από ένα ρομποτικό δάκτυλο με εκμετάλλευση της κύλισης του άκρου του πάνω στο εκάστοτε αντικείμενο.

scholarly journals Testing the Possibility of Model-based Pavlovian Control of Attention to Threat

2019 ◽  
Vol 31 (1) ◽  
pp. 36-48 ◽  
Author(s):  
Deborah Talmi ◽  
Martina Slapkova ◽  
Matthias J. Wieser
Keyword(s):  
Attention Capture ◽  
Conditioned Stimuli ◽  
Learning Stage ◽  
Internal Representations ◽  
Model Based Control ◽  
Model Based ◽  
Model Free ◽  
Role Of Information ◽  
Instrumental Control

Signals for reward or punishment attract attention preferentially, a principle termed value-modulated attention capture (VMAC). The mechanisms that govern the allocation of attention can be described with a terminology that is more often applied to the control of overt behaviors, namely, the distinction between instrumental and Pavlovian control, and between model-free and model-based control. Although instrumental control of VMAC can be either model-free or model-based, it is not known whether Pavlovian control of VMAC can be model-based. To decide whether this is possible, we measured steady-state visual evoked potentials (SSVEPs) while 20 healthy adults took part in a novel task. During the learning stage, participants underwent aversive threat conditioning with two conditioned stimuli (CSs): one that predicted pain (CS+) and one that predicted safety (CS−). Instructions given before the test stage allowed participants to infer whether novel, ambiguous CSs (new_CS+/new_CS−) were threatening or safe. Correct inference required combining stored internal representations and new propositional information, the hallmark of model-based control. SSVEP amplitudes quantified the amount of attention allocated to novel CSs on their very first presentation, before they were ever reinforced. We found that SSVEPs were higher for new_CS+ than new_CS−. This result is potentially indicative of model-based Pavlovian control of VMAC, but additional controls are necessary to verify this conclusively. This result underlines the potential transformative role of information and inference in emotion regulation.

Sign in / Sign up

Export Citation Format

Share Document