scholarly journals Influenca: a gamified assessment of value-based decision-making for longitudinal studies

2021 ◽  
Author(s):  
Monja P. Neuser ◽  
Franziska Kräutlein ◽  
Anne Kühnel ◽  
Vanessa Teckentrup ◽  
Jennifer Svaldi ◽  
...  
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Learning Task ◽  
Reward Learning ◽  
Initial Validation ◽  
Cross Platform ◽  
Test Retest Reliability ◽  
Ecological Momentary ◽  
Momentary Assessment ◽  
Effective Medication

AbstractReinforcement learning is a core facet of motivation and alterations have been associated with various mental disorders. To build better models of individual learning, repeated measurement of value-based decision-making is crucial. However, the focus on lab-based assessment of reward learning has limited the number of measurements and the test-retest reliability of many decision-related parameters is therefore unknown. Here, we developed an open-source cross-platform application Influenca that provides a novel reward learning task complemented by ecological momentary assessment (EMA) for repeated assessment over weeks. In this task, players have to identify the most effective medication by selecting the best option after integrating offered points with changing probabilities (according to random Gaussian walks). Participants can complete up to 31 levels with 150 trials each. To encourage replay on their preferred device, in-game screens provide feedback on the progress. Using an initial validation sample of 127 players (2904 runs), we found that reinforcement learning parameters such as the learning rate and reward sensitivity show low to medium intra-class correlations (ICC: 0.22-0.52), indicating substantial within- and between-subject variance. Notably, state items showed comparable ICCs as reinforcement learning parameters. To conclude, our innovative and openly customizable app framework provides a gamified task that optimizes repeated assessments of reward learning to better quantify intra- and inter-individual differences in value-based decision-making over time.

scholarly journals Effects of Depressive Symptoms, Feelings, and Interoception on Reward-Based Decision-Making: Investigation Using Reinforcement Learning Model

2020 ◽  
Vol 10 (8) ◽  
pp. 508
Author(s):  
Hiroyoshi Ogishima ◽  
Shunta Maeda ◽  
Yuki Tanaka ◽  
Hironori Shimada
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Depressive Symptoms ◽  
Emotional Experience ◽  
Learning Task ◽  
Experimental Manipulation ◽  
Detection Task ◽  
Control Group ◽  
Before And After ◽  
Experimental Group

Background: In this study, we examined the relationships between reward-based decision-making in terms of learning rate, memory rate, exploration rate, and depression-related subjective emotional experience, in terms of interoception and feelings, to understand how reward-based decision-making is impaired in depression. Methods: In all, 52 university students were randomly assigned to an experimental group and a control group. To manipulate interoception, the participants in the experimental group were instructed to tune their internal somatic sense to the skin-conductance-response waveform presented on a display. The participants in the control group were only instructed to stay relaxed. Before and after the manipulation, the participants completed a probabilistic reversal-learning task to assess reward-based decision-making using reinforcement learning modeling. Similarly, participants completed a probe-detection task, a heartbeat-detection task, and self-rated scales. Results: The experimental manipulation of interoception was not successful. In the baseline testing, reinforcement learning modeling indicated a marginally-significant correlation between the exploration rate and depressive symptoms. However, the exploration rate was significantly associated with lower interoceptive attention and higher depressive feeling. Conclusions: The findings suggest that situational characteristics may be closely involved in reward exploration and highlight the clinically-meaningful possibility that intervention for affective processes may impact reward-based decision-making in those with depression.

scholarly journals What Do People With Schizophrenia Do All Day? Ecological Momentary Assessment of Real-World Functioning in Schizophrenia

2019 ◽  
Author(s):  
Eric Granholm ◽  
Jason L Holden ◽  
Tanya Mikhael ◽  
Peter C Link ◽  
Joel Swendsen ◽  
...  
Keyword(s):  
Social Interactions ◽  
Real World ◽  
Ecological Momentary Assessment ◽  
Reliability And Validity ◽  
Supported Housing ◽  
Self Report ◽  
New Treatments ◽  
Test Retest Reliability ◽  
Ecological Momentary ◽  
Momentary Assessment

Abstract Schizophrenia is a major cause of disability worldwide. As new treatments for functioning are tested, the need grows to demonstrate real-world functioning gains. Ecological momentary assessment (EMA) may provide a more ecologically valid measure of functioning. In this study, smartphone-based EMA was used to signal participants with schizophrenia (N = 100) and controls (N = 71) 7 times a day for 7 days to respond to brief questionnaires about social interactions and functioning behaviors. Excellent adherence was found, with both groups completing an average of 85% of surveys and only 3% of participants with schizophrenia excluded for poor adherence. Four-week test–retest reliability was high (r = .83 for total productive behaviors). Relative to controls, participants with schizophrenia reported significantly less total productive activity (d = 1.2), fewer social interactions (d = 0.3), more nonproductive behaviors (d = 1.0; watching TV, resting), and more time at home (d = 0.8). Within the schizophrenia group, participants living independently showed better functioning on EMA relative to participants in supported housing (d = 0.8) and participants engaged in vocational activities showed better functioning than individuals not engaged in vocational activities (d = 0.55). Modest correlations were found between EMA and an in-lab self-report measure of functioning activities performed in the community, but not between EMA and measures of functional capacity or potential. This study demonstrated the feasibility, sensitivity reliability, and validity of EMA methods to assess functioning in schizophrenia. EMA provides a much-needed measure of what individuals with schizophrenia are actually doing in real-world contexts. These results also suggest that there may be important disjunctions between indices of abilities and actual real-world functioning.

scholarly journals Deficits in reinforcement learning but no link to apathy in patients with schizophrenia

2017 ◽  
Vol 7 (1) ◽  
Author(s):  
Matthias N. Hartmann-Riemer ◽  
Steffen Aschenbrenner ◽  
Magdalena Bossert ◽  
Celina Westermann ◽  
Erich Seifritz ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Negative Symptoms ◽  
Learning Task ◽  
Reward Learning ◽  
Positive Outcomes ◽  
Learning Deficits ◽  
Loss Avoidance ◽  
Healthy Control ◽  
Precise Representation ◽  
The Stability

Abstract Negative symptoms in schizophrenia have been linked to selective reinforcement learning deficits in the context of gains combined with intact loss-avoidance learning. Fundamental mechanisms of reinforcement learning and choice are prediction error signaling and the precise representation of reward value for future decisions. It is unclear which of these mechanisms contribute to the impairments in learning from positive outcomes observed in schizophrenia. A recent study suggested that patients with severe apathy symptoms show deficits in the representation of expected value. Considering the fundamental relevance for the understanding of these symptoms, we aimed to assess the stability of these findings across studies. Sixty-four patients with schizophrenia and 19 healthy control participants performed a probabilistic reward learning task. They had to associate stimuli with gain or loss-avoidance. In a transfer phase participants indicated valuation of the previously learned stimuli by choosing among them. Patients demonstrated an overall impairment in learning compared to healthy controls. No effects of apathy symptoms on task indices were observed. However, patients with schizophrenia learned better in the context of loss-avoidance than in the context of gain. Earlier findings were thus partially replicated. Further studies are needed to clarify the mechanistic link between negative symptoms and reinforcement learning.

scholarly journals Trading off the cost of conflict against expected rewards

2018 ◽  
Author(s):  
Nura Sidarus ◽  
Stefano Palminteri ◽  
Valérian Chambon
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Cognitive Control ◽  
Reversal Learning ◽  
Computational Models ◽  
Learning Task ◽  
Learning Rates ◽  
Different Types ◽  
Irrelevant Distractors ◽  
The Cost

AbstractValue-based decision-making involves trading off the cost associated with an action against its expected reward. Research has shown that both physical and mental effort constitute such subjective costs, biasing choices away from effortful actions, and discounting the value of obtained rewards. Facing conflicts between competing action alternatives is considered aversive, as recruiting cognitive control to overcome conflict is effortful. Yet, it remains unclear whether conflict is also perceived as a cost in value-based decisions. The present study investigated this question by embedding irrelevant distractors (flanker arrows) within a reversal-learning task, with intermixed free and instructed trials. Results showed that participants learned to adapt their choices to maximize rewards, but were nevertheless biased to follow the suggestions of irrelevant distractors. Thus, the perceived cost of being in conflict with an external suggestion could sometimes trump internal value representations. By adapting computational models of reinforcement learning, we assessed the influence of conflict at both the decision and learning stages. Modelling the decision showed that conflict was avoided when evidence for either action alternative was weak, demonstrating that the cost of conflict was traded off against expected rewards. During the learning phase, we found that learning rates were reduced in instructed, relative to free, choices. Learning rates were further reduced by conflict between an instruction and subjective action values, whereas learning was not robustly influenced by conflict between one’s actions and external distractors. Our results show that the subjective cost of conflict factors into value-based decision-making, and highlights that different types of conflict may have different effects on learning about action outcomes.

Psychometric Properties and Prospective Predictive Utility of the Contrast Avoidance Questionnaires

2020 ◽  
pp. 1-13
Author(s):  
Evan J. White ◽  
DeMont M. Grant ◽  
Jacob D. Kraft ◽  
Danielle L. Taylor ◽  
Danielle E. Deros ◽  
...  
Keyword(s):  
Negative Affect ◽  
Predictive Ability ◽  
Self Report ◽  
Theoretical Research ◽  
Predictive Utility ◽  
Independent Evaluation ◽  
Key Factor ◽  
Test Retest Reliability ◽  
Ecological Momentary ◽  
Momentary Assessment

Abstract. Recent theoretical research posits that a key factor in the development and maintenance of pathological worry is the avoidance of sudden emotional shifts. Recently two self-report instruments were developed to index this phenomenon (Contrast Avoidance Questionnaire – Worry [CAQ-W] and Contrast Avoidance Questionnaire – General Emotion [CAQ-GE]). This work employed a multi-study design to provide an independent evaluation of the latent structure of these measures; additionally, the validity and longitudinal predictive ability of the measures was examined. Findings of the factor analytic work support a two-factor solution for each of the questionnaires. These were defined as the following subscales: CAQ-W: Emotional Contrast; Negative Affect, and for the CAQ-GE: Avoidance; Discomfort. Test-retest reliability of the measure was strong and the identified subscales demonstrated differential predictive ability regarding future worry and depression. Results indicate that subscales related to the experience of negative affect rather than avoidance, display predictive utility of future symptoms. These findings are somewhat discrepant with extant literature on ecological momentary assessment (EMA) assessment of contrast avoidance indicating the need for refinement in the self-report measurement of this construct.

Planning Complexity Registers as a Cost in Metacontrol

2018 ◽  
Vol 30 (10) ◽  
pp. 1391-1404 ◽  
Author(s):  
Wouter Kool ◽  
Samuel J. Gershman ◽  
Fiery A. Cushman
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Formal Analysis ◽  
Learning Task ◽  
Costs And Benefits ◽  
Model Free ◽  
Versus Model ◽  
Specific Manner ◽  
Decision Making Processes ◽  
Planning Approaches

Decision-making algorithms face a basic tradeoff between accuracy and effort (i.e., computational demands). It is widely agreed that humans can choose between multiple decision-making processes that embody different solutions to this tradeoff: Some are computationally cheap but inaccurate, whereas others are computationally expensive but accurate. Recent progress in understanding this tradeoff has been catalyzed by formalizing it in terms of model-free (i.e., habitual) versus model-based (i.e., planning) approaches to reinforcement learning. Intuitively, if two tasks offer the same rewards for accuracy but one of them is much more demanding, we might expect people to rely on habit more in the difficult task: Devoting significant computation to achieve slight marginal accuracy gains would not be “worth it.” We test and verify this prediction in a sequential reinforcement learning task. Because our paradigm is amenable to formal analysis, it contributes to the development of a computational model of how people balance the costs and benefits of different decision-making processes in a task-specific manner; in other words, how we decide when hard thinking is worth it.

scholarly journals Test-Retest Reliability of Ecological Momentary Assessment in Audiology Research

2020 ◽  
Vol 31 (08) ◽  
pp. 599-612
Author(s):  
Yu-Hsiang Wu ◽  
Elizabeth Stangl ◽  
Octav Chipara ◽  
Xuyang Zhang
Keyword(s):  
Ecological Momentary Assessment ◽  
Hearing Aid ◽  
Speech Understanding ◽  
Listening Effort ◽  
Retest Reliability ◽  
Test Retest Reliability ◽  
Ecological Momentary ◽  
Talker Familiarity ◽  
Momentary Assessment ◽  
Context Characteristics

Abstract Background Ecological momentary assessment (EMA) is a methodology involving repeated surveys to collect in situ data that describe respondents' current or recent experiences and related contexts in their natural environments. Audiology literature investigating the test-retest reliability of EMA is scarce. Purpose This article examines the test-retest reliability of EMA in measuring the characteristics of listening contexts and listening experiences. Research Design An observational study. Study Sample Fifty-one older adults with hearing loss. Data Collection and Analysis The study was part of a larger study that examined the effect of hearing aid technologies. The larger study had four trial conditions and outcome was measured using a smartphone-based EMA system. After completing the four trial conditions, participants repeated one of the conditions to examine the EMA test-retest reliability. The EMA surveys contained questions that assessed listening context characteristics including talker familiarity, talker location, and noise location, as well as listening experiences including speech understanding, listening effort, loudness satisfaction, and hearing aid satisfaction. The data from multiple EMA surveys collected by each participant were aggregated in each of the test and retest conditions. Test-retest correlation on the aggregated data was then calculated for each EMA survey question to determine the reliability of EMA. Results At the group level, listening context characteristics and listening experience did not change between the test and retest conditions. The test-retest correlation varied across the EMA questions, with the highest being the questions that assessed talker location (median r = 1.0), reverberation (r = 0.89), and speech understanding (r = 0.85), and the lowest being the items that quantified noise location (median r = 0.63), talker familiarity (r = 0.46), listening effort (r = 0.61), loudness satisfaction (r = 0.60), and hearing aid satisfaction (r = 0.61). Conclusion Several EMA questions yielded appropriate test-retest reliability results. The lower test-retest correlations for some EMA survey questions were likely due to fewer surveys completed by participants and poorly designed questions. Therefore, the present study stresses the importance of using validated questions in EMA. With sufficient numbers of surveys completed by respondents and with appropriately designed survey questions, EMA could have reasonable test-retest reliability in audiology research.

scholarly journals Neural Signatures of Prediction Errors in a Decision-Making Task Are Modulated by Action Execution Failures

2018 ◽  
Author(s):  
Samuel D. McDougle ◽  
Peter A. Butcher ◽  
Darius Parvin ◽  
Fasial Mushtaq ◽  
Yael Niv ◽  
...  
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Prediction Error ◽  
Learning Task ◽  
Prediction Errors ◽  
Action Execution ◽  
Model Driven ◽  
Assignment Of Credit ◽  
Level Information ◽  
High Level

AbstractDecisions must be implemented through actions, and actions are prone to error. As such, when an expected outcome is not obtained, an individual should not only be sensitive to whether the choice itself was suboptimal, but also whether the action required to indicate that choice was executed successfully. The intelligent assignment of credit to action execution versus action selection has clear ecological utility for the learner. To explore this scenario, we used a modified version of a classic reinforcement learning task in which feedback indicated if negative prediction errors were, or were not, associated with execution errors. Using fMRI, we asked if prediction error computations in the human striatum, a key substrate in reinforcement learning and decision making, are modulated when a failure in action execution results in the negative outcome. Participants were more tolerant of non-rewarded outcomes when these resulted from execution errors versus when execution was successful but the reward was withheld. Consistent with this behavior, a model-driven analysis of neural activity revealed an attenuation of the signal associated with negative reward prediction error in the striatum following execution failures. These results converge with other lines of evidence suggesting that prediction errors in the mesostriatal dopamine system integrate high-level information during the evaluation of instantaneous reward outcomes.

Sign in / Sign up

Export Citation Format

Share Document