scholarly journals Context-dependent reinforcement learning impairment in depression

2021 ◽  
Author(s):  
henri Vandendriessche ◽  
Amel Demmou ◽  
Sophie Bavard ◽  
Julien Yadak ◽  
Cédric Lemogne ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Positive Reinforcement ◽  
Original Learning ◽  
Expected Value ◽  
Learning Performance ◽  
Transfer Phase ◽  
Learning Context ◽  
Context Analysis ◽  
The Poor ◽  
Context Dependent

Backgrounds:Value-based decision-making impairment in depression is a complex phenomenon: while some studies did find evidence of blunted reward learning and reward-related signals in the brain, others indicate no effect. Here we test whether such reward sensitivity deficits are dependent on the overall value of the decision problem.Methods:We used a two-armed bandit task that includes two different contexts: one ‘rich’ context where both options were associated with an overall positive expected value and a ‘poor’ context where options were associated with overall negative expected value. We tested patients (N=30) undergoing a major depressive episode and age, gender and socio-economically matched controls (N=26). To assess whether differences in learning performance were due to a decision or a value-update process, we also analysed performance in a transfer phase, performed immediately after the learning phase. ResultsHealthy subjects showed similar learning performance in the ‘rich’ and the ‘poor’ contexts, while patients showed reduced learning in the ‘poor’ context. Analysis of the transfer phase showed that the context-dependent deficit in patients generalized when options were extrapolated from their original learning context, thus suggesting that the effect of depression has to be traced to the outcome encoding, rather than the decision phase.ConclusionsOur results illustrate that reinforcement learning deficits in depression are complex and depend on the value of the context. We show that depressive patients have a specific trouble in contexts with an overall negative state value, supporting the relevance of setting up patients in a spiral of positive reinforcement.

scholarly journals Context-dependent memory effects in two immersive virtual reality environments: On Mars and underwater

2020 ◽  
Author(s):  
Yeon Soon Shin ◽  
Rolando Masís-Obando ◽  
Neggin Keshavarzian ◽  
Riya Dáve ◽  
Kenneth A. Norman
Keyword(s):  
Virtual Reality ◽  
Memory Effect ◽  
Original Learning ◽  
Memory Effects ◽  
Immersive Virtual Reality ◽  
Laboratory Setting ◽  
Learning Context ◽  
Body Of Knowledge ◽  
Context Dependent ◽  
Study Context

AbstractThe context-dependent memory effect, in which memory for an item is better when the retrieval context matches the original learning context, has proved to be difficult to reproduce in a laboratory setting. In an effort to identify a set of features that generate a robust context-dependent memory effect, we developed a paradigm in virtual reality using two semantically distinct virtual contexts: underwater and Mars environments, each with a separate body of knowledge (schema) associated with it. We show that items are better recalled when retrieved in the same context as the study context; we also show that the size of the effect is larger for items deemed context-relevant at encoding, suggesting that context-dependent memory effects may depend on items being integrated into an active schema.

scholarly journals Impaired expected value computations coupled with overreliance on prediction error learning in schizophrenia

2017 ◽  
Author(s):  
D Hernaus ◽  
JM Gold ◽  
JA Waltz ◽  
MJ Frank
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Prediction Error ◽  
Expected Value ◽  
Strongly Correlated ◽  
Transfer Phase ◽  
Robust Performance ◽  
Reward Prediction ◽  
Error Learning ◽  
Choice Bias

AbstractBackgroundWhile many have emphasized impaired reward prediction error (RPE) signaling in schizophrenia, multiple studies suggest that some decision-making deficits may arise from overreliance on RPE systems together with a compromised ability to represent expected value. Guided by computational frameworks, we formulated and tested two scenarios in which maladaptive representation of expected value should be most evident, thereby delineating conditions that may evoke decision-making impairments in schizophrenia.MethodsIn a modified reinforcement learning paradigm, 42 medicated people with schizophrenia (PSZ) and 36 healthy volunteers learned to select the most frequently rewarded option in a 75-25 pair: once when presented with more deterministic (90–10) and once when presented with more probabilistic (60–40) pairs. Novel and old combinations of choice options were presented in a subsequent transfer phase. Computational modeling was employed to elucidate contributions from RPE systems (“actor-critic”) and expected value (“Q-leaming”).ResultsPSZ showed robust performance impairments with increasing value difference between two competing options, which strongly correlated with decreased contributions from expected value-based (“Q-leaming”) learning. Moreover, a subtle yet consistent contextual choice bias for the “probabilistic” 75 option was present in PSZ, which could be accounted for by a context-dependent RPE in the “actor-critic”.ConclusionsWe provide evidence that decision-making impairments in schizophrenia increase monotonically with demands placed on expected value computations. A contextual choice bias is consistent with overreliance on RPE-based learning, which may signify a deficit secondary to the maladaptive representation of expected value. These results shed new light on conditions under which decisionmaking impairments may arise.

scholarly journals Context-dependent memory effects in VR

2020 ◽  
Author(s):  
Yeon Soon Shin ◽  
Rolando Masís-Obando ◽  
Neggin Keshavarzian ◽  
Riya Davé ◽  
Kenneth Norman
Keyword(s):  
Virtual Reality ◽  
Memory Effect ◽  
Original Learning ◽  
Memory Effects ◽  
Laboratory Setting ◽  
Learning Context ◽  
Body Of Knowledge ◽  
Context Dependent ◽  
Study Context

The context-dependent memory effect, in which memory for an item is better when the retrieval context matches the original learning context, has proved to be difficult to reproduce in a laboratory setting. In an effort to identify a set of features that generate a robust context-dependent memory effect, we developed a paradigm in virtual reality using two semantically distinct virtual contexts: underwater and Mars environments, each with a separate body of knowledge (schema) associated with it. We show that items are better recalled when retrieved in the same context as the study context; we also show that the size of the effect is larger for items deemed context-relevant at encoding, highlighting the importance of integrating items into an active schema in generating this effect.

scholarly journals Approximate Value Iteration in the Reinforcement Learning Context. Application to Electrical Power System Control.

2005 ◽  
Vol 3 (1) ◽  
Author(s):  
Damien Ernst ◽  
Mevludin Glavic ◽  
Pierre Geurts ◽  
Louis Wehenkel
Keyword(s):  
Reinforcement Learning ◽  
Power System ◽  
Control Problem ◽  
Learning Algorithm ◽  
Electrical Power ◽  
Complex Case ◽  
Iteration Algorithm ◽  
Value Iteration ◽  
Learning Context ◽  
Power System Control

In this paper we explain how to design intelligent agents able to process the information acquired from interaction with a system to learn a good control policy and show how the methodology can be applied to control some devices aimed to damp electrical power oscillations. The control problem is formalized as a discrete-time optimal control problem and the information acquired from interaction with the system is a set of samples, where each sample is composed of four elements: a state, the action taken while being in this state, the instantaneous reward observed and the successor state of the system. To process this information we consider reinforcement learning algorithms that determine an approximation of the so-called Q-function by mimicking the behavior of the value iteration algorithm. Simulations are first carried on a benchmark power system modeled with two state variables. Then we present a more complex case study on a four-machine power system where the reinforcement learning algorithm controls a Thyristor Controlled Series Capacitor (TCSC) aimed to damp power system oscillations.

Christian Witness Indices: Some Positive Reinforcement

1998 ◽  
Vol 26 (2) ◽  
pp. 181-189
Author(s):  
Donald P. Brandt
Keyword(s):  
Positive Reinforcement ◽  
Statistical Tests ◽  
World Vision ◽  
The Poor ◽  
Christian Witness ◽  
Difficult Subject

We usually see the poor through lenses of physical poverty. Unfortunately, we have few measures to gauge the “lost” or spiritually poor. Spiritual indices developed by World Vision are described and then examined using four statistical tests. Results show that the indices are generally reliable. The measures, however, should be used in tandem as spiritual poverty is a very difficult subject to evaluate objectively.

scholarly journals A framework to foster analysis skill for self-directed activities in data-rich environment

2021 ◽  
Vol 16 (1) ◽  
Author(s):  
Yuanyuan Yang ◽  
Rwitajit Majumdar ◽  
Huiyong Li ◽  
Gökhan Akçapinar ◽  
Brendan Flanagan ◽  
...  
Keyword(s):  
Physical Activity ◽  
Undergraduate Students ◽  
Skill Acquisition ◽  
Learning System ◽  
Activity Levels ◽  
Learning Context ◽  
Context Analysis ◽  
Activity Data ◽  
Actual Usage ◽  
Goal System

AbstractSelf-direction skill is considered a vital skill for twenty-first-century learners in both the learning context and physical activity context. Analysis skill for self-directed activities requires the students to analyze their own activity data for understanding their status in that activity. It is an important phase that determines whether an appropriate plan can be set or not. This research presents a framework designed to foster students’ analysis skill in self-directed activities, which aims (1) to build a technology-enabled learning system allowing students to practice analyzing data from their own daily contexts, (2) to propose an approach to model student’s analysis skill acquisition level and process, and (3) to provide automated support and feedback for analysis skill development tasks. The analysis module based on the proposed framework was implemented in the GOAL system which synchronized data from learners’ physical and reading activities. A study was conducted with 51 undergraduate students to find reliable indicators for the model to then measure students’ analysis skills. By further analyzing students’ actual usage of the GOAL system, we found the actual activity levels and their preferences regarding analysis varied for the chosen contexts (learning and physical activity). The different context preference groups were almost equal, highlighting the utility of a system that integrates data from multiple contexts. Such a system can potentially respond to students’ individual preferences to execute and acquire self-direction skill.

Learning context-dependent word embeddings based on dependency parsing

2020 ◽  
Vol 19 (4) ◽  
pp. 334
Author(s):  
Ke Yan ◽  
Jie Chen ◽  
Wenhao Zhu ◽  
Xin Jin ◽  
Guannan Hu
Keyword(s):  
Word Embeddings ◽  
Dependency Parsing ◽  
Learning Context ◽  
Context Dependent

scholarly journals Neural circuits for learning context-dependent associations of stimuli

2018 ◽  
Vol 107 ◽  
pp. 48-60 ◽  
Author(s):  
Henghui Zhu ◽  
Ioannis Ch. Paschalidis ◽  
Michael E. Hasselmo
Keyword(s):  
Neural Circuits ◽  
Learning Context ◽  
Context Dependent

scholarly journals Vision-Based Robot Navigation through Combining Unsupervised Learning and Hierarchical Reinforcement Learning

Sensors ◽  
2019 ◽  
Vol 19 (7) ◽  
pp. 1576 ◽  
Author(s):  
Xiaomao Zhou ◽  
Tao Bai ◽  
Yanbin Gao ◽  
Yuntao Han
Keyword(s):  
Reinforcement Learning ◽  
Unsupervised Learning ◽  
Learning Algorithm ◽  
Spatial Scales ◽  
Learning Performance ◽  
Head Direction ◽  
Topological Map ◽  
Topological Maps ◽  
Hierarchical Reinforcement Learning ◽  
Continuous State

Extensive studies have shown that many animals’ capability of forming spatial representations for self-localization, path planning, and navigation relies on the functionalities of place and head-direction (HD) cells in the hippocampus. Although there are numerous hippocampal modeling approaches, only a few span the wide functionalities ranging from processing raw sensory signals to planning and action generation. This paper presents a vision-based navigation system that involves generating place and HD cells through learning from visual images, building topological maps based on learned cell representations and performing navigation using hierarchical reinforcement learning. First, place and HD cells are trained from sequences of visual stimuli in an unsupervised learning fashion. A modified Slow Feature Analysis (SFA) algorithm is proposed to learn different cell types in an intentional way by restricting their learning to separate phases of the spatial exploration. Then, to extract the encoded metric information from these unsupervised learning representations, a self-organized learning algorithm is adopted to learn over the emerged cell activities and to generate topological maps that reveal the topology of the environment and information about a robot’s head direction, respectively. This enables the robot to perform self-localization and orientation detection based on the generated maps. Finally, goal-directed navigation is performed using reinforcement learning in continuous state spaces which are represented by the population activities of place cells. In particular, considering that the topological map provides a natural hierarchical representation of the environment, hierarchical reinforcement learning (HRL) is used to exploit this hierarchy to accelerate learning. The HRL works on different spatial scales, where a high-level policy learns to select subgoals and a low-level policy learns over primitive actions to specialize on the selected subgoals. Experimental results demonstrate that our system is able to navigate a robot to the desired position effectively, and the HRL shows a much better learning performance than the standard RL in solving our navigation tasks.

Sign in / Sign up

Export Citation Format

Share Document