Differential motor learning via reward and punishment

2019 ◽  
Vol 73 (2) ◽  
pp. 249-259 ◽  
Author(s):  
Yanlong Song ◽  
Siyuan Lu ◽  
Ann L Smiley-Oyen
Keyword(s):  
Loss Aversion ◽  
Visual Target ◽  
Visuomotor Adaptation ◽  
Explicit Learning ◽  
Prediction Errors ◽  
Model Free ◽  
Explicit Processes ◽  
Multiple Processes ◽  
Sensory Prediction ◽  
Reward And Punishment

Visuomotor adaptation involves multiple processes such as explicit learning, implicit learning from sensory prediction errors, and model-free mechanisms like use-dependent plasticity. Recent findings show that reward and punishment differently affect visuomotor adaptation. This study examined whether punishment and reward had distinct effects on explicit learning. When participants practised adapting to a large, abrupt visual rotation during reaching for a virtual visual target, visual feedback of the cursor was not provided. Only performance-based scalar reward or punishment feedback (money gained or lost) was used, thereby emphasising explicit processes during adaptation. The results revealed that punishment, compared with reward, induced faster adaptation and greater variability of reaching in the initial phase of adaptation. We interpret these findings as reflecting enhanced explicit learning, likely due to loss aversion.

scholarly journals Can patients with cerebellar disease switch learning mechanisms to reduce their adaptation deficits?

Brain ◽  
2019 ◽  
Vol 142 (3) ◽  
pp. 662-673 ◽  
Author(s):  
Aaron L Wong ◽  
Cherie L Marvel ◽  
Jordan A Taylor ◽  
John W Krakauer
Keyword(s):  
Prediction Error ◽  
Poor Performance ◽  
Visuomotor Adaptation ◽  
Cerebellar Degeneration ◽  
Prediction Errors ◽  
Cerebellar Disease ◽  
Learning Mechanisms ◽  
Visuomotor Rotation ◽  
Age Related ◽  
Sensory Prediction

Abstract Systematic perturbations in motor adaptation tasks are primarily countered by learning from sensory-prediction errors, with secondary contributions from other learning processes. Despite the availability of these additional processes, particularly the use of explicit re-aiming to counteract observed target errors, patients with cerebellar degeneration are surprisingly unable to compensate for their sensory-prediction error deficits by spontaneously switching to another learning mechanism. We hypothesized that if the nature of the task was changed—by allowing vision of the hand, which eliminates sensory-prediction errors—patients could be induced to preferentially adopt aiming strategies to solve visuomotor rotations. To test this, we first developed a novel visuomotor rotation paradigm that provides participants with vision of their hand in addition to the cursor, effectively setting the sensory-prediction error signal to zero. We demonstrated in younger healthy control subjects that this promotes a switch to strategic re-aiming based on target errors. We then showed that with vision of the hand, patients with cerebellar degeneration could also switch to an aiming strategy in response to visuomotor rotations, performing similarly to age-matched participants (older controls). Moreover, patients could retrieve their learned aiming solution after vision of the hand was removed (although they could not improve beyond what they retrieved), and retain it for at least 1 year. Both patients and older controls, however, exhibited impaired overall adaptation performance compared to younger healthy controls (age 18–33 years), likely due to age-related reductions in spatial and working memory. Patients also failed to generalize, i.e. they were unable to adopt analogous aiming strategies in response to novel rotations. Hence, there appears to be an inescapable obligatory dependence on sensory-prediction error-based learning—even when this system is impaired in patients with cerebellar disease. The persistence of sensory-prediction error-based learning effectively suppresses a switch to target error-based learning, which perhaps explains the unexpectedly poor performance by patients with cerebellar degeneration in visuomotor adaptation tasks.

scholarly journals Reduced transfer of visuomotor adaptation is associated with aberrant sense of agency in Schizophrenia

2018 ◽  
Author(s):  
Sonia Bansal ◽  
Karthik G Murthy ◽  
Justin Fitzgerald ◽  
Barbara L. Schwartz ◽  
Wilsaan M. Joiner
Keyword(s):  
Negative Symptoms ◽  
Motor Behavior ◽  
Visuomotor Adaptation ◽  
Sense Of Agency ◽  
Common Source ◽  
Prediction Errors ◽  
Movement Trajectories ◽  
Action Awareness ◽  
Before And After ◽  
Sensory Prediction

ABSTRACTOne deficit associated with schizophrenia (SZ) is the reduced ability to distinguish sensations resulting from self-caused actions from those due to external sources. This reduced sense of agency (SoA, awareness of ownership over self-generated actions) is hypothesized to result from a diminished utilization of internal monitoring signals of self-movement (i.e., efferent copy) which subsequently impairs forming and utilizing sensory prediction errors (differences between the predicted and actual sensory consequences resulting from movement). Here, we investigated the connections between clinical SZ symptoms and motor adaptation, a process that uses sensory prediction errors to update motor output. Schizophrenia patients (SZP, N=30) and non-psychiatric healthy control subjects (HC, N=31) adapted to altered movement visual feedback, and then applied the motor recalibration to untested contexts (i.e., the spatial generalization to untrained targets). Although adaptation was similar for SZP and controls, the extent of generalization was significantly less for SZP; movement trajectories made by patients to the furthest untrained target (135°) before and after adaptation were largely indistinguishable. Interestingly, deficits in the generalization were correlated to positive symptoms of psychosis (e.g., hallucinations), but not negative symptoms. Generalization was also associated with subjective measures of SoA across both SZP and HC, emphasizing the major role action awareness plays in motor behavior, and suggesting that tendencies to misattribute agency, even in HC, manifest in abnormal motor performance. We discuss the possible link of these findings to cerebellar circuit abnormalities that may be a common source for deficits in the utilization of sensory prediction errors and aberrant SoA.

scholarly journals Flexible explicit but rigid implicit learning in a visuomotor adaptation task

2015 ◽  
Vol 113 (10) ◽  
pp. 3836-3849 ◽  
Author(s):  
Krista M. Bond ◽  
Jordan A. Taylor
Keyword(s):  
Implicit Learning ◽  
Computational Models ◽  
Visuomotor Adaptation ◽  
Task Demands ◽  
Similar Degree ◽  
Explicit Learning ◽  
Prediction Errors ◽  
Visuomotor Rotation ◽  
Indirect Measures ◽  
Task Conditions

There is mounting evidence for the idea that performance in a visuomotor rotation task can be supported by both implicit and explicit forms of learning. The implicit component of learning has been well characterized in previous experiments and is thought to arise from the adaptation of an internal model driven by sensorimotor prediction errors. However, the role of explicit learning is less clear, and previous investigations aimed at characterizing the explicit component have relied on indirect measures such as dual-task manipulations, posttests, and descriptive computational models. To address this problem, we developed a new method for directly assaying explicit learning by having participants verbally report their intended aiming direction on each trial. While our previous research employing this method has demonstrated the possibility of measuring explicit learning over the course of training, it was only tested over a limited scope of manipulations common to visuomotor rotation tasks. In the present study, we sought to better characterize explicit and implicit learning over a wider range of task conditions. We tested how explicit and implicit learning change as a function of the specific visual landmarks used to probe explicit learning, the number of training targets, and the size of the rotation. We found that explicit learning was remarkably flexible, responding appropriately to task demands. In contrast, implicit learning was strikingly rigid, with each task condition producing a similar degree of implicit learning. These results suggest that explicit learning is a fundamental component of motor learning and has been overlooked or conflated in previous visuomotor tasks.

scholarly journals Can patients with cerebellar disease switch learning mechanisms to reduce their adaptation deficits?

2018 ◽  
Author(s):  
Aaron L. Wong ◽  
Cherie L. Marvel ◽  
Jordan A. Taylor ◽  
John W. Krakauer
Keyword(s):  
Spinocerebellar Ataxia ◽  
Prediction Error ◽  
Visuomotor Adaptation ◽  
Cerebellar Degeneration ◽  
Prediction Errors ◽  
Healthy Controls ◽  
Learning Mechanisms ◽  
Visuomotor Rotation ◽  
Age Related ◽  
Sensory Prediction

ABSTRACTSystematic perturbations in motor adaptation tasks are primarily countered by learning from sensory-prediction errors, with secondary contributions from other learning processes. Despite the availability of these additional processes, particularly the use of explicit re-aiming to counteract observed target errors, patients with cerebellar degeneration are surprisingly unable to compensate for their sensory-prediction-error deficits by spontaneously switching to another learning mechanism. We hypothesized that if the nature of the task was changed – by allowing vision of the hand, which eliminates sensory-prediction errors – patients could be induced to preferentially adopt aiming strategies to solve visuomotor rotations. To test this, we first developed a novel visuomotor rotation paradigm that provides participants with vision of their hand in addition to the cursor, effectively setting the sensory-prediction-error signal to zero. We demonstrated in younger healthy controls that this promotes a switch to strategic re-aiming based on target errors. We then showed that with vision of the hand, patients with spinocerebellar ataxia could also switch to an aiming strategy in response to visuomotor rotations, performing similarly to age-matched participants (older controls). Moreover, patients could retrieve their learned aiming solution after vision of the hand was removed, and retain it for at least one year. Both patients and older controls, however, exhibited impaired overall adaptation performance compared to younger healthy controls (age, 18-33), likely due to age-related reductions in spatial and working memory. Moreover, patients failed to generalize, i.e., they were unable to adopt analogous aiming strategies in response to novel rotations, nor could they further improve their performance without vision of the hand. Hence, there appears to be an inescapable obligatory dependence on sensory-prediction-error-based learning – even when this system is impaired in patients with cerebellar degeneration. The persistence of sensory-prediction-error-based learning effectively suppresses a switch to target-error-based learning, which perhaps explains the unexpectedly poor performance by patients with spinocerebellar ataxia in visuomotor adaptation tasks.

scholarly journals Rethinking dopamine as generalized prediction error

2017 ◽  
Author(s):  
Matthew P.H. Gardner ◽  
Geoffrey Schoenbaum ◽  
Samuel J. Gershman
Keyword(s):  
Reinforcement Learning ◽  
Prediction Error ◽  
Dopamine Neurons ◽  
Prediction Errors ◽  
Model Free ◽  
Reward Prediction ◽  
Midbrain Dopamine ◽  
Sensory Prediction ◽  
Lines Of Evidence ◽  
Midbrain Dopamine Neurons

AbstractMidbrain dopamine neurons are commonly thought to report a reward prediction error, as hypothesized by reinforcement learning theory. While this theory has been highly successful, several lines of evidence suggest that dopamine activity also encodes sensory prediction errors unrelated to reward. Here we develop a new theory of dopamine function that embraces a broader conceptualization of prediction errors. By signaling errors in both sensory and reward predictions, dopamine supports a form of reinforcement learning that lies between model-based and model-free algorithms. This account remains consistent with current canon regarding the correspondence between dopamine transients and reward prediction errors, while also accounting for new data suggesting a role for these signals in phenomena such as sensory preconditioning and identity unblocking, which ostensibly draw upon knowledge beyond reward predictions.

scholarly journals Practice induces a qualitative change in the memory representation for visuomotor learning

2019 ◽  
Vol 122 (3) ◽  
pp. 1050-1059 ◽  
Author(s):  
David M. Huberdeau ◽  
John W. Krakauer ◽  
Adrian M. Haith
Keyword(s):  
Qualitative Change ◽  
Visuomotor Adaptation ◽  
Skill Learning ◽  
Memory Representation ◽  
Prediction Errors ◽  
Visuomotor Rotation ◽  
Repeated Practice ◽  
Target Locations ◽  
Sensory Prediction ◽  
Automatic Action

Adaptation of our movements to changes in the environment is known to be supported by multiple learning processes that operate in parallel. One is an implicit recalibration process driven by sensory-prediction errors; the other process counters the perturbation through more deliberate compensation. Prior experience is known to enable adaptation to occur more rapidly, a phenomenon known as “savings,” but exactly how experience alters each underlying learning process remains unclear. We measured the relative contributions of implicit recalibration and deliberate compensation to savings across 2 days of practice adapting to a visuomotor rotation. The rate of implicit recalibration showed no improvement with repeated practice. Instead, practice led to deliberate compensation being expressed even when preparation time was very limited. This qualitative change is consistent with the proposal that practice establishes a cached association linking target locations to appropriate motor output, facilitating a transition from deliberate to automatic action selection. NEW & NOTEWORTHY Recent research has shown that savings for visuomotor adaptation is attributable to retrieval of intentional, strategic compensation. This does not seem consistent with the implicit nature of memory for motor skills and calls into question the validity of visuomotor adaptation of reaching movements as a model for motor skill learning. Our findings suggest a solution: that additional practice adapting to a visuomotor perturbation leads to the caching of the initially explicit strategy for countering it.

scholarly journals Mini-review: The Role of the Cerebellum in Visuomotor Adaptation

2021 ◽  
Author(s):  
Elinor Tzvi ◽  
Sebastian Loens ◽  
Opher Donchin
Keyword(s):  
Premotor Cortex ◽  
Motor Program ◽  
Visuomotor Adaptation ◽  
Prediction Errors ◽  
Visuomotor Rotation ◽  
Non Invasive ◽  
Behavioral Studies ◽  
Sensory Prediction ◽  
The Brain

AbstractThe incredible capability of the brain to quickly alter performance in response to ever-changing environment is rooted in the process of adaptation. The core aspect of adaptation is to fit an existing motor program to altered conditions. Adaptation to a visuomotor rotation or an external force has been well established as tools to study the mechanisms underlying sensorimotor adaptation. In this mini-review, we summarize recent findings from the field of visuomotor adaptation. We focus on the idea that the cerebellum plays a central role in the process of visuomotor adaptation and that interactions with cortical structures, in particular, the premotor cortex and the parietal cortex, may be crucial for this process. To this end, we cover a range of methodologies used in the literature that link cerebellar functions and visuomotor adaptation; behavioral studies in cerebellar lesion patients, neuroimaging and non-invasive stimulation approaches. The mini-review is organized as follows: first, we provide evidence that sensory prediction errors (SPE) in visuomotor adaptation rely on the cerebellum based on behavioral studies in cerebellar patients. Second, we summarize structural and functional imaging studies that provide insight into spatial localization as well as visuomotor adaptation dynamics in the cerebellum. Third, we discuss premotor — cerebellar interactions and how these may underlie visuomotor adaptation. And finally, we provide evidence from transcranial direct current and magnetic stimulation studies that link cerebellar activity, beyond correlational relationships, to visuomotor adaptation .

scholarly journals Rethinking dopamine as generalized prediction error

2018 ◽  
Vol 285 (1891) ◽  
pp. 20181645 ◽  
Author(s):  
Matthew P. H. Gardner ◽  
Geoffrey Schoenbaum ◽  
Samuel J. Gershman
Keyword(s):  
Reinforcement Learning ◽  
Prediction Error ◽  
Dopamine Neurons ◽  
Prediction Errors ◽  
Model Free ◽  
Reward Prediction ◽  
Midbrain Dopamine ◽  
Sensory Prediction ◽  
Lines Of Evidence ◽  
Midbrain Dopamine Neurons

Midbrain dopamine neurons are commonly thought to report a reward prediction error (RPE), as hypothesized by reinforcement learning (RL) theory. While this theory has been highly successful, several lines of evidence suggest that dopamine activity also encodes sensory prediction errors unrelated to reward. Here, we develop a new theory of dopamine function that embraces a broader conceptualization of prediction errors. By signalling errors in both sensory and reward predictions, dopamine supports a form of RL that lies between model-based and model-free algorithms. This account remains consistent with current canon regarding the correspondence between dopamine transients and RPEs, while also accounting for new data suggesting a role for these signals in phenomena such as sensory preconditioning and identity unblocking, which ostensibly draw upon knowledge beyond reward predictions.

scholarly journals A Synthesis of the Many Errors and Learning Processes of Visuomotor Adaptation

2021 ◽  
Author(s):  
J. Ryan Morehead ◽  
Jean-Jacques Orban de Xivry
Keyword(s):  
Task Performance ◽  
Visuomotor Adaptation ◽  
Learning Processes ◽  
Skill Learning ◽  
Sensorimotor Adaptation ◽  
Prediction Errors ◽  
Control Measurement ◽  
Performance Error ◽  
Nuanced Understanding ◽  
Sensory Prediction

Visuomotor adaptation has one of the oldest experimental histories in psychology and neuroscience, yet its precise nature has always been a topic of debate. Here we offer a survey and synthesis of recent work on visuomotor adaptation that we hope will prove illuminating for this ongoing dialogue. We discuss three types of error signals that drive learning in adaptation tasks: task performance error, sensory prediction-error, and a binary target hitting error. Each of these errors has been shown to drive distinct learning processes. Namely, both target hitting errors and putative sensory prediction-errors drive an implicit change in visuomotor maps, while task performance error drives learning of explicit strategy use and non-motor decision-making. Each of these learning processes contributes to the overall learning that takes place in visuomotor adaptation tasks, and although the learning processes and error signals are independent, they interact in a complex manner. We outline many task contexts where the operation of these processes is counter-intuitive and offer general guidelines for their control, measurement and interpretation. We believe this new framework unifies several disparate threads of research in sensorimotor adaptation that often seem in conflict. We conclude by explaining how this more nuanced understanding of errors and learning processes could lend itself to the analysis of other types of sensorimotor adaptation, of motor skill learning, of the neural processing underlying sensorimotor adaptation in humans, of animal models and of brain computer interfaces.

Sign in / Sign up

Export Citation Format

Share Document