Reward timing matters in motor learning

Reward can improve motor learning and the consolidation of motor memories. Identifying the features of reward feedback that are critical for motor learning is a necessary step for successful integration into rehabilitation programs. One central feature of reward feedback that may affect motor learning is its timing, that is, the delay after which reward is delivered following movement execution. In fact, research on associative learning has shown that short and long reward delays (e.g., 1 and 6 s following action execution) activate preferentially the striatum and the hippocampus, respectively, which both contribute with varying degrees to motor learning. Given the distinct functional role of these two areas, we hypothesized that reward timing could modulate how people learn and consolidate a new motor skill. In sixty healthy participants, we found that delaying reward delivery by a few seconds influenced motor learning dynamics. Indeed, training with a short reward delay (i.e., 1 s) induced slow, yet continuous gains in performance, while a long reward delay (i.e., 6 s) led to initially high learning rates that were followed by an early plateau in the learning curve and a lower endpoint performance. Moreover, participants who successfully learned the skill with a short reward delay displayed overnight consolidation, while those who trained with a long reward delay exhibited an impairment in the consolidation of the motor memory. Overall, our data show that reward timing affects motor learning, potentially by modulating the engagement of different learning processes, a finding that could be exploited in future rehabilitation programs.

Comparison of the Effectiveness of Motivational Manipulation and Neurofeedback Methods on Sensitivity to Reward, Delay Discounting and Impulsivity in Children with Attention Deficit / Hyperactivity Disorder

Purpose; The aim of this study was to comparison of the effectiveness of two methods of motivational manipulation and neurofeedback on sensitivity to reward, delay discounting and impulsivity in children with attention deficit / hyperactivity disorder. Methods; The present study was an experimental study with pre-test, post-test, follow-up and control group. The study population consisted of children aged 7 to 12 years with ADHD. Available sampling method was used for sampling in this study. Sample size was selected based on the purpose and method of research (in experimental research, the sample size of at least 30 people in each group is recommended) 90 people. In this study, neurofeedback and motivational manipulation were performed on the intervention group as an intervention, each session was performed for 12 sessions and each session was performed for 45 minutes. Balloon risk test and delay discounting test were used to collect data in pre-test, post-test and follow-up stages and the data were analyzed using MANCOVA statistical method in SPSS-23. Results; By controlling the effects of pretest on posttest, the difference between the groups in the posttest was statistically significant between the variables of impulsivity and delay at the level of P <0.01 and between the variables of reward sensitivity at the level of P <0.05. The results also showed that neurofeedback method (M = 4.66) had a greater effect on reward processing than motivational manipulation method (M = 2.31) compared to the control group, which was significant at the level of P <0.01. But the difference between the mean of motivational manipulation (M = 2.31) in comparison with the control group was not significant. Conclusion; Voluntary activation of dopaminergic regions of the brain by neurofeedback and motivational manipulation leads to endogenous dopamine control in these structures, leading to successful regulation or inhibitory control and reduced cravings, which reduces impulsivity, delay discounting, and sensitivity to reward.

Impulsivity as Bayesian inference under dopaminergic control

Bayesian models successfully account for several of dopamine (DA)’s effects on contextual calibration in interval timing and reward estimation. In these models, DA controls the precision of stimulus encoding, which is weighed against contextual information when making decisions. When DA levels are high, the animal relies more heavily on the (highly precise) stimulus encoding, whereas when DA levels are low, the context affects decisions more strongly. Here, we extend this idea to intertemporal choice tasks, in which agents must choose between small rewards delivered soon and large rewards delivered later. Beginning with the principle that animals will seek to maximize their reward rates, we show that the Bayesian model predicts a number of curious empirical findings. First, the model predicts that higher DA levels should normally promote selection of the larger/later option, which is often taken to imply that DA decreases ‘impulsivity.’ However, if the temporal precision is sufficiently decreased, higher DA levels should have the opposite effect—promoting selection of the smaller/sooner option (more impulsivity). Second, in both cases, high enough levels of DA can result in preference reversals. Third, selectively decreasing the temporal precision, without manipulating DA, should promote selection of the larger/later option. Fourth, when a different post-reward delay is associated with each option, animals will not learn the option-delay contingencies, but this learning can be salvaged when the post-reward delays are made more salient. Finally, the Bayesian model predicts a correlation between behavioral phenotypes: Animals that are better timers will also appear less impulsive.Significance StatementDoes dopamine make animals more or less impulsive? Though impulsivity features prominently in several dopamine-related conditions, how dopamine actually influences impulsivity has remained unclear. In intertemporal choice tasks (ITCs), wherein animals must choose between small rewards delivered soon and large rewards delivered later, administering dopamine makes animals more willing to wait for larger/later rewards in some conditions (consistent with lower impulsivity), but less willing in others. We hypothesize that dopamine does not necessarily influence impulsivity at all, but rather gates the influence of contextual information during decision making. We show that this account explains an array of curious findings in ITCs, including the seemingly conflicting results above. Our work encourages a reexamination of ITCs as a method for assessing impulsivity.