Abstract
Background
We have found that measures of reinforcement learning (RL) performance correlate with negative symptoms severity in adult schizophrenia patients as well as in adolescents and young adults seeking psychiatric services. Most of these tasks assess reinforcement learning in stable environments, however. In unstable, or volatile environments, adaptive learning and decision making depend on the ability to use one’s own uncertainty to modulate attention to feedback. In stable RL environments, parameters called learning rates (signified by ⍺) capture the impact of prediction errors on changes in association strength with each subject having a single learning rate for a given kind of prediction error (positive and negative, e.g.). In volatile environments, learning rates might be more appropriately modeled as dynamic, modulated by uncertainty. Furthermore, uncertainty is known to guide what is called “the explore/exploit trade-off” – the threshold for choosing more informative options potentially at the expense of options with higher expected value.
Methods
We have examined the contribution of uncertainty processing to the emergence of negative symptoms in people along the schizophrenia spectrum, in several ways. First, in conjunction with fMRI, we administered 26 patients with schizophrenia (PSZ) and 23 healthy volunteers (HV) a 3-choice version of a probabilistic reversal learning task that required participants to resolve uncertainty and determine the new best option after sudden, sporadic contingency shifts. Second, we assessed the role of uncertainty in driving decision making under ambiguity, using two distinct tasks in cohorts of schizophrenia patients and healthy volunteers. Motivational symptoms were assessed in PSZ using the Scale for the Assessment of Negative Symptoms (SANS), from which we computed scores for Avolition/Role-Functioning, Anhedonia/Asociality, and an Avolition/Anhedonia/Asociality (AAA) factor.
Results
In the context of the 3-choice version of a probabilistic reversal learning task, we found that SZ patients with more severe anhedonia and avolition show a reduced ability to dynamically modulate learning rates in a volatile environment. A follow-up psychophysiological interaction analysis revealed decreased dmPFC-VS connectivity concurrent with learning rate modulation, most prominently in individuals with the most severe motivational deficits. Finally, in the context of decision making under ambiguity, we have found that SZ patients with more severe anhedonia and avolition, as measured by the SANS, show a reduced tendency to explore contingences in the service of reducing uncertainty. Furthermore, we found that mean negative symptom scores correlated negatively with change in information weight, a model-based measure of directed exploration.
Discussion
These results indicate that multiple potential mechanisms underlie motivational deficits in schizophrenia spectrum disorders, including processes related to the ability to flexibly modulate learning and decision making according to one’s level of certainty about contingencies in the environment. That is, beyond deficits in reward-seeking behavior, a reduced ability to use uncertainty to modulate learning rates and a reduced tendency to engage in information-seeking behavior may make substantial contributions to negative symptoms in people with psychotic illness and people at risk for psychotic illness. The ability to dynamically value actions in terms of both prospective reward and information is likely to contribute deficits in motivation across diagnoses.
Confirmation Bias in the Course of Instructed Reinforcement Learning in Schizophrenia-Spectrum Disorders