Goal-directed control on interval schedules does not depend on the action-outcome correlation

When an organism’s action is based on an anticipation of its consequences, that action is said to be goal-directed. It has long been thought that goal-directed control is made possible by experiencing a strong correlation between response rates and reward rates (Dickinson, 1985). To test this idea, we designed a set of experiments to determine whether the response rate-reward rate correlation is a reliable predictor of goal-directed control on interval schedules. In Experiment 1, rats were trained on random interval (RI) schedules in which the response rate-reward rate correlation was manipulated across groups. In tests of reward devaluation, rats behaved in a goal-directed manner regardless of the experienced correlation. In Experiment 2, rats once again experienced either a strong or weak correlation, but on RI schedules with lower overall reward densities. This time, behavior appeared habitual regardless of the experienced correlation. Experiment 3 confirmed that the density of the RI schedule influences goal-directed control, and also revealed that extensive training on these schedules resulted in goal-directed action. Finally, in Experiment 4 goal-directed responding was greater and emerged sooner on fixed than random interval schedules, but, again, was manifest after extensive training on the RI schedule. Taken together, our data suggest that goal-directed and habitual control are not determined by the correlation between response rates and reward rates. We discuss the importance of temporal uncertainty, action-outcome contiguity, and reinforcement probability in goal-directed control.

Hippocampal-amygdala interactions mediate uncertainty-dependent resistance to extinction following fear conditioning

The partial reinforcement extinction effect (PREE) is a paradoxical learning phenomenon in which omission of reinforcement during acquisition results in more persistent conditioned responding in extinction. Here, we report a significant PREE with an inverted-U, entropy-like distribution against reinforcement probability following tone foot shock fear conditioning in rats, which was associated with increased neural activity in hippocampus and amygdala as indexed by p-ERK and c-fos immunolabelling. In vivo electrophysiological recordings of local field potentials (LFPs) showed that 50% reinforcement was associated with increases in the frequency and power of tone-evoked theta oscillations in both the subiculum region of hippocampus and in basolateral amygdala (BLA) during both acquisition (Day 1) and extinction (Day 2) sessions. Tone-evoked LFPs in 50% reinforced animals also showed increases in coherence and bidirectional Granger Causality between hippocampus and amygdala. The results support a Bayesian interpretation of the PREE, in which the phenomenon is driven by increases in the entropy or uncertainty of stimulus contingencies, and indicate a crucial role for hippocampus in mediating this uncertainty-dependent effect.

Human instrumental performance in ratio and interval contingencies: A challenge for associative theory

Associative learning theories regard the probability of reinforcement as the critical factor determining responding. However, the role of this factor in instrumental conditioning is not completely clear. In fact, free-operant experiments show that participants respond at a higher rate on variable ratio than on variable interval schedules even though the reinforcement probability is matched between the schedules. This difference has been attributed to the differential reinforcement of long inter-response times (IRTs) by interval schedules, which acts to slow responding. In the present study, we used a novel experimental design to investigate human responding under random ratio (RR) and regulated probability interval (RPI) schedules, a type of interval schedule that sets a reinforcement probability independently of the IRT duration. Participants responded on each type of schedule before a final choice test in which they distributed responding between two schedules similar to those experienced during training. Although response rates did not differ during training, the participants responded at a lower rate on the RPI schedule than on the matched RR schedule during the choice test. This preference cannot be attributed to a higher probability of reinforcement for long IRTs and questions the idea that similar associative processes underlie classical and instrumental conditioning.

Age and sex differences in behavioral flexibility, sensitivity to reward value, and risky decision-making

Compared to adults, adolescent behavior is often characterized by reduced behavioral flexibility, increased sensitivity to reward, and increased likelihood to take risks. These traits, which have been hypothesized to confer heightened vulnerability to psychopathologies such as substance use disorders (SUDs), have been the focus of studies in laboratory animal models that seek to understand their neural underpinnings. However, rodent studies to date have typically used only males and have adopted standard methodological practices (e.g., weight loss inducing food restriction) that are likely to have a disparate impact on adolescents compared to adults. Here, we used adolescent and adult Sprague-Dawley rats of both sexes to study instrumental behavior tasks that assess behavioral flexibility (strategy shifting and reversal learning; Exp. 1), sensitivity to reward value (outcome devaluation; Exp. 2), and risky decision making (probability discounting; Exp. 3). In Exp. 1, we found that adolescents were faster to acquire reversal learning than adults but there were no differences in strategy shifting. In Exps. 2 and 3, adolescents and adults were equally sensitive to changes in reward value and exhibited similar reductions in preference for a large reward when reinforcement probability was decreased. However, adolescents responded more efficiently and earned reinforcers at a higher rate than their same-sex, adult counterparts. Together, these findings provide only limited support for the existence of an “adolescent-typical” phenotype in Sprague-Dawley rats and instead suggest that age differences in the expression of these behaviors may depend on conditions such as pubertal status and motivational state.