Agency rescues competition for credit assignment among predictive cues from adverse learning conditions

A fundamental assumption of learning theories is that the credit assigned to predictive cues is not simply determined by their probability of reinforcement, but by their ability to compete with other cues present during learning. This assumption has guided behavioral and neural science research for decades, and tremendous empirical and theoretical advances have been made identifying the mechanisms of cue competition. However, when learning conditions are not optimal (e.g., when training is massed), cue competition is attenuated. This failure of the learning system exposes the individual’s vulnerability to form spurious associations in the real world. Here, we uncover that cue competition in rats can be rescued when conditions are suboptimal provided that the individual has agency over the learning experience. Our findings reveal a new effect of agency over learning on credit assignment among predictive cues, and open new avenues of investigation into the underlying mechanisms.

The effects of spatial stability and cue type on spatial learning: Implications for theories of parallel memory systems

Some theories of spatial learning predict that associative rules apply under only limited circumstances. For example, learning based on a boundary has been claimed to be immune to cue competition effects because boundary information is the basis for the formation of a cognitive map, whilst landmark learning does not involve cognitive mapping. This is referred to as the cue type hypothesis. However, it has also been claimed that cue stability is a prerequisite for the formation of a cognitive map, meaning that whichever cue type was perceived as stable would enter a cognitive map and thus be immune to cue competition, while unstable cues will be subject to cue competition, regardless of cue type. In experiments 1 and 2 we manipulated the stability of boundary and landmark cues when learning the location of two hidden goals. One goal location was constant with respect to the boundary, and the other constant with respect to the landmark cues. For both cue types, the presence of distal orientation cues provided directional information. For half the participants the landmark cues were unstable relative to the boundary and orientation cues, whereas for the remainder of the participants the boundary was unstable relative to landmarks and orientation cues. In a second stage of training, all cues remained stable so that both goal locations could be learned with respect to both landmark and boundary information. According to the cue type hypothesis, boundary information should block learning about landmarks regardless of cue stability. According to the cue stability hypothesis, however, landmarks should block learning about the boundary when the landmarks appear stable relative to the boundary. Regardless of cue type or stability the results showed reciprocal blocking, contrary to both formulations of incidental cognitive mapping. Experiment 3 established that the results of Experiments 1 and 2 could not be explained in terms of difficulty in learning certain locations with respect to different cue types. In a final experiment, following training in which both landmarks and boundary cues signalled two goal locations, a new goal location was established with respect to the landmark cues, before testing with the boundary, which had never been used to define the new goal location. The results of this novel test of the interaction between boundary and landmark cues indicated that new learning with respect to the landmark had a profound effect on navigation with respect to the boundary, counter to the predictions of incidental cognitive mapping of boundaries.

Representing uncertainty in the Rescorla-Wagner model: Blocking, the redundancy effect, and outcome base rate

It is generally assumed that the Rescorla and Wagner (1972) model adequately accommodates the full results of simple cue competition experiments in humans (e.g. Dickinson et al., 1984), while the Bush and Mosteller (1951) model cannot. We present simulations that demonstrate this assumption is wrong in at least some circumstances. The Rescorla-Wagner model, as usually applied, fits the full results of a simple forward cue-competition experiment no better than the Bush-Mosteller model. Additionally, we present a novel finding, where letting the associative strength of all cues start at an intermediate value (rather than zero), allows this modified model to provide a better account of the experimental data than the (equivalently modified) Bush-Mosteller model. This modification also allows the Rescorla-Wagner model to account for a redundancy effect experiment (Uengoer et al., 2013); something that the unmodified model is not able to do. Furthermore, the modified Rescorla-Wagner model can accommodate the effect of varying the proportion of trials on which the outcome occurs (i.e. the base rate) on the redundancy effect (Jones et al., 2019). Interestingly, the initial associative strength of cues varies in line with the outcome base rate. We propose that this modification provides a simple way of mathematically representing uncertainty about the causal status of novel cues within the confines of the Rescorla-Wagner model. The theoretical implications of this modification are discussed. We also briefly introduce free and open resources to support formal modelling in associative learning. Keywords: associative learning, prediction error, uncertainty, modelling, blocking, redundancy effect, open science.

Agency rescues competition for credit assignment among predictive cues from adverse learning conditions

A fundamental assumption of learning theories is that the credit assigned to predictive cues is not simply determined by their probability of reinforcement, but by their ability to compete with other cues present during learning. This assumption has guided behavioral and neural science research for decades, and tremendous empirical and theoretical advances have been made identifying the mechanisms of cue competition. However, when learning conditions are not optimal (e.g., when training is massed), credit assignment is no longer competitive. This is a catastrophic failure of the learning system that exposes the individual’s vulnerability to form spurious associations in the real world. Here, we uncover that cue competition can be rescued when conditions are suboptimal provided that the individual has agency over the learning experience. Our findings reveal a new connection between agency over learning and credit assignment to cues, and open new avenues of investigation into the underlying mechanisms.

Representing uncertainty in the Rescorla-Wagner model: blocking, the redundancy effect, and outcome base rate

It is generally assumed that the Rescorla and Wagner (1972) model adequately accommodates the full results of simple cue competition experiments in humans (e.g. Dickinson et al., 1984), while the Bush and Mosteller (1951) model cannot. We present simulations that demonstrate this assumption is wrong in at least some circumstances. The Rescorla-Wagner model, as usually applied, fits the full results of a simple forward cue-competition experiment no better than the Bush-Mosteller model. Additionally, we present a novel finding, where letting the associative strength of all cues start at an intermediate value (rather than zero), allows this modified model to provide a better account of the experimental data than the (equivalently modified) Bush-Mosteller model. This modification also allows the Rescorla-Wagner model to account for a redundancy effect experiment (Uengoer et al., 2013); something that the unmodified model is not able to do. Furthermore, the modified Rescorla-Wagner model can accommodate the effect of varying the proportion of trials on which the outcome occurs (i.e. the base rate) on the redundancy effect (Jones et al., 2019). Interestingly, the initial associative strength of cues varies in line with the outcome base rate. We propose that this modification provides a simple way of mathematically representing uncertainty about the causal status of novel cues within the confines of the Rescorla-Wagner model. The theoretical implications of this modification are discussed. We also briefly introduce free and open resources to support formal modelling in associative learning.

Approaching text genre

In the present study, we sought to clarify how differences in contextualized experience influence the performance of participants engaged in genre decision-making. Using a simple learning algorithm, we ran a series of computational simulations to model the effects that context and cue competition have on the way readers of different backgrounds make genre decisions. Next, we used the results of those simulations as predictions for our behavioural genre decision experiment. Differences in test performance were strongly influenced by the factors that have long been known to influence learning: Cue competition and its embedding in a specific context jointly modulate what gets learned and that inevitably affects later performance. We discuss our findings in the context of learning and literary genres.

Can We Set Aside Previous Experience in a Familiar Causal Scenario?

Causal and predictive learning research often employs intuitive and familiar hypothetical scenarios to facilitate learning novel relationships. The allergist task, in which participants are asked to diagnose the allergies of a fictitious patient, is one example of this. In such studies, it is common practice to ask participants to ignore their existing knowledge of the scenario and make judgments based only on the relationships presented within the experiment. Causal judgments appear to be sensitive to instructions that modify assumptions about the scenario. However, the extent to which prior knowledge continues to affect competition for associative learning, even after participants are instructed to disregard it, is unknown. To answer this, we created a cue competition design that capitalized on prevailing beliefs about the allergenic properties of various foods. High and low allergenic foods were paired with foods moderately associated with allergy to create two compounds; high + moderate and low + moderate. We expected high allergenic foods to produce greater competition for associative memory than low allergenic foods. High allergenic foods may affect learning either because they generate a strong memory of allergy or because they are more salient in the context of the task. We therefore also manipulated the consistency of the high allergenic cue-outcome relationship with prior beliefs about the nature of the allergies. A high allergenic food that is paired with an inconsistent allergenic outcome should generate more prediction error and thus more competition for learning, than one that is consistent with prior beliefs. Participants were instructed to either use or ignore their knowledge of food allergies to complete the task. We found that while participants were able to set aside their prior knowledge when making causal judgments about the foods in question, associative memory was weaker for the cues paired with highly allergenic foods than cues paired with low allergenic foods regardless of instructions. The consistency manipulation had little effect on this result, suggesting that the effects in associative memory are most likely driven by selective attention to highly allergenic cues. This has implications for theories of causal learning as well as the way causal learning tasks are designed.