Explaining the effectiveness of fear extinction through latent-cause inference

Acquiring fear responses to predictors of aversive outcomes is crucial for survival. At the same time, it is important to be able to modify such associations when they are maladaptive, for instance in treating anxiety and trauma-related disorders. Standard extinction procedures can reduce fear temporarily, but with sufficient delay or with reminders of the aversive experience, fear often returns. The latent-cause inference framework explains the return of fear by presuming that animals learn a rich model of the environment, in which the standard extinction procedure triggers the inference of a new latent cause, preventing the extinguishing of the original aversive associations. This computational framework had previously inspired an alternative extinction paradigm -- gradual extinction -- which indeed was shown to be more effective in reducing fear. However, the original framework was not sufficient to explain the pattern of results seen in the experiments. Here, we propose a formal model to explain the effectiveness of gradual extinction, in contrast to the ineffectiveness of standard extinction and a gradual reverse control procedure. We demonstrate through quantitative simulation that our model can explain qualitative behavioral differences across different extinction procedures as seen in the empirical study. We verify the necessity of several key assumptions added to the latent-cause framework, which suggest potential general principles of animal learning and provide novel predictions for future experiments.

Fast Learning in Complex Domains

Bayesian formulations of learning imply that whenever the evidence for a correlation between events in an animal’s habitat is sufficient, the correlation is learned. This implies that regularities can be learnt rapidly, from small numbers of learning examples. This speed of learning gives maximum possible fitness, and no faster learning is possible. There is evidence in many domains that animals and people can learn at nearly Bayesian optimal speeds. These domains include associative conditioning, and the more complex domains of navigation and language. There are computational models of learning which learn at near-Bayesian speeds in complex domains, and which can scale well – to learn thousands of pieces of knowledge (i.e., relations and associations). These are not neural net models. They can be defined in computational terms, as algorithms and data structures at David Marr’s [1] Level Two. Their key data structures are composite feature structures, which are graphs of multiple linked nodes. This leads to the hypothesis that animal learning results not from deep neural nets (which typically require thousands of training exam-ples), but from neural implementations of the Level Two models of fast learning; and that neu-rons provide the facilities needed to implement those models at Marr’s Level Three. The required facilities include feature structures, dynamic binding, one-shot memory for many feature struc-tures, pattern-based associative retrieval, unification and generalization of feature structures. These may be supported by multiplexing of data and metadata in the same neural fibres.

A one-shot learning signal in monkey prefrontal cortex

Much animal learning is slow, with cumulative changes in behavior driven by reward prediction errors. When the abstract structure of a problem is known, however, both animals and formal learning models can rapidly attach new items to their roles within this structure, sometimes in a single trial. Frontal cortex is likely to play a key role in this process. To examine information seeking and use in a known problem structure, we trained monkeys in a novel explore/exploit task, requiring the animal first to test objects for their association with reward, then, once rewarded objects were found, to re-select them on further trials for further rewards. Many cells in the frontal cortex showed an explore/exploit preference, changing activity in a signal trial to align with one-shot learning in the monkeys’ behaviour. In contrast to this binary switch, these cells showed little evidence of continuous changes linked to expectancy or prediction error. Explore/exploit preferences were independent for two stages of the trial, object selection and receipt of feedback. Within an established task structure, frontal activity may control the separate operations of explore and exploit, switching in one trial between the two.Significance statementMuch animal learning is slow, with cumulative changes in behavior driven by reward prediction errors. When the abstract structure a problem is known, however, both animals and formal learning models can rapidly attach new items to their roles within this structure. To address transitions in neural activity during one-shot learning, we trained monkeys in an explore/exploit task using familiar objects and a highly familiar task structure. In contrast to continuous changes reflecting expectancy or prediction error, frontal neurons showed a binary, one-shot switch between explore and exploit. Within an established task structure, frontal activity may control the separate operations of exploring alternative objects to establish their current role, then exploiting this knowledge for further reward.

Examining the mechanisms underlying the acquisition of animal tool behaviour

Despite major advances in the study of animal tool behaviour, researchers continue to debate how exactly certain behaviours are acquired. While specific mechanisms, such as genetic predispositions or action copying, are sometimes suspected to play a major role in behavioural acquisition, controlled experiments are required to provide conclusive evidence. In this opinion piece, we refer to classic ethological methodologies to emphasize the need for studying the relative contributions of different factors to the emergence of specific tool behaviours. We describe a methodology, consisting of a carefully staged series of baseline and social-learning conditions, that enables us to tease apart the roles of different mechanisms in the development of behavioural repertoires. Experiments employing our proposed methodology will not only advance our understanding of animal learning and culture, but as a result, will also help inform hypotheses about human cognitive, cultural and technological evolution. More generally, our conceptual framework is suitable for guiding the detailed investigation of other seemingly complex animal behaviours.

Why Learning?

Once the genes delegate the responsibility of decision-making to the brain, the most important function of the brain is to develop successful decision-making strategies by incorporating new information about the animal’s environment. The complexity of this process increased during evolution, and in mammals, including humans, the brain utilizes multiple learning strategies to produce the most appropriate motor responses. After illustrating this using response and place learning, this chapter reviews the history of research on animal learning, including a potential conflict between different learning strategies. In particular, the author addresses the important role of classical conditioning and instrumental conditioning in learning.