Optimism and pessimism in optimised replay

The replay of task-relevant trajectories is known to contribute to memory consolidation and improved task performance. A wide variety of experimental data show that the content of replayed sequences is highly specific and can be modulated by reward as well as other prominent task variables. However, the rules governing the choice of sequences to be replayed still remain poorly understood. One recent theoretical suggestion is that the prioritization of replay experiences in decision-making problems is based on their effect on the choice of action. We show that this implies that subjects should replay sub-optimal actions that they dysfunctionally choose rather than optimal ones, when, by being forgetful, they experience large amounts of uncertainty in their internal models of the world. We use this to account for recent experimental data demonstrating exactly pessimal replay, fitting model parameters to the individual subjects’ choices.

Motor imagery helps updating internal models during microgravity exposure

Skilled movements result from a mixture of feedforward and feedback mechanisms conceptualized by internal models. These mechanisms subserve both motor execution and motor imagery. Current research suggests that imagery allows updating feedforward mechanisms, leading to better performance in familiar contexts. Does this still hold in radically new contexts? Here, we test this ability by asking participants to imagine swinging arm movements around shoulder in normal gravity condition and in microgravity in which studies showed that movements slow down. We timed several cycles of actual and imagined arm pendular movements in three groups of subjects during parabolic flight campaign. The first, control, group remained on the ground. The second group was exposed to microgravity but did not imagine movements inflight. The third group was exposed to microgravity and imagined movements inflight. All groups performed and imagined the movements before and after the flight. We predicted that a mere exposure to microgravity would induce changes in imagined movement duration. We found this held true for the group who imagined the movements, suggesting an update of internal representations of gravity. However, we did not find a similar effect in the group exposed to microgravity despite the fact participants lived the same gravitational variations as the first group. Overall, these results suggest that motor imagery contributes to update internal representations of movement in unfamiliar environments, while a mere exposure proved to be insufficient.

Human latent-state generalization through prototype learning with discriminative attention

People cannot access the latent causes giving rise to experience. How then do they approximate the high-dimensional feature space of the external world with lower-dimensional internal models that generalize to novel examples or contexts? Here, we developed and tested a theoretical framework that internally identifies states by feature regularity (i.e., prototype states) and selectively attends to features according to their informativeness for discriminating between likely states. To test theoretical predictions, we developed experimental tasks where human subjects first learn through reward-feedback internal models of latent states governing actions associated with multi-feature stimuli. We then analyzed subjects’ response patterns to novel examples and contexts. These combined theoretical and experimental results reveal that the human ability to generalize actions involves the formation of prototype states with flexible deployment of top-down attention to discriminative features. These cognitive strategies underlie the human ability to generalize learned latent states in high-dimensional environments.

An internal model approach for motor behavior

Interacting with the environment requires a remarkable ability to control, learn, and adapt motor skills to ever-changing conditions. The intriguing complexity involved in the process of controlling, learning, and adapting motor skills has led to the development of many theoretical approaches to explain and investigate motor behavior. This paper will present a theoretical approach built upon the top-down mode of motor control that shows substantial internal coherence and has a large and growing body of empirical evidence: The Internal Models. The Internal Models are representations of the external world within the CNS, which learn to predict this external world, simulate behaviors based on sensory inputs, and transform these predictions into motor actions. We present the Internal Models’ background based on two main structures, Inverse and Forward models, explain how they work, and present some applicability.

A cerebellar internal model calibrates a feedback controller involved in sensorimotor control

Animals must adapt their behavior to survive in a changing environment. Behavioral adaptations can be evoked by two mechanisms: feedback control and internal-model-based control. Feedback controllers can maintain the sensory state of the animal at a desired level under different environmental conditions. In contrast, internal models learn the relationship between the motor output and its sensory consequences and can be used to recalibrate behaviors. Here, we present multiple unpredictable perturbations in visual feedback to larval zebrafish performing the optomotor response and show that they react to these perturbations through a feedback control mechanism. In contrast, if a perturbation is long-lasting, fish adapt their behavior by updating a cerebellum-dependent internal model. We use modelling and functional imaging to show that the neuronal requirements for these mechanisms are met in the larval zebrafish brain. Our results illustrate the role of the cerebellum in encoding internal models and how these can calibrate neuronal circuits involved in reactive behaviors depending on the interactions between animal and environment.

Time for Space and the Stability of Prospective Control: Reaching-to-Grasp Gibson

Gibson formulated an approach to goal-directed behavior using prospective information in the context of visually guided locomotion and manual behavior. The former was Gibson's paradigm case, but it is the rapidity of targeted reaching that has provided the special challenge for stable control. Recent treatments of visually guided reaching assume that internal forward models are required to generate stable behavior given delays caused by neural transmission times. Internal models are representations of the sort eschewed by Gibson in favor of prospective information. Reaching is usually described as guided using relative distances of hand and target, but prospective information is usually temporal rather than spatial. We describe proportional rate control models that incorporate time dimensioned prospective information and show they remain stable in the face of delays. The use of time-dimensioned prospective information removes the need for internal models for stable behavior despite neural transmission delays and allows Gibson's approach to prevail.

The role of motor inhibition during covert speech production

Covert speech is accompanied by a subjective multisensory experience with auditory and kinaesthetic components. An influential hypothesis states that these sensory percepts result from a simulation of the corresponding motor action that relies on the same internal models recruited for the control of overt speech. This simulationnist view raises the question of how it is possible to imagine speech without executing it. In this perspective, we discuss the possible role(s) played by motor inhibition during covert speech production. We suggest that considering covert speech as an inhibited form of overt speech maps naturally to the purported progressive internalisation of overt speech during childhood. However, we argue that the role of motor inhibition may differ widely across different forms of covert speech (e.g., condensed vs. expanded covert speech) and that considering this variety helps reconciling seemingly contradictory findings from the neuroimaging literature.

From internal models toward metacognitive AI

In several papers published in Biological Cybernetics in the 1980s and 1990s, Kawato and colleagues proposed computational models explaining how internal models are acquired in the cerebellum. These models were later supported by neurophysiological experiments using monkeys and neuroimaging experiments involving humans. These early studies influenced neuroscience from basic, sensory-motor control to higher cognitive functions. One of the most perplexing enigmas related to internal models is to understand the neural mechanisms that enable animals to learn large-dimensional problems with so few trials. Consciousness and metacognition—the ability to monitor one’s own thoughts, may be part of the solution to this enigma. Based on literature reviews of the past 20 years, here we propose a computational neuroscience model of metacognition. The model comprises a modular hierarchical reinforcement-learning architecture of parallel and layered, generative-inverse model pairs. In the prefrontal cortex, a distributed executive network called the “cognitive reality monitoring network” (CRMN) orchestrates conscious involvement of generative-inverse model pairs in perception and action. Based on mismatches between computations by generative and inverse models, as well as reward prediction errors, CRMN computes a “responsibility signal” that gates selection and learning of pairs in perception, action, and reinforcement learning. A high responsibility signal is given to the pairs that best capture the external world, that are competent in movements (small mismatch), and that are capable of reinforcement learning (small reward-prediction error). CRMN selects pairs with higher responsibility signals as objects of metacognition, and consciousness is determined by the entropy of responsibility signals across all pairs. This model could lead to new-generation AI, which exhibits metacognition, consciousness, dimension reduction, selection of modules and corresponding representations, and learning from small samples. It may also lead to the development of a new scientific paradigm that enables the causal study of consciousness by combining CRMN and decoded neurofeedback.

Temporo-cerebellar connectivity underlies timing constraints in audition

The flexible and efficient adaptation to dynamic, rapid changes in the auditory environment likely involves generating and updating of internal models. Such models arguably exploit connections between the neocortex and the cerebellum, supporting proactive adaptation. Here we tested whether temporo-cerebellar disconnection is associated with the processing of sound at short-timescales. First, we identify lesion-specific deficits for the encoding of short timescale spectro-temporal non-speech and speech properties in patients with left posterior temporal cortex stroke. Second, using lesion- guided probabilistic tractography in healthy participants, we revealed bidirectional temporo-cerebellar connectivity with cerebellar dentate nuclei and crura I/II. These findings support the view that the encoding and modeling of rapidly modulated auditory spectro-temporal properties can rely on a temporo-cerebellar interface. We discuss these findings in view of the conjecture that proactive adaptation to a dynamic environment via internal models is a generalizable principle.