A Formal Account of Structuring Motor Actions With Sensory Prediction for a Naive Agent

For naive robots to become truly autonomous, they need a means of developing their perceptive capabilities instead of relying on hand crafted models. The sensorimotor contingency theory asserts that such a way resides in learning invariants of the sensorimotor flow. We propose a formal framework inspired by this theory for the description of sensorimotor experiences of a naive agent, extending previous related works. We then use said formalism to conduct a theoretical study where we isolate sufficient conditions for the determination of a sensory prediction function. Furthermore, we also show that algebraic structure found in this prediction can be taken as a proxy for structure on the motor displacements, allowing for the discovery of the combinatorial structure of said displacements. Both these claims are further illustrated in simulations where a toy naive agent determines the sensory predictions of its spatial displacements from its uninterpreted sensory flow, which it then uses to infer the combinatorics of said displacements.

Intentional Binding Effects in the Experience of Noticing the Regularity of a Perceptual-Motor Task

Noticing the regularity of the task is necessary to enhance motor performance. The experience of noticing further motivates improvement in motor performance. Motor control is explained by a comparator model that modifies the motor command to reduce discrepancies between sensory predictions and actual outcomes. A similar model could apply to sense of agency (SoA). SoA refers to the sensation of controlling one’s own actions and, through them, the outcomes in the external world. SoA may also be enhanced by the experience of noticing errors. We recently reported gradual enhancement of SoA in participants with high perceptual-motor performance. However, what component of the motor task changed the SoA is unclear. In this study, we aimed to investigate the influence over time of the experience of noticing during a motor task on SoA. Participants performed an implicit regularity perceptual-motor task and an intentional binding task (a method that can quantitatively measure SoA) simultaneously. We separated participants into groups after the experiment based on noticing or not noticing the regularity. SoA was gradually enhanced in the noticing group, compared with that of the non-noticing group. The results suggest that the experience of noticing may enhance SoA during perceptual-motor tasks.

Topographic connectivity reveals task-dependent retinotopic processing throughout the human brain.

The human visual system is organized as a hierarchy of maps that share the retina's topography. Although retinotopic maps have been identified throughout the brain, how much of the brain is visually organized remains unknown. Here we demonstrate widespread stable visual organization beyond the traditional visual system by analyzing topographic connectivity with primary visual cortex during moviewatching, rest, and retinotopic mapping. Detailed visual-spatial organization derived from retinotopic connectivity is modulated by experimental condition. Specifically, traditionally visual regions alternate with default mode network and hippocampus in preferentially representing the center of the visual field. This visual role of hippocampus would allow it to implement sensory predictions by interfacing between abstract memories and concrete perceptions. These results indicate that pervasive sensory coding facilitates the communication between far-flung brain regions.

Phantom auditory perception (tinnitus) is characterised by stronger anticipatory auditory predictions

How phantom perceptions arise and the factors that make individuals prone to such experiences are not well understood. An attractive phenomenon to study these questions is tinnitus, a very common auditory phantom perception which is not explained by hyperactivity in the auditory pathway alone. Our framework posits that a predisposition to developing (chronic) tinnitus is dependent on individual traits relating to the formation and utilization of sensory predictions. Predictions of auditory stimulus frequency (remote from tinnitus frequency) were studied using a paradigm parametrically modulating regularity (i.e. predictability) of tone sequences and applying decoding techniques on magnetoencephalographic (MEG) data. For processes likely linked to short-term memory, individuals with tinnitus showed an enhanced anticipatory prediction pattern associated with increasing sequence regularity. In contrast, individuals without tinnitus engaged the same processes following the onset of the to-be-decoded sound. We posit that this tendency to optimally anticipate static and changing auditory inputs may determine which individuals faced with persistent auditory pathway hyperactivity factor it into auditory predictions, and thus perceive it as tinnitus. While our study constitutes a first step relating vulnerability to tinnitus with predictive processing, longitudinal studies are needed to confirm the predisposition model of tinnitus development.

Action Intention-based and Stimulus Regularity-based Predictions: Same or Different?

We act on the environment to produce desired effects, but we also adapt to the environmental demands by learning what to expect next, based on experience: How do action-based predictions and sensory predictions relate to each other? We explore this by implementing a self-generation oddball paradigm, where participants performed random sequences of left and right button presses to produce frequent standard and rare deviant tones. By manipulating the action–tone association as well as the likelihood of a button press over the other one, we compare ERP effects evoked by the intention to produce a specific tone, tone regularity, and both intention and regularity. We show that the N1b and Tb components of the N1 response are modulated by violations of tone regularity only. However, violations of action intention as well as of regularity elicit MMN responses, which occur similarly in all three conditions. Regardless of whether the predictions at sensory levels were based on either intention, regularity, or both, the tone deviance was further and equally well detected at hierarchically higher processing level, as reflected in similar P3a effects between conditions. We did not observe additive prediction errors when intention and regularity were violated concurrently, suggesting the two integrate despite presumably having independent generators. Even though they are often discussed as individual prediction sources in the literature, this study represents to our knowledge the first to directly compare them. Finally, these results show how, in the context of action, our brain can easily switch between top–down intention-based expectations and bottom–up regularity cues to efficiently predict future events.

Emotional learning promotes perceptual predictions by remodeling stimulus representation in visual cortex

Emotions exert powerful effects on perception and memory, notably by modulating activity in sensory cortices so as to capture attention. Here, we examine whether emotional significance acquired by a visual stimulus can also change its cortical representation by linking neuronal populations coding for different memorized versions of the same stimulus, a mechanism that would facilitate recognition across different appearances. Using fMRI, we show that after pairing a given face with threat through conditioning, viewing this face activates the representation of another viewpoint of the same person, which itself was never conditioned, leading to robust repetition-priming across viewpoints in the ventral visual stream (including medial fusiform, lateral occipital, and anterior temporal cortex). We also observed a functional-anatomical segregation for coding view-invariant and view-specific identity information. These results indicate emotional signals may induce plasticity of stimulus representations in visual cortex, serving to generate new sensory predictions about different appearances of threat-associated stimuli.

Filtering Compensation for Delays and Prediction Errors during Sensorimotor Control

Compensating for sensorimotor noise and for temporal delays has been identified as a major function of the nervous system. Although these aspects have often been described separately in the frameworks of optimal cue combination or motor prediction during movement planning, control-theoretic models suggest that these two operations are performed simultaneously, and mounting evidence supports that motor commands are based on sensory predictions rather than sensory states. In this letter, we study the benefit of state estimation for predictive sensorimotor control. More precisely, we combine explicit compensation for sensorimotor delays and optimal estimation derived in the context of Kalman filtering. We show, based on simulations of human-inspired eye and arm movements, that filtering sensory predictions improves the stability margin of the system against prediction errors due to low-dimensional predictions or to errors in the delay estimate. These simulations also highlight that prediction errors qualitatively account for a broad variety of movement disorders typically associated with cerebellar dysfunctions. We suggest that adaptive filtering in cerebellum, instead of often-assumed feedforward predictions, may achieve simple compensation for sensorimotor delays and support stable closed-loop control of movements.

In vivo assay of cortical microcircuitry in frontotemporal dementia: a platform for experimental medicine studies

The analysis of neural circuits can provide critical insights into the mechanisms of neurodegeneration and dementias, and offer potential quantitative biological tools to assess novel therapeutics. Here we use behavioural variant frontotemporal dementia (bvFTD) as a model disease. We demonstrate that inversion of canonical microcircuit models to non-invasive human magnetoecphalography can identify the regional- and laminar-specificity of bvFTD pathophysiology, and their parameters can accurately differentiate patients from matched healthy controls. Using such models, we show that changes in local coupling in frontotemporal dementia underlie the failure to adequately establish sensory predictions, leading to altered prediction error responses in a cortical information-processing hierarchy. Using machine learning, this model-based approach provided greater case-control classification accuracy than conventional evoked cortical responses. We suggest that this approach provides an in vivo platform for testing mechanistic hypotheses about disease progression and pharmacotherapeutics.