An active inference model of hierarchical action understanding, learning and imitation

We advance a novel computational model of the acquisition of a hierarchical action repertoire and its use for observation, understanding and motor control. The model is grounded in a principled framework to understand brain and cognition: active inference. We exemplify the functioning of the model by presenting four simulations of a tennis learner who observes a teacher performing tennis shots and forms hierarchical representations of the observed actions - including both actions that are already in her repertoire and novel actions - and finally imitates them. Our simulations that show that the agent’s oculomotor activity implements an active information sampling strategy that permits inferring the kinematics aspects of the observed movement, which lie at the lowest level of the action hierarchy. In turn, this low-level kinematic inference supports higher-level inferences about deeper aspects of the observed actions, such as their proximal goals and intentions. Finally, the inferred action representations can steer imitative motor responses, but interfere with the execution of different actions. Taken together, our simulations show that the same hierarchical active inference model provides a unified account of action observation, understanding, learning and imitation. Finally, our model provides a computational rationale to explain the neurobiological underpinnings of visuomotor cognition, including the multiple routes for action understanding in the dorsal and ventral streams and mirror mechanisms.

Shared Representations in Athletes: Segmenting Action Sequences From Taekwondo Reveals Implicit Agreement

How do athletes represent actions from their sport? How are these representations structured and which knowledge is shared among experts in the same discipline? To address these questions, the event segmentation task was used. Experts in Taekwondo and novices indicated how they would subjectively split videos of Taekwondo form sequences into meaningful units. In previous research, this procedure was shown to unveil the structure of internal action representations and to be affected by sensorimotor knowledge. Without specific instructions on the grain size of segmentation, experts tended to integrate over longer episodes which resulted in a lower number of single units. Moreover, in accordance with studies in figure-skating and basketball, we expected higher agreement among experts on where to place segmentation marks, i.e., boundaries. In line with this hypothesis, significantly more overlap of boundaries was found within the expert group as compared to the control group. This was observed even though the interindividual differences in the selected grain size were huge and expertise had no systematic influence here. The absence of obvious goals or objects to structure Taekwondo forms underlines the importance of shared expert knowledge. Further, experts might have benefited from sensorimotor skills which allowed to simulate the observed actions more precisely. Both aspects may explain stronger agreement among experts even in unfamiliar Taekwondo forms. These interpretations are descriptively supported by the participants’ statements about features which guided segmentation and by an overlap of the group’s agreed boundaries with those of an experienced referee. The study shows that action segmentation can be used to provide insights into structure and content of action representations specific to experts. The mechanisms underlying shared knowledge among Taekwondoists and among experts in general are discussed on the background of current theoretic frameworks.

Sound-Action Symbolism

Recent evidence has shown linkages between actions and segmental elements of speech. For instance, close-front vowels are sound symbolically associated with the precision grip, and front vowels are associated with forward-directed limb movements. The current review article presents a variety of such sound-action effects and proposes that they compose a category of sound symbolism that is based on grounding a conceptual knowledge of a referent in articulatory and manual action representations. In addition, the article proposes that even some widely known sound symbolism phenomena such as the sound-magnitude symbolism can be partially based on similar sensorimotor grounding. It is also discussed that meaning of suprasegmental speech elements in many instances is similarly grounded in body actions. Sound symbolism, prosody, and body gestures might originate from the same embodied mechanisms that enable a vivid and iconic expression of a meaning of a referent to the recipient.

Does Temporal Distance Influence Abstraction? A Large Pre-Registered Experiment

Construal level theory has been extraordinarily generative both within and beyond social psychology, yet the individual effects that form the empirical foundation of the theory have yet to be carefully probed and precisely estimated using large samples and preregistered analysis plans. In a highly powered and preregistered study, we tested the effect of temporal distance on abstraction, using one of the most common operationalizations of temporal distance (thinking about a future point in time that is one day vs. one year from today) and one of the most common operationalizations of abstraction (preference for more abstract vs. concrete action representations, as assessed by the Behavioral Identification Form). Participants preferred significantly more abstract action representations in the distant (vs. near) future condition, with an effect size of d = .276, 95% CI [.097, .455]. We discuss implications, future directions, and constraints on the generality of these results.

Consolidation of human skill linked to waking hippocampo-neocortical replay

The introduction of rest intervals interspersed with practice strengthens wakeful consolidation of skill. The mechanisms by which the brain binds discrete action representations into consolidated, highly temporally-resolved skill sequences during waking rest are not known. To address this question, we recorded magnetoencephalography (MEG) during acquisition and rapid consolidation of a sequential motor skill. We report the presence of highly prominent, fast waking neural replay during the same rest periods in which rapid consolidation occurs. The observed replay was temporally compressed by approximately 20x relative to the acquired skill, occurred in both forward and reverse directions, was selective for the trained sequence and predicted the magnitude of skill consolidation. Replay representations extended beyond the hippocampus and entrorhinal cortex to the contralateral sensorimotor cortex. These results document the presence of robust hippocampo-neocortical replay supporting rapid wakeful consolidation of skill.

Reward-related choices determine information timing and flow across macaque lateral prefrontal cortex

Prefrontal cortex is critical for cognition. Although much is known about the representation of cognitive variables in the prefrontal cortex, much less is known about the spatio-temporal neural dynamics that underlie cognitive operations. In the present study, we examined information timing and flow across the lateral prefrontal cortex (LPFC), while monkeys carried out a two-armed bandit reinforcement learning task in which they had to learn to select rewarding actions or rewarding objects. When we analyzed signals independently within subregions of the LPFC, we found a task-specific, caudo-rostral gradient in the strength and timing of signals related to chosen objects and chosen actions. In addition, when we characterized information flow among subregions, we found that information flow from action to object representations was stronger from the dorsal to ventral LPFC, and information flow from object to action representations was stronger from the ventral to dorsal LPFC. The object to action effects were more pronounced in object blocks, and also reflected learning specifically in these blocks. These results suggest anatomical segregation followed by the rapid integration of information within the LPFC.