Back to Reality: Differences in Learning Strategy in a Simplified Virtual and a Real Throwing Task

Virtual environments have been widely utilized in motor neuroscience and rehabilitation as they afford tight control of sensorimotor conditions and readily afford visual and haptic manipulations. However, typically studies have only examined performance in the virtual testbeds, without asking how performance in the virtual environment compares to behavior in the real world. To test that, this study compared throwing in a virtual and real set-up where the task parameters were precisely matched. Even though the virtual task only required a single-joint arm movement, similar to many simplified movement assays in motor neuroscience, throwing accuracy and precision was significantly better in the real task; only after three days did the performance reach same levels. To gain more insight into the structure of the learning process, movement variability was decomposed into deterministic and stochastic contributions to distinct stages of learning by using the TNC method: Tolerance was optimized first and was higher in the virtual environment, suggesting that more familiarization and exploration is needed in the virtual task. Covariation and noise showed far fewer and only contributes late in the real task, indicating that subjects reached the stage of fine-tuning of variability only in the real task. These results showed that while the tasks were precisely matched, the simplified movements in the virtual environment required more practice to be successful. These findings resonate with the reported problems in transfer of therapeutic benefits from virtual to real environments and alert that the use of virtual environments in research and rehabilitation needs more caution.

Hierarchical motor control in mammals and machines

Advances in artificial intelligence are stimulating interest in neuroscience. However, most attention is given to discrete tasks with simple action spaces, such as board games and classic video games. Less discussed in neuroscience are parallel advances in “synthetic motor control”. While motor neuroscience has recently focused on optimization of single, simple movements, AI has progressed to the generation of rich, diverse motor behaviors across multiple tasks, at humanoid scale. It is becoming clear that specific, well-motivated hierarchical design elements repeatedly arise when engineering these flexible control systems. We review these core principles of hierarchical control, relate them to hierarchy in the nervous system, and highlight research themes that we anticipate will be critical in solving challenges at this disciplinary intersection.

Detecting task-dependent modulation of spatiotemporal module via tensor decomposition: application to kinematics and EMG data for walking and running at various speed

How the central nervous system (CNS) controls many joints and muscles is a fundamental question in motor neuroscience and related research areas. An attractive hypothesis is the module hypothesis: the CNS controls groups of joints or muscles (i.e., spatial modules) while providing time-varying motor commands (i.e., temporal modules) to the spatial modules rather than controlling each joint or muscle separately. Another fundamental question is how the CNS generates numerous repertories of movement patterns. One hypothesis is that the CNS modulates the spatial and/or temporal modules depending on the required tasks. It is thus essential to quantify the spatial module, the temporal module, and the task-dependent modulation of those modules. Although previous methods attempted to quantify these aspects, they considered the modulation in only the spatial or temporal module. These limitations were possibly due to the constraints inherent to conventional methods for quantifying the spatial and temporal modules. Here, we demonstrate the effectiveness of tensor decomposition in quantifying the spatial module, the temporal module, and the task-dependent modulation of these modules without such limitations. We further demonstrate that the tensor decomposition provides a new perspective on the task-dependent modulation of spatiotemporal modules: in switching from walking to running, the CNS modulates the peak timing in the temporal module while recruiting proximal muscles in the corresponding spatial module.Author summaryThere are at least two fundamental questions in motor neuroscience and related research areas: 1) how does the central nervous system (CNS) control many joints and muscles and 2) how does the CNS generate numerous repertories of movement patterns. One possible answer to question 1) is that the CNS controls groups of joints or muscles (i.e., spatial modules) while providing time-varying motor commands (i.e., temporal modules) to the spatial modules rather than controlling each joint or muscle separately. One possible answer to question 2) is that the CNS modulates the spatial and/or temporal module depending on the required tasks. It is thus essential to quantify the spatial module, the temporal module, and the task-dependent modulation of those modules. Here, we demonstrate the effectiveness of tensor decomposition in quantifying the modules and those task-dependent modulations while overcoming the shortcomings inherent to previous methods. We further show that the tensor decomposition provides a new perspective on how the CNS switches between walking and running. The CNS modulated the peak timing in the temporal module while recruiting proximal muscles in the corresponding spatial module.

Extrinsic and intrinsic dynamics in movement intermittency

What determines how we move in the world? Motor neuroscience often focusses either on intrinsic rhythmical properties of motor circuits or extrinsic sensorimotor feedback loops. Here we show that the interplay of both intrinsic and extrinsic dynamics is required to explain the intermittency observed in continuous tracking movements. Using spatiotemporal perturbations in humans, we demonstrate that apparently discrete submovements made 2–3 times per second reflect constructive interference between motor errors and continuous feedback corrections that are filtered by intrinsic circuitry in the motor system. Local field potentials in monkey motor cortex revealed characteristic signatures of a Kalman filter, giving rise to both low-frequency cortical cycles during movement, and delta oscillations during sleep. We interpret these results within the framework of optimal feedback control, and suggest that the intrinsic rhythmicity of motor cortical networks reflects an internal model of external dynamics, which is used for state estimation during feedback-guided movement.Editorial note: This article has been through an editorial process in which the authors decide how to respond to the issues raised during peer review. The Reviewing Editor's assessment is that all the issues have been addressed (<xref ref-type="decision-letter" rid="SA1">see decision letter</xref>).

Extrinsic and Intrinsic Dynamics in Movement Intermittency

What determines how we move in the world? Motor neuroscience often focusses either on intrinsic rhythmical properties of motor circuits or extrinsic sensorimotor feedback loops. Here we show that the interplay of both intrinsic and extrinsic dynamics is required to explain the intermittency observed in continuous tracking movements. Using spatiotemporal perturbations in humans, we demonstrate that apparently discrete submovements made 2-3 times per second reflect constructive interference between motor errors and continuous feedback corrections that are filtered by intrinsic circuitry in the motor system. Local field potentials in monkey motor cortex revealed characteristic signatures of a Kalman filter giving rise to both low-frequency cortical cycles during movement, and delta oscillations during sleep. We interpret these results within the framework of optimal feedback control, and suggest that the intrinsic rhythmicity of motor cortical networks reflects an internal model of external dynamics which is used for state estimation during feedback-guided movement.

Detection of task-relevant and task-irrelevant motion sequences: application to motor adaptation in goal-directed and whole-body movements

Motor variability is inevitable in our body movements and is discussed from several various perspectives in motor neuroscience and biomechanics; it can originate from the variability of neural activities, it can reflect a large degree of freedom inherent in our body movements, it can decrease muscle fatigue, or it can facilitate motor learning. How to evaluate motor variability is thus a fundamental question in motor neuroscience and biomechanics. Previous methods have quantified (at least) two striking features of motor variability; the smaller variability in the task-relevant dimension than in the task-irrelevant dimension and the low-dimensional structure that is often referred to as synergy or principal component. However, those previous methods were not only unsuitable for quantifying those features simultaneously but also applicable in some limited conditions (e.g., a method cannot consider motion sequence, and another method cannot consider how each motion is relevant to performance). Here, we propose a flexible and straightforward machine learning technique that can quantify task-relevant variability, task-irrelevant variability, and the relevance of each principal component to task performance while considering the motion sequence and the relevance of each motion sequence to task performance in a data-driven manner. We validate our method by constructing a novel experimental setting to investigate goal-directed and whole-body movements. Furthermore, our setting enables the induction of motor adaptation by using perturbation and evaluating the modulation of task-relevant and task-irrelevant variabilities through motor adaptation. Our method enables the identification of a novel property of motor variability; the modulation of those variabilities differs depending on the perturbation schedule. Although a gradually imposed perturbation does not increase both task-relevant and task-irrelevant variabilities, a constant perturbation increases task-relevant variability.