The Dopamine System and Automatization of Movement Sequences: A Review With Relevance for Speech and Stuttering

The last decades of research have gradually elucidated the complex functions of the dopamine system in the vertebrate brain. The multiple roles of dopamine in motor function, learning, attention, motivation, and the emotions have been difficult to reconcile. A broad and detailed understanding of the physiology of cerebral dopamine is of importance in understanding a range of human disorders. One of the core functions of dopamine involves the basal ganglia and the learning and execution of automatized sequences of movements. Speech is one of the most complex and highly automatized sequential motor behaviors, though the exact roles that the basal ganglia and dopamine play in speech have been difficult to determine. Stuttering is a speech disorder that has been hypothesized to be related to the functions of the basal ganglia and dopamine. The aim of this review was to provide an overview of the current understanding of the cerebral dopamine system, in particular the mechanisms related to motor learning and the execution of movement sequences. The primary aim was not to review research on speech and stuttering, but to provide a platform of neurophysiological mechanisms, which may be utilized for further research and theoretical development on speech, speech disorders, and other behavioral disorders. Stuttering and speech are discussed here only briefly. The review indicates that a primary mechanism for the automatization of movement sequences is the merging of isolated movements into chunks that can be executed as units. In turn, chunks can be utilized hierarchically, as building blocks of longer chunks. It is likely that these mechanisms apply also to speech, so that frequent syllables and words are produced as motor chunks. It is further indicated that the main learning principle for sequence learning is reinforcement learning, with the phasic release of dopamine as the primary teaching signal indicating successful sequences. It is proposed that the dynamics of the dopamine system constitute the main neural basis underlying the situational variability of stuttering.

The role of feedback in the production of skilled finger sequences

Actions involving fine control of the hand, for example grasping an object, rely heavily on sensory information from the fingertips. While the integration of feedback during execution of individual movements is well understood, less is known about the use of sensory feedback in the control of skilled movement sequences. To address this gap, we trained participants to produce sequences of finger movements on a keyboard-like device over a four-day training period. Participants received haptic, visual, and auditory feedback indicating the occurrence of each finger press. We then either transiently delayed or advanced the feedback for a single press by a small amount of time (30 or 60 ms). We observed that participants rapidly adjusted their ongoing finger press by either accelerating or prolonging the ongoing press, in accordance with the direction of the perturbation. Furthermore, we could show that this rapid behavioural modulation was driven by haptic feedback. While these feedback-driven adjustments reduced in size with practice, they were still clearly present at the end of training. In contrast to the directionally-specific effect we observed on the perturbed press, a feedback perturbation resulted in a delayed onset of the subsequent presses irrespective of perturbation direction or feedback modality. This observation is consistent with a hierarchical organization of skilled movement sequences, with different levels reacting distinctly to sensory perturbations.

A Neural Network Model for Learning 3D Object Representations Through Haptic Exploration

Humans initially learn about objects through the sense of touch, in a process called “haptic exploration.” In this paper, we present a neural network model of this learning process. The model implements two key assumptions. The first is that haptic exploration can be thought of as a type of navigation, where the exploring hand plays the role of an autonomous agent, and the explored object is this agent's “local environment.” In this scheme, the agent's movements are registered in the coordinate system of the hand, through slip sensors on the palm and fingers. Our second assumption is that the learning process rests heavily on a simple model of sequence learning, where frequently-encountered sequences of hand movements are encoded declaratively, as “chunks.” The geometry of the object being explored places constraints on possible movement sequences: our proposal is that representations of possible, or frequently-attested sequences implicitly encode the shape of the explored object, along with its haptic affordances. We evaluate our model in two ways. We assess how much information about the hand's actual location is conveyed by its internal representations of movement sequences. We also assess how effective the model's representations are in a reinforcement learning task, where the agent must learn how to reach a given location on an explored object. Both metrics validate the basic claims of the model. We also show that the model learns better if objects are asymmetrical, or contain tactile landmarks, or if the navigating hand is articulated, which further constrains the movement sequences supported by the explored object.

Eye-movement replay supports episodic remembering

When we bring to mind something we have seen before, our eyes spontaneously reproduce a pattern strikingly similar to that made during the original encounter. Eye-movements can then serve the opposite purpose to acquiring new visual information; they can serve as self-generated cues, pointing to memories already stored. By isolating separable properties within the closely bound chain of where and when we look, we demonstrate that specific components of dynamically reinstated eye-movement sequences, facilitate different aspects of episodic remembering. We also show that the fidelity with which a series of connected eye-movements from initial encoding is reproduced during subsequent retrieval, predicts the quality of the recalled memory. Our findings indicate that eye movements are “replayed” to assemble visuospatial relations as we remember. Distinct dimensions of these scanpaths differentially contribute depending on the goal-relevant memory.

Evolution of combinatorial structure in learned forms through embodied iterated learning in a robot collective

In this study, we use mobile robots as physical entities to model the iterated learning of collections of forms that consist of randomly generated movement sequences. The robots implement an abstract model of embodied iterated social learning in which the forms evolve due to limited perceptual abilities of the robots during multiple learning cycles. It is shown that shared chunks that consisted of similar movement sequences emerge in the learned forms, and as these emergent shared sequences can be learned with high accuracy, they cause a cumulative increase in the learnability of the collections. Therefore, we are able to present robotic experiments in which embodied learning on robots leads to combinatorial structure as a result of cultural interactions in the form of iterated learning without a communicative task.

Lateral prefrontal cortex as a hub for music production with gradation from structural rules to movement sequences

Complex sequential behaviours, such as speaking or playing music, often entail the flexible, rule-based chaining of single acts. However, it remains unclear how the brain translates abstract structural rules into concrete series of movements. Here we demonstrate a multi-level contribution of anatomically distinct cognitive and motor networks to the execution of novel musical sequences. We combined functional and diffusion-weighted neuroimaging to dissociate high-level structural and low-level motor planning of musical chord sequences executed on a piano. Fronto-temporal and fronto-parietal neural networks were involved when sequences violated pianists’ structural or motor plans, respectively. Prefrontal cortex is identified as a hub where both networks converge within an anterior-to-posterior gradient of action control linking abstract structural rules to concrete movement sequences.

The planning horizon for movement sequences

When performing a long chain of actions in rapid sequence, future movements need to be planned concurrently with ongoing action. However, how far ahead we plan, and whether this ability improves with practice, is currently unknown. Here we designed an experiment in which healthy volunteers produced sequences of 14 finger presses quickly and accurately on a keyboard in response to numerical stimuli. On every trial, participants were only shown a fixed number of stimuli ahead of the current keypress. The size of this viewing window varied between 1 (next digit revealed with the pressing of the current key) and 14 (full view of the sequence). Participants practiced the task for five days and their performance was continuously assessed on random sequences. Our results indicate that participants used the available visual information to plan multiple actions into the future, but that the planning horizon was limited: receiving information about more than 3 movements ahead did not result in faster sequence production. Over the course of practice, we found larger performance improvements for larger viewing windows and an expansion of the planning horizon. These findings suggest that the ability to plan future responses during ongoing movement constitutes an important aspect of skillful movement. Based on the results, we propose a framework to investigate the neuronal processes underlying simultaneous planning and execution.Significance StatementAlthough skill learning has typically focused on the training of specific movement sequences, practice improves performance even for random sequences. Here we hypothesize that a fundamental aspect of skilled sequential behavior is the ability to plan multiple actions into the future, both before and during execution. By controlling the amount of visual information available for motor planning, we show that people plan at least three movements beyond current action and that this planning horizon expands with practice. Our results suggest that coordinating ongoing movement and planning of future actions is central to a wide range of cognitive and motor tasks, providing a new perspective on the neural implementation of motor planning in the context of sequential behavior.