A Potential Role for Reinforcement Learning in Speech Production

2021 ◽  
pp. 1-17
Author(s):  
Benjamin Parrell
Keyword(s):  
Reinforcement Learning ◽  
Motor Development ◽  
Sensory Feedback ◽  
Potential Role ◽  
Motor Behavior ◽  
Critical Role ◽  
Motor Task ◽  
Current Evidence ◽  
Speech Learning ◽  
Speech Motor

Abstract Reinforcement learning, the ability to change motor behavior based on external reward, has been suggested to play a critical role in early stages of speech motor development and is widely used in clinical rehabilitation for speech motor disorders. However, no current evidence exists that demonstrates the capability of reinforcement to drive changes in human speech behavior. Speech provides a unique test of the universality of reinforcement learning across motor domains: Speech is a complex, high-dimensional motor task whose goals do not specify a task to be performed in the environment but ultimately must be self-generated by each speaker such that they are understood by those around them. Across four experiments, we examine whether reinforcement learning alone is sufficient to drive changes in speech behavior and parametrically test two features known to affect reinforcement learning in reaching: how informative the reinforcement signal is as well as the availability of sensory feedback about the outcomes of one's motor behavior. We show that learning does occur and is more likely when participants receive auditory feedback that gives an implicit target for production, although they do not explicitly imitate that target. Contrary to results from upper limb control, masking feedback about movement outcomes has no effect on speech learning. Together, our results suggest a potential role for reinforcement learning in speech but that it likely operates differently than in other motor domains.

scholarly journals Speech motor behavior can be altered through reinforcement learning

2020 ◽  
Author(s):  
Benjamin Parrell
Keyword(s):  
Reinforcement Learning ◽  
Motor Development ◽  
Sensory Feedback ◽  
Motor Behavior ◽  
Critical Role ◽  
Motor Task ◽  
Current Evidence ◽  
High Dimensional ◽  
Speech Learning ◽  
Speech Motor

AbstractReinforcement learning, the ability to change motor behavior based on external reward, has been suggested to play a critical role in early stages of speech motor development and is widely used in clinical rehabilitation for speech motor disorders. However, no current evidence exists that demonstrates the capability of reinforcement to drive changes in human speech behavior. Speech provides a unique test of the universality of reinforcement learning across motor domains: speech is a complex, high-dimensional motor task whose goals do not specify a task to be performed in the environment but ultimately must be self-generated by each speaker such that they are understood by those around them. Reinforcement learning may thus be more difficult for speech, given its high-dimensional and redundant motor system, while speech may also be particularly responsive to reinforcement given the ultimate goal is typically reliant on such feedback from our interlocutors. Across four experiments, we establish whether reinforcement learning alone is sufficient to drive changes in speech behavior and parametrically test two features known to affect reinforcement learning in reaching: how informative the reinforcement signal is as well as the availability of sensory feedback about the outcomes of one’s motor behavior. We show that reinforcement learning can alter speech behavior and that more informative reward signals lead to greater learning. Contrary to results from upper limb control, masking feedback about movement outcomes has no effect on speech learning. Our results suggest reinforcement learning is active in speech but may operate differently than in other motor domains.

scholarly journals Interactions between motor exploration and reinforcement learning

2019 ◽  
Vol 122 (2) ◽  
pp. 797-808 ◽  
Author(s):  
Shintaro Uehara ◽  
Firas Mawase ◽  
Amanda S. Therrien ◽  
Kendra M. Cherry-Allen ◽  
Pablo Celnik
Keyword(s):  
Reinforcement Learning ◽  
Motor System ◽  
Critical Role ◽  
Training Session ◽  
Motor Task ◽  
Preceding Trial ◽  
Successful Outcome ◽  
Subsequent Learning ◽  
Binary Feedback ◽  
As If

Motor exploration, a trial-and-error process in search for better motor outcomes, is known to serve a critical role in motor learning. This is particularly relevant during reinforcement learning, where actions leading to a successful outcome are reinforced while unsuccessful actions are avoided. Although early on motor exploration is beneficial to finding the correct solution, maintaining high levels of exploration later in the learning process might be deleterious. Whether and how the level of exploration changes over the course of reinforcement learning, however, remains poorly understood. Here we evaluated temporal changes in motor exploration while healthy participants learned a reinforcement-based motor task. We defined exploration as the magnitude of trial-to-trial change in movements as a function of whether the preceding trial resulted in success or failure. Participants were required to find the optimal finger-pointing direction using binary feedback of success or failure. We found that the magnitude of exploration gradually increased over time when participants were learning the task. Conversely, exploration remained low in participants who were unable to correctly adjust their pointing direction. Interestingly, exploration remained elevated when participants underwent a second training session, which was associated with faster relearning. These results indicate that the motor system may flexibly upregulate the extent of exploration during reinforcement learning as if acquiring a specific strategy to facilitate subsequent learning. Also, our findings showed that exploration affects reinforcement learning and vice versa, indicating an interactive relationship between them. Reinforcement-based tasks could be used as primers to increase exploratory behavior leading to more efficient subsequent learning. NEW & NOTEWORTHY Motor exploration, the ability to search for the correct actions, is critical to learning motor skills. Despite this, whether and how the level of exploration changes over the course of training remains poorly understood. We showed that exploration increased and remained high throughout training of a reinforcement-based motor task. Interestingly, elevated exploration persisted and facilitated subsequent learning. These results suggest that the motor system upregulates exploration as if learning a strategy to facilitate subsequent learning.

Can the Developmental Lag in Motor Abilities of Deaf Children Be Partly Attributed to Localization Problems?

1992 ◽  
Vol 9 (4) ◽  
pp. 343-352
Author(s):  
Geert J.P. Savelsbergh ◽  
J. Bernard Netelenbos
Keyword(s):  
Motor Development ◽  
Spatial Information ◽  
Motor Behavior ◽  
Motor Task ◽  
Head Movements ◽  
Field Of View ◽  
Deaf Children ◽  
Visual Localization ◽  
Auditory Information ◽  
Motor Abilities

Spatial information for the execution of motor behavior is acquired by orienting eye and head movements. This information can be found in our direct field of view as well as outside this field. Auditory information is especially helpful in directing our attention to information outside our initial visual field of view. Two topics on the effect of an auditory loss are discussed. Experimental evidence is provided which shows that deaf children have problems in orienting to visual stimuli situated outside their field of view. An overview is given from several studies in which the eye and head movements of deaf children are analyzed. Second, it is suggested that specific visual localization problems are partly responsible for deaf children’s characteristic lag in motor development. The latter is illustrated in two studies involving the gross motor task of ball catching.

scholarly journals Body side-specific control of motor activity during turning in a walking animal

eLife ◽  
2016 ◽  
Vol 5 ◽  
Author(s):  
Matthias Gruhn ◽  
Philipp Rosenbaum ◽  
Till Bockemühl ◽  
Ansgar Büschges
Keyword(s):  
Sensory Feedback ◽  
Neural Control ◽  
Motor Behavior ◽  
Large Body ◽  
Motor Task ◽  
Stick Insect ◽  
Body Side ◽  
Neuronal Mechanisms ◽  
Specific Control ◽  
Environmental Demands

Animals and humans need to move deftly and flexibly to adapt to environmental demands. Despite a large body of work on the neural control of walking in invertebrates and vertebrates alike, the mechanisms underlying the motor flexibility that is needed to adjust the motor behavior remain largely unknown. Here, we investigated optomotor-induced turning and the neuronal mechanisms underlying the differences between the leg movements of the two body sides in the stick insect Carausius morosus. We present data to show that the generation of turning kinematics in an insect are the combined result of descending unilateral commands that change the leg motor output via task-specific modifications in the processing of local sensory feedback as well as modification of the activity of local central pattern generating networks in a body-side-specific way. To our knowledge, this is the first study to demonstrate the specificity of such modifications in a defined motor task.

Physiologic Studies Provide New Perspectives on Early Speech Development

2010 ◽  
Vol 20 (2) ◽  
pp. 29-36
Author(s):  
Erin M. Wilson ◽  
Ignatius S. B. Nip
Keyword(s):  
Motor Control ◽  
Motor Development ◽  
Motion Tracking ◽  
Speech Development ◽  
Speech Motor Control ◽  
Therapeutic Approaches ◽  
Facial Motion ◽  
Disordered Speech ◽  
Speech Motor ◽  
Physiologic Data

Abstract Although certain speech development milestones are readily observable, the developmental course of speech motor control is largely unknown. However, recent advances in facial motion tracking systems have been used to investigate articulator movements in children and the findings from these studies are being used to further our understanding of the physiologic basis of typical and disordered speech development. Physiologic work has revealed that the emergence of speech is highly dependent on the lack of flexibility in the early oromotor system. It also has been determined that the progression of speech motor development is non-linear, a finding that has motivated researchers to investigate how variables such as oromotor control, cognition, and linguistic factors affect speech development in the form of catalysts and constraints. Physiologic data are also being used to determine if non-speech oromotor behaviors play a role in the development of speech. This improved understanding of the physiology underlying speech, as well as the factors influencing its progression, helps inform our understanding of speech motor control in children with disordered speech and provide a framework for theory-driven therapeutic approaches to treatment.

Human Cerebral Potentials During Motor Training Under Different Forms of Sensory Feedback

1999 ◽  
Vol 13 (4) ◽  
pp. 234-244
Author(s):  
Uwe Niederberger ◽  
Wolf-Dieter Gerber
Keyword(s):  
Healthy Subjects ◽  
Sensory Feedback ◽  
Motor Task ◽  
Tactile Feedback ◽  
Proprioceptive Feedback ◽  
Motor Training ◽  
Feedback Group ◽  
Emg Feedback ◽  
The Right ◽  
Training Success

Abstract In two experiments with four and two groups of healthy subjects, a novel motor task, the voluntary abduction of the right big toe, was trained. This task cannot usually be performed without training and is therefore ideal for the study of elementary motor learning. A systematic variation of proprioceptive, tactile, visual, and EMG feedback was used. In addition to peripheral measurements such as the voluntary range of motion and EMG output during training, a three-channel EEG was recorded over Cz, C3, and C4. The movement-related brain potential during distinct periods of the training was analyzed as a central nervous parameter of the ongoing learning process. In experiment I, we randomized four groups of 12 subjects each (group P: proprioceptive feedback; group PT: proprioceptive and tactile feedback; group PTV: proprioceptive, tactile, and visual feedback; group PTEMG: proprioceptive, tactile, and EMG feedback). Best training results were reported from the PTEMG and PTV groups. The movement-preceding cortical activity, in the form of the amplitude of the readiness potential at the time of EMG onset, was greatest in these two groups. Results of experiment II revealed a similar effect, with a greater training success and a higher electrocortical activation under additional EMG feedback compared to proprioceptive feedback alone. Sensory EMG feedback as evaluated by peripheral and central nervous measurements appears to be useful in motor training and neuromuscular re-education.

Using serum biomarkers for identifying unstable carotid plaque: update of current evidence

2020 ◽  
Vol 26 ◽  
Author(s):  
Areti Sofogianni ◽  
Konstantinos Tziomalos ◽  
Triantafyllia Koletsa ◽  
Apostolos G. Pitoulias ◽  
Lemonia Skoura ◽  
...  
Keyword(s):  
High Risk ◽  
Great Proportion ◽  
Current Literature ◽  
Early Identification ◽  
Carotid Plaque ◽  
Potential Role ◽  
Carotid Atherosclerosis ◽  
Serum Biomarkers ◽  
Current Evidence

: Carotid atherosclerosis is responsible for a great proportion of ischemic strokes. Early identification of unstable or vulnerable carotid plaques and therefore of patients at high risk for stroke is of significant medical and socioeconomical value. We reviewed the current literature and discuss the potential role of the most important serum biomarkers in identifying patients with carotid atherosclerosis who are at high risk for atheroembolic stroke.

Rehabilitation of Post Stroke Sensory Dysfunction—A Scoping Review

2021 ◽  
pp. 251660852098429
Author(s):  
Dorcas B. C. Gandhi ◽  
Ivy Anne Sebastian ◽  
Komal Bhanot
Keyword(s):  
Motor Behavior ◽  
Sensory Modality ◽  
Sensory System ◽  
Sensory Stimulation ◽  
Current Evidence ◽  
Cochrane Library ◽  
Post Stroke ◽  
Sensory Dysfunction ◽  
Electronic Database Search ◽  
External Stimuli

Sensory dysfunction is one of the common impairments that occurs post stroke. With sensory changes in all modalities, it also affects the quality of life and incites suicidal thoughts. The article attempts to review and describe the current evidence of various approaches of assessment and rehabilitation for post-stroke sensory dysfunction. After extensive electronic database search across Medline, Embase, EBSCO, and Cochrane library, it generated 2433 results. After screening according to inclusion and exclusion criteria, we included 11 studies. We categorized data based on type of sensory deficits and prevalence, role of sensory system on motor behavior, type of intervention, sensory modality targeted, and dosage of intervention and outcome measures used for rehabilitation. Results found the strong evidence of involvement of primary and secondary motor areas involved in processing and responding to somatosensation, respectively. We divided rehabilitation approaches into sensory stimulation approach and sensory retraining approach focused on using external stimuli and relearning, respectively. However, with varied aims and targeted sensory involvement, the study applicability is affected. Thus, this emerges the need of extensive research in future for evidence-based practice of assessments and rehabilitation on post-stroke sensory rehabilitation.

scholarly journals Optimal Energy Management of a Grid-Tied Solar PV-Battery Microgrid: A Reinforcement Learning Approach

Energies ◽  
2021 ◽  
Vol 14 (9) ◽  
pp. 2700
Author(s):  
Grace Muriithi ◽  
Sunetra Chowdhury
Keyword(s):  
Renewable Energy ◽  
Reinforcement Learning ◽  
Energy Management ◽  
Renewable Energy Sources ◽  
Critical Role ◽  
Winter Season ◽  
Learning Approach ◽  
Solar Pv ◽  
Optimal Energy ◽  
Optimal Energy Management

In the near future, microgrids will become more prevalent as they play a critical role in integrating distributed renewable energy resources into the main grid. Nevertheless, renewable energy sources, such as solar and wind energy can be extremely volatile as they are weather dependent. These resources coupled with demand can lead to random variations on both the generation and load sides, thus complicating optimal energy management. In this article, a reinforcement learning approach has been proposed to deal with this non-stationary scenario, in which the energy management system (EMS) is modelled as a Markov decision process (MDP). A novel modification of the control problem has been presented that improves the use of energy stored in the battery such that the dynamic demand is not subjected to future high grid tariffs. A comprehensive reward function has also been developed which decreases infeasible action explorations thus improving the performance of the data-driven technique. A Q-learning algorithm is then proposed to minimize the operational cost of the microgrid under unknown future information. To assess the performance of the proposed EMS, a comparison study between a trading EMS model and a non-trading case is performed using a typical commercial load curve and PV profile over a 24-h horizon. Numerical simulation results indicate that the agent learns to select an optimized energy schedule that minimizes energy cost (cost of power purchased from the utility and battery wear cost) in all the studied cases. However, comparing the non-trading EMS to the trading EMS model operational costs, the latter one was found to decrease costs by 4.033% in summer season and 2.199% in winter season.

Clinical Assessment of Oropharyngeal Motor Development in Young Children

1987 ◽  
Vol 52 (3) ◽  
pp. 271-277 ◽  
Author(s):  
JoAnne Robbins ◽  
Thomas Klee
Keyword(s):  
Clinical Application ◽  
Clinical Assessment ◽  
Motor Development ◽  
Structural Integrity ◽  
Vocal Tract ◽  
Developmental Change ◽  
Clinical Protocol ◽  
Motor Functioning ◽  
Motor Abilities ◽  
Speech Motor

A clinical protocol was developed for the purpose of assessing the oral and speech motor abilities of children. An 86-item test was administered to 90 normally developing children aged 2:6–6:11. Evaluations of the structural integrity of the vocal tract did not show developmental change, although evaluations of oral and speech motor functioning changed significantly with age. The functional portion of the protocol was most sensitive to developmental change up to age 3:6, with an asymptote in performance thereafter. Clinical application of the protocol is discussed.

Sign in / Sign up

Export Citation Format

Share Document