A Multi-Step Neural Control for Motor Brain-Machine Interface by Reinforcement Learning

2013 ◽  
Vol 461 ◽  
pp. 565-569 ◽  
Author(s):  
Fang Wang ◽  
Kai Xu ◽  
Qiao Sheng Zhang ◽  
Yi Wen Wang ◽  
Xiao Xiang Zheng
Keyword(s):  
Reinforcement Learning ◽  
Neural Activity ◽  
Neural Control ◽  
Brain Machine Interface ◽  
Learning Approach ◽  
Complex Task ◽  
Neural Data ◽  
Neural Spikes ◽  
Performance Improvements ◽  
Machine Interface

Brain-machine interfaces (BMIs) decode cortical neural spikes of paralyzed patients to control external devices for the purpose of movement restoration. Neuroplasticity induced by conducting a relatively complex task within multistep, is helpful to performance improvements of BMI system. Reinforcement learning (RL) allows the BMI system to interact with the environment to learn the task adaptively without a teacher signal, which is more appropriate to the case for paralyzed patients. In this work, we proposed to apply Q(λ)-learning to multistep goal-directed tasks using users neural activity. Neural data were recorded from M1 of a monkey manipulating a joystick in a center-out task. Compared with a supervised learning approach, significant BMI control was achieved with correct directional decoding in 84.2% and 81% of the trials from naïve states. The results demonstrate that the BMI system was able to complete a task by interacting with the environment, indicating that RL-based methods have the potential to develop more natural BMI systems.

scholarly journals Reinforcement Learning Based Fast Self-Recalibrating Decoder for Intracortical Brain–Machine Interface

Sensors ◽  
2020 ◽  
Vol 20 (19) ◽  
pp. 5528
Author(s):  
Peng Zhang ◽  
Lianying Chao ◽  
Yuting Chen ◽  
Xuan Ma ◽  
Weihua Wang ◽  
...  
Keyword(s):  
Clinical Practice ◽  
Reinforcement Learning ◽  
Transfer Learning ◽  
Classification Accuracy ◽  
Brain Machine Interface ◽  
Brain Machine Interfaces ◽  
Neural Data ◽  
Neural Recordings ◽  
New Knowledge ◽  
Machine Interface

Background: For the nonstationarity of neural recordings in intracortical brain–machine interfaces, daily retraining in a supervised manner is always required to maintain the performance of the decoder. This problem can be improved by using a reinforcement learning (RL) based self-recalibrating decoder. However, quickly exploring new knowledge while maintaining a good performance remains a challenge in RL-based decoders. Methods: To solve this problem, we proposed an attention-gated RL-based algorithm combining transfer learning, mini-batch, and weight updating schemes to accelerate the weight updating and avoid over-fitting. The proposed algorithm was tested on intracortical neural data recorded from two monkeys to decode their reaching positions and grasping gestures. Results: The decoding results showed that our proposed algorithm achieved an approximate 20% increase in classification accuracy compared to that obtained by the non-retrained classifier and even achieved better classification accuracy than the daily retraining classifier. Moreover, compared with a conventional RL method, our algorithm improved the accuracy by approximately 10% and the online weight updating speed by approximately 70 times. Conclusions: This paper proposed a self-recalibrating decoder which achieved a good and robust decoding performance with fast weight updating and might facilitate its application in wearable device and clinical practice.

scholarly journals Toward an Autonomous Brain Machine Interface: Integrating Sensorimotor Reward Modulation and Reinforcement Learning

2015 ◽  
Vol 35 (19) ◽  
pp. 7374-7387 ◽  
Author(s):  
B. T. Marsh ◽  
V. S. A. Tarigoppula ◽  
C. Chen ◽  
J. T. Francis
Keyword(s):  
Reinforcement Learning ◽  
Brain Machine Interface ◽  
Machine Interface

scholarly journals Neural control of finger movement via intracortical brain–machine interface

2017 ◽  
Vol 14 (6) ◽  
pp. 066004 ◽  
Author(s):  
Z T Irwin ◽  
K E Schroeder ◽  
P P Vu ◽  
A J Bullard ◽  
D M Tat ◽  
...  
Keyword(s):  
Neural Control ◽  
Brain Machine Interface ◽  
Finger Movement ◽  
Machine Interface

Design of Transfer Reinforcement Learning Under Low Task Similarity

2018 ◽  
Author(s):  
Xiongqing Liu ◽  
Yan Jin
Keyword(s):  
Reinforcement Learning ◽  
Case Studies ◽  
Collision Avoidance ◽  
Negative Transfer ◽  
Learning Approach ◽  
Complex Task ◽  
Simple Task ◽  
Task Similarity

In this paper, a deep reinforcement learning approach was implemented to achieve autonomous collision avoidance. A transfer reinforcement learning approach (TRL) was proposed by introducing two concepts: transfer belief — how much confidence the agent puts in the expert’s experience, and transfer period — how long the agent’s decision is influenced by the expert’s experience. Various case studies have been conducted on transfer from a simple task — single static obstacle, to a complex task — multiple dynamic obstacles. It is found that if two tasks have low similarity, it is better to decrease initial transfer belief and keep a relatively longer transfer period, in order to reduce negative transfer and boost learning. Student agent’s learning variance grows significantly if using too short transfer period.

Brain-Machine Interface Control via Reinforcement Learning

2007 ◽  
Author(s):  
Jack DiGiovanna ◽  
Babak Mahmoudi ◽  
Jeremiah Mitzelfelt ◽  
Justin C. Sanchez ◽  
Jose C. Principe
Keyword(s):  
Reinforcement Learning ◽  
Brain Machine Interface ◽  
Interface Control ◽  
Machine Interface

Coadaptive Brain–Machine Interface via Reinforcement Learning

2009 ◽  
Vol 56 (1) ◽  
pp. 54-64 ◽  
Author(s):  
J. DiGiovanna ◽  
B. Mahmoudi ◽  
J. Fortes ◽  
J.C. Principe ◽  
J.C. Sanchez
Keyword(s):  
Reinforcement Learning ◽  
Brain Machine Interface ◽  
Machine Interface
Sign in / Sign up

Export Citation Format

Share Document