scholarly journals Reinforcement Learning Based Fast Self-Recalibrating Decoder for Intracortical Brain–Machine Interface

Sensors ◽  
2020 ◽  
Vol 20 (19) ◽  
pp. 5528
Author(s):  
Peng Zhang ◽  
Lianying Chao ◽  
Yuting Chen ◽  
Xuan Ma ◽  
Weihua Wang ◽  
...  
Keyword(s):  
Clinical Practice ◽  
Reinforcement Learning ◽  
Transfer Learning ◽  
Classification Accuracy ◽  
Brain Machine Interface ◽  
Brain Machine Interfaces ◽  
Neural Data ◽  
Neural Recordings ◽  
New Knowledge ◽  
Machine Interface

Background: For the nonstationarity of neural recordings in intracortical brain–machine interfaces, daily retraining in a supervised manner is always required to maintain the performance of the decoder. This problem can be improved by using a reinforcement learning (RL) based self-recalibrating decoder. However, quickly exploring new knowledge while maintaining a good performance remains a challenge in RL-based decoders. Methods: To solve this problem, we proposed an attention-gated RL-based algorithm combining transfer learning, mini-batch, and weight updating schemes to accelerate the weight updating and avoid over-fitting. The proposed algorithm was tested on intracortical neural data recorded from two monkeys to decode their reaching positions and grasping gestures. Results: The decoding results showed that our proposed algorithm achieved an approximate 20% increase in classification accuracy compared to that obtained by the non-retrained classifier and even achieved better classification accuracy than the daily retraining classifier. Moreover, compared with a conventional RL method, our algorithm improved the accuracy by approximately 10% and the online weight updating speed by approximately 70 times. Conclusions: This paper proposed a self-recalibrating decoder which achieved a good and robust decoding performance with fast weight updating and might facilitate its application in wearable device and clinical practice.

A Multi-Step Neural Control for Motor Brain-Machine Interface by Reinforcement Learning

2013 ◽  
Vol 461 ◽  
pp. 565-569 ◽  
Author(s):  
Fang Wang ◽  
Kai Xu ◽  
Qiao Sheng Zhang ◽  
Yi Wen Wang ◽  
Xiao Xiang Zheng
Keyword(s):  
Reinforcement Learning ◽  
Neural Activity ◽  
Neural Control ◽  
Brain Machine Interface ◽  
Learning Approach ◽  
Complex Task ◽  
Neural Data ◽  
Neural Spikes ◽  
Performance Improvements ◽  
Machine Interface

Brain-machine interfaces (BMIs) decode cortical neural spikes of paralyzed patients to control external devices for the purpose of movement restoration. Neuroplasticity induced by conducting a relatively complex task within multistep, is helpful to performance improvements of BMI system. Reinforcement learning (RL) allows the BMI system to interact with the environment to learn the task adaptively without a teacher signal, which is more appropriate to the case for paralyzed patients. In this work, we proposed to apply Q(λ)-learning to multistep goal-directed tasks using users neural activity. Neural data were recorded from M1 of a monkey manipulating a joystick in a center-out task. Compared with a supervised learning approach, significant BMI control was achieved with correct directional decoding in 84.2% and 81% of the trials from naïve states. The results demonstrate that the BMI system was able to complete a task by interacting with the environment, indicating that RL-based methods have the potential to develop more natural BMI systems.

scholarly journals Towards Autonomous Intra-cortical Brain Machine Interfaces: Applying Bandit Algorithms for Online Reinforcement Learning

2020 ◽  
Author(s):  
Shoeb Shaikh ◽  
Rosa So ◽  
Tafadzwa Sibindi ◽  
Camilo Libedinsky ◽  
Arindam Basu
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
State Of The Art ◽  
Discrete Control ◽  
Brain Machine Interface ◽  
Control Task ◽  
Brain Machine Interfaces ◽  
Discrete State ◽  
Supervised Methods ◽  
Machine Interface

AbstractThis paper presents application of Banditron - an online reinforcement learning algorithm (RL) in a discrete state intra-cortical Brain Machine Interface (iBMI) setting. We have analyzed two datasets from non-human primates (NHPs) - NHP A and NHP B each performing a 4-option discrete control task over a total of 8 days. Results show average improvements of ≈ 15%, 6% in NHP A and 15%, 21% in NHP B over state of the art algorithms - Hebbian Reinforcement Learning (HRL) and Attention Gated Reinforcement Learning (AGREL) respectively. Apart from yielding a superior decoding performance, Banditron is also the most computationally friendly as it requires two orders of magnitude less multiply-and-accumulate operations than HRL and AGREL. Furthermore, Banditron provides average improvements of at least 40%, 15% in NHPs A, B respectively compared to popularly employed supervised methods - LDA, SVM across test days. These results pave the way towards an alternate paradigm of temporally robust hardware friendly reinforcement learning based iBMIs.

scholarly journals Toward an Autonomous Brain Machine Interface: Integrating Sensorimotor Reward Modulation and Reinforcement Learning

2015 ◽  
Vol 35 (19) ◽  
pp. 7374-7387 ◽  
Author(s):  
B. T. Marsh ◽  
V. S. A. Tarigoppula ◽  
C. Chen ◽  
J. T. Francis
Keyword(s):  
Reinforcement Learning ◽  
Brain Machine Interface ◽  
Machine Interface

Classification Accuracy of an Imagined-Movements Mental Task Set for Brain-Machine Interface

2008 ◽  
Author(s):  
Michael Lochinvar S. Abundo ◽  
Eliezer A. Marco ◽  
Gino Francisco R. Mempin ◽  
Marc Caesar R. Talampas ◽  
Luis S. Sison
Keyword(s):  
Classification Accuracy ◽  
Brain Machine Interface ◽  
Mental Task ◽  
Task Set ◽  
Machine Interface

scholarly journals The Reconnecting the Hand and Arm with Brain (ReHAB) Commentary on “An Integrated Brain-Machine Interface Platform With Thousands of Channels” (Preprint)

2019 ◽  
Author(s):  
Robert F Kirsch ◽  
A Bolu Ajiboye ◽  
Jonathan P Miller
Keyword(s):  
Neurological Disorders ◽  
Brain Machine Interface ◽  
Loss Of Function ◽  
Brain Machine Interfaces ◽  
Electrode Interface ◽  
Machine Interface ◽  
Almost All ◽  
Neuroprosthetic Devices ◽  
The Brain

UNSTRUCTURED Intracortical brain-machine interfaces are a promising technology for allowing people with chronic and severe neurological disorders that resulted in loss of function to potentially regain those functions through neuroprosthetic devices. The penetrating microelectrode arrays used in almost all previous studies of intracortical brain-machine interfaces in people had a limited recording life (potentially due to issues with long-term biocompatibility), as well as a limited number of recording electrodes with limited distribution in the brain. Significant advances are required in this array interface to deal with the issues of long-term biocompatibility and lack of distributed recordings. The Musk and Neuralink manuscript proposes a novel and potentially disruptive approach to advancing the brain-electrode interface technology, with the potential of addressing many of these hurdles. Our commentary addresses the potential advantages of the proposed approach, as well as the remaining challenges to be addressed.

Brain-Machine Interface Control via Reinforcement Learning

2007 ◽  
Author(s):  
Jack DiGiovanna ◽  
Babak Mahmoudi ◽  
Jeremiah Mitzelfelt ◽  
Justin C. Sanchez ◽  
Jose C. Principe
Keyword(s):  
Reinforcement Learning ◽  
Brain Machine Interface ◽  
Interface Control ◽  
Machine Interface
Sign in / Sign up

Export Citation Format

Share Document