Classification of Individual Finger Movements from Right Hand Using fNIRS Signals

Functional near-infrared spectroscopy (fNIRS) is a comparatively new noninvasive, portable, and easy-to-use brain imaging modality. However, complicated dexterous tasks such as individual finger-tapping, particularly using one hand, have been not investigated using fNIRS technology. Twenty-four healthy volunteers participated in the individual finger-tapping experiment. Data were acquired from the motor cortex using sixteen sources and sixteen detectors. In this preliminary study, we applied standard fNIRS data processing pipeline, i.e. optical densities conversation, signal processing, feature extraction, and classification algorithm implementation. Physiological and non-physiological noise is removed using 4th order band-pass Butter-worth and 3rd order Savitzky–Golay filters. Eight spatial statistical features were selected: signal-mean, peak, minimum, Skewness, Kurtosis, variance, median, and peak-to-peak form data of oxygenated haemoglobin changes. Sophisticated machine learning algorithms were applied, such as support vector machine (SVM), random forests (RF), decision trees (DT), AdaBoost, quadratic discriminant analysis (QDA), Artificial neural networks (ANN), k-nearest neighbors (kNN), and extreme gradient boosting (XGBoost). The average classification accuracies achieved were 0.75±0.04, 0.75±0.05, and 0.77±0.06 using k-nearest neighbors (kNN), Random forest (RF) and XGBoost, respectively. KNN, RF and XGBoost classifiers performed exceptionally well on such a high-class problem. The results need to be further investigated. In the future, a more in-depth analysis of the signal in both temporal and spatial domains will be conducted to investigate the underlying facts. The accuracies achieved are promising results and could open up a new research direction leading to enrichment of control commands generation for fNIRS-based brain-computer interface applications.

Extra-Neural Signals From Severed Nerves Enable Intrinsic Hand Movements In Transhumeral Amputations

Robotic prostheses controlled by myoelectric signals can restore limited but important hand function in individuals with upper limb amputation. The lack of individual finger control highlights the yet insurmountable gap to fully replace a biologic hand. Implanted electrodes around severed nerves have been used to elicit sensations perceived as arising from the missing limb, but using such extra-neural electrodes to record motor signals that allow for the decoding of phantom movements has remained elusive. Here, we showed the feasibility of using signals from non-penetrating neural electrodes to decode intrinsic hand and finger movements in individuals with above-elbow amputations. We found that information recorded with extra-neural electrodes alone was enough to decode phantom hand and individual finger movements with high accuracy, and as expected, the addition of myoelectric signals reduced classification errors both in offline and in real-time decoding.

Decoding Neural Activity in Sulcal and White Matter Areas of the Brain to Accurately Predict Individual Finger Movement and Tactile Stimuli of the Human Hand

Millions of people worldwide suffer motor or sensory impairment due to stroke, spinal cord injury, multiple sclerosis, traumatic brain injury, diabetes, and motor neuron diseases such as ALS (amyotrophic lateral sclerosis). A brain-computer interface (BCI), which links the brain directly to a computer, offers a new way to study the brain and potentially restore impairments in patients living with these debilitating conditions. One of the challenges currently facing BCI technology, however, is to minimize surgical risk while maintaining efficacy. Minimally invasive techniques, such as stereoelectroencephalography (SEEG) have become more widely used in clinical applications in epilepsy patients since they can lead to fewer complications. SEEG depth electrodes also give access to sulcal and white matter areas of the brain but have not been widely studied in brain-computer interfaces. Here we show the first demonstration of decoding sulcal and subcortical activity related to both movement and tactile sensation in the human hand. Furthermore, we have compared decoding performance in SEEG-based depth recordings versus those obtained with electrocorticography electrodes (ECoG) placed on gyri. Initial poor decoding performance and the observation that most neural modulation patterns varied in amplitude trial-to-trial and were transient (significantly shorter than the sustained finger movements studied), led to the development of a feature selection method based on a repeatability metric using temporal correlation. An algorithm based on temporal correlation was developed to isolate features that consistently repeated (required for accurate decoding) and possessed information content related to movement or touch-related stimuli. We subsequently used these features, along with deep learning methods, to automatically classify various motor and sensory events for individual fingers with high accuracy. Repeating features were found in sulcal, gyral, and white matter areas and were predominantly phasic or phasic-tonic across a wide frequency range for both HD (high density) ECoG and SEEG recordings. These findings motivated the use of long short-term memory (LSTM) recurrent neural networks (RNNs) which are well-suited to handling transient input features. Combining temporal correlation-based feature selection with LSTM yielded decoding accuracies of up to 92.04 ± 1.51% for hand movements, up to 91.69 ± 0.49% for individual finger movements, and up to 83.49 ± 0.72% for focal tactile stimuli to individual finger pads while using a relatively small number of SEEG electrodes. These findings may lead to a new class of minimally invasive brain-computer interface systems in the future, increasing its applicability to a wide variety of conditions.

Identifying and Decoding Temporally Correlated Neural Patterns in the Brain to Accurately Predict Individual Finger Movement and Tactile Stimuli of the Human Hand Using Stereoelectroencephalography

Millions of people worldwide suffer motor or sensory impairment due to stroke, spinal cord injury, multiple sclerosis, traumatic brain injury, diabetes, and motor neuron diseases such as ALS (amyotrophic lateral sclerosis). A brain-computer interface (BCI), which links the brain directly to a computer, offers a new way to study the brain and potentially restore function in patients living with debilitating conditions. One of the challenges currently facing BCI technology, however, is how to minimize surgical risk. Minimally invasive techniques, such as stereoelectroencephalography (SEEG) have become more widely used in clinical applications since they can lead to fewer complications. SEEG electrodes also give access to sulcal and white matter areas of the brain but have not been widely studied in brain-computer interfaces. We therefore investigated the viability of using SEEG electrodes in a BCI for recording and decoding neural signals related to movement and the sense of touch and compared its performance to electrocorticography electrodes (ECoG) placed on gyri. Initial poor decoding performance and the observation that most neural modulation patterns were variable trial-to-trial and transient (significantly shorter than the sustained finger movements studied), led to the development of a feature selection method based on temporal autocorrelation, a repeatability metric. An algorithm based on temporal autocorrelation was developed to isolate features that consistently repeated (required for accurate decoding) and possessed information content related to movement or touch-related stimuli. We subsequently used these features, along with deep learning methods, to automatically classify various motor and sensory events for individual fingers with high accuracy. Repeating features were found in sulcal, gyral, and white matter areas and were predominantly phasic or phasic-tonic across a wide frequency range for both HD (high density) ECoG and SEEG recordings. These findings motivated the use of long short-term memory (LSTM) recurrent neural networks (RNNs) which are well-suited to handling both transient and sustained input features. Combining temporal autocorrelation-based feature selection with LSTM yielded decoding accuracies of up to 92.04 +/- 1.51% for hand movements, up to 91.69 +/- 0.49% for individual finger movements, and up to 80.64 +/- 1.64% for focal tactile stimuli to the finger pads and palm while using a relatively small number of SEEG electrodes. These findings may lead to a new class of minimally invasive brain-computer interface systems in the future, increasing its applicability to a wider variety of conditions.

Differential neural plasticity of individual fingers revealed by fMRI neurofeedback

Previous work has shown that fMRI activity patterns associated with individual fingers can be shifted by temporary impairment of the hand. Here, we investigated whether these neural activity patterns could be modulated endogenously and whether any behavioral changes result from this modulation. We used decoded neurofeedback in healthy individuals to encourage participants to shift the neural activity pattern in sensorimotor cortex of the middle finger towards the index finger, and the ring finger towards the little finger. We first mapped the neural activity patterns for all fingers of the right hand in an fMRI pattern localizer session. Then, in three subsequent neurofeedback sessions, participants were rewarded after middle/ring finger presses according to their activity pattern overlap during each trial. A force-sensitive keyboard was used to ensure that participants were not altering their physical finger coordination patterns. We found evidence that participants could learn to shift the activity pattern of the ring finger but not of the middle finger. Increased variability of these activity patterns during the localizer session was associated with the ability of participants to modulate them using neurofeedback. Participants also showed an increased preference for the ring finger but not for the middle finger in a post-neurofeedback motor task. Our results show that neural activity and behaviors associated with the ring finger are more readily modulated than those associated with the middle finger. These results have broader implications for rehabilitation of individual finger movements, which may be limited or enhanced by individual finger plasticity after neurological injury.

Towards a common template for neural reinforcement of finger individuation

The inability to individuate finger movements is a common impairment following stroke. Conventional physical therapy ignores underlying neural changes with recovery, leaving it unclear why sensorimotor function often remains impaired. Functional MRI neurofeedback can monitor neural activity and reinforce it towards a healthy template to restore function. However, identifying an individualized training template may not be possible depending on the severity of impairment. In this study, we investigated the use of functional alignment of brain data across healthy participants to create an idealized neural template to be used as a training target for new participants. We employed multi-voxel pattern analyses to assess the prediction accuracy and robustness to missing data of pre-trained functional templates corresponding to individual finger presses. We found a significant improvement in classification accuracy (p < 0.001) of individual finger presses when group data was aligned based on function (88%) rather than anatomy (46%). Importantly, we found no significant drop in performance when aligning a new participant to a pre-established template as compared to including this new participant in the creation of a new template. These results indicate that functionally aligned templates could provide an effective surrogate training target for patients following neurological injury.

Real-Time Linear Prediction of Simultaneous and Independent Movements of Two Finger Groups Using an Intracortical Brain-Machine Interface

SUMMARYModern brain-machine interfaces can return function to people with paralysis, but current hand neural prostheses are unable to reproduce control of individuated finger movements. Here, for the first time, we present a real-time, high-speed, linear brain-machine interface in nonhuman primates that utilizes intracortical neural signals to bridge this gap. We created a novel task that systematically individuates two finger groups, the index finger and the middle-ring-small fingers combined, presenting separate targets for each group. During online brain control, the ReFIT Kalman filter demonstrated the capability of individuating movements of each finger group with high performance, enabling a nonhuman primate to acquire two targets simultaneously at 1.95 targets per second, resulting in an average information throughput of 2.1 bits per second. To understand this result, we performed single unit tuning analyses. Cortical neurons were active for movements of an individual finger group, combined movements of both finger groups, or both. Linear combinations of neural activity representing individual finger group movements predicted the neural activity during combined finger group movements with high accuracy, and vice versa. Hence, a linear model was able to explain how cortical neurons encode information about multiple dimensions of movement simultaneously. Additionally, training ridge regressing decoders with independent component movements was sufficient to predict untrained higher-complexity movements. Our results suggest that linear decoders for brain-machine interfaces may be sufficient to execute high-dimensional tasks with the performance levels required for naturalistic neural prostheses.