Extra-Neural Signals From Severed Nerves Enable Intrinsic Hand Movements In Transhumeral Amputations

Robotic prostheses controlled by myoelectric signals can restore limited but important hand function in individuals with upper limb amputation. The lack of individual finger control highlights the yet insurmountable gap to fully replace a biologic hand. Implanted electrodes around severed nerves have been used to elicit sensations perceived as arising from the missing limb, but using such extra-neural electrodes to record motor signals that allow for the decoding of phantom movements has remained elusive. Here, we showed the feasibility of using signals from non-penetrating neural electrodes to decode intrinsic hand and finger movements in individuals with above-elbow amputations. We found that information recorded with extra-neural electrodes alone was enough to decode phantom hand and individual finger movements with high accuracy, and as expected, the addition of myoelectric signals reduced classification errors both in offline and in real-time decoding.

OpenSim Visualization of the Classification of Finger Movements Based on Electromyography Signal as the Single-Input Variable during Predefined Movements

A classifier is commonly generated for multifunctional prostheses control or also as input devices in human–computer interfaces. The complementary use of the open-access biomechanical simulation software, OpenSim, is demonstrated for the hand-movement classification performance visualization. A classifier was created from a previously captured database, which has 15 finger movements that were acquired during synchronized hand-movement repetitions with an 8-electrode sensor array placed on the forearm; a 92.89% recognition based on a complete movement was obtained. The OpenSim’s upper limb wrist model is employed, with movement in each of the joints of the hand–fingers. Several hand-motion visualizations were then generated, for the ideal hand movements, and for the best and the worst (53.03%) reproduction, to perceive the classification error in a specific task movement. This demonstrates the usefulness of this simulation tool before applying the classifier to a multifunctional prosthesis.

Mapping Surgeons Hand/Finger Movements to Surgical Tool Motion During Conventional Microsurgery Using Machine Learning

Purpose: Recent developments in robotics and artificial intelligence (AI) have led to significant advances in healthcare technologies enhancing robot-assisted minimally invasive surgery (RAMIS) in some surgical specialties. However, current human–robot interfaces lack intuitive teleoperation and cannot mimic surgeon’s hand/finger sensing required for fine motion micro-surgeries. These limitations make teleoperated robotic surgery not less suitable for, e.g. cardiac surgery and it can be difficult to learn for established surgeons. We report a pilot study showing an intuitive way of recording and mapping surgeon’s gross hand motion and the fine synergic motion during cardiac micro-surgery as a way to enhance future intuitive teleoperation. Methods: We set to develop a prototype system able to train a Deep Neural Network (DNN) by mapping wrist, hand and surgical tool real-time data acquisition (RTDA) inputs during mock-up heart micro-surgery procedures. The trained network was used to estimate the tools poses from refined hand joint angles. Outputs of the network were surgical tool orientation and jaw angle acquired by an optical motion capture system. Results: Based on surgeon’s feedback during mock micro-surgery, the developed wearable system with light-weight sensors for motion tracking did not interfere with the surgery and instrument handling. The wearable motion tracking system used 12 finger/thumb/wrist joint angle sensors to generate meaningful datasets representing inputs of the DNN network with new hand joint angles added as necessary based on comparing the estimated tool poses against measured tool pose. The DNN architecture was optimized for the highest estimation accuracy and the ability to determine the tool pose with the least mean squared error. This novel approach showed that the surgical instrument’s pose, an essential requirement for teleoperation, can be accurately estimated from recorded surgeon’s hand/finger movements with a mean squared error (MSE) less than 0.3%. Conclusion: We have developed a system to capture fine movements of the surgeon’s hand during micro-surgery that could enhance future remote teleoperation of similar surgical tools during micro-surgery. More work is needed to refine this approach and confirm its potential role in teleoperation.

An exploratory study to improve reading and comprehending mathematical expressions in braille

Braille readers read and comprehend mathematical expressions while moving their fingertips over braille characters. The aim of this exploratory study was to investigate the effect of an intervention that teaches braille readers who use a braille display to use finger movements with a focus on the expression’s mathematical structure. The finger movements involved movements where the two index fingers are about one or two braille cells apart and movements where the index fingers explore different parts of the expression. We investigated to what extent the intervention supports an interplay between finger movements and the expression’s mathematical structure to make the process of calculating the value of an expression easier and to make braille readers more aware of the expression’s structure. Three braille readers, respectively in Grades 7, 8, and 11, received the intervention consisting of five individual lessons. During the pre-, post-, and retention test, the braille readers’ finger movements were video recorded, as well as the time needed to read and process the mathematical tasks. Four expressions were selected for further analysis. The results show that during the posttest, each braille reader required at least 29% less time to read and process the expressions. The retention test results were even better. Scanpaths indicated that braille readers picked up features of mathematical structures more easily after the intervention. Based on our findings, we recommend that braille readers receive lessons in tactile reading strategies that support the reading and processing of mathematical expressions and equations.

Task Complexity and Image Clarity Facilitate Motor and Visuo-Motor Activities in Mirror Therapy in Post-stroke Patients

Introduction: Mirror therapy is effective in the recovery of upper-limb function among post-stroke patients. An important component of mirror therapy is imagining finger movements. This study aimed to determine the influence of finger movement complexity and mirror image clarity on facilitating motor and visuo-motor activities in post-stroke patients.Methods: Fifteen post-stroke patients and 18 right-handed healthy participants performed simple or complex finger tapping while viewing mirror images of these movements at varying levels of clarity. The physical setup was identical to typical mirror therapy. Functional near infrared spectroscopy (fNIRS) was used to capture the brain activities elicited in the bilateral primary motor cortices (M1) and the precuneus using a block experimental design.Results: In both study groups, the “complex finger-tapping task with blurred mirror image” condition resulted in lower intensity (p < 0.01) and authenticity (p < 0.01) of the kinesthetic mirror illusion, and higher levels of perceived effort in generating the illusion (p < 0.01), relative to the “simple finger-tapping with clear mirror image” condition. Greater changes in the oxygenated hemoglobin (HbO) concentration were recorded at the ipsilesional and ipsilateral M1 in the “complex finger-tapping task with blurred mirror image” condition relative to that recorded in the “simple finger-tapping task with clear mirror image” condition (p = 0.03). These HbO concentration changes were not significant in the precuneus. Post-stroke patients showed greater changes than their healthy counterparts at the ipsilesional M1 (F = 5.08; p = 0.03; partial eta squared = 0.14) and the precuneus (F = 7.71; p < 0.01; partial eta squared = 0.20).Conclusion: The complexity and image clarity of the finger movements increased the neural activities in the ipsilesional motor cortex in the post-stroke patients. These findings suggest plausible roles for top-down attention and working memory in the treatment effects of mirror therapy. Future research can aim to corroborate these findings by using a longitudinal design to examine the use of mirror therapy to promote upper limb motor recovery in post-stroke patients.

Flexible resistance-type strain sensors toward monitoring finger movements

Flexible strain sensors with superior flexibility and high sensitivity are critical to artificial intelligence. And it is favorable to develop highly sensitive strain sensors with simple and cost effective method. Here, we have prepared carbon nanotubes enhanced thermal polyurethane nanocomposites with good mechanical and electrical properties for fabrication of highly sensitive strain sensors. The nanomaterials have been prepared through simple but effective solvent evaporation method, and the cheap polyurethane has been utilized as main raw materials. Only a small quantity of carbon nanotubes with mass content of 5% has been doped into polyurethane matrix with purpose of enhancing mechanical and electrical properties of the nanocomposites. As a result, the flexible nanocomposite films present highly sensitive resistance response under external strain stimulus. The strain sensors based on these flexible composite films deliver excellent sensitivity and conformality under mechanical conditions, and detect finger movements precisely under different bending angles.

Real-Time Brain-Machine Interface Achieves High-Velocity Prosthetic Finger Movements using a Biologically-Inspired Neural Network Decoder

Despite the rapid progress and interest in brain-machine interfaces that restore motor function, the performance of prosthetic fingers and limbs has yet to mimic native function. The algorithm that converts brain signals to a control signal for the prosthetic device is one of the limitations in achieving rapid and realistic finger movements. To achieve more realistic finger movements, we developed a shallow feed-forward neural network, loosely inspired by the biological neural pathway, to decode real-time two-degree-of-freedom finger movements. Using a two-step training method, a recalibrated feedback intention–trained (ReFIT) neural network achieved a higher throughput with higher finger velocities and more natural appearing finger movements than the ReFIT Kalman filter, which represents the current standard. The neural network decoders introduced herein are the first to demonstrate real-time decoding of continuous movements at a level superior to the current state-of-the-art and could provide a starting point to using neural networks for the development of more naturalistic brain-controlled prostheses.