Wearable Multimodal - Serious Game System For Hand and Cognitive Rehabilitation After Stroke

Objective: Stroke often leads to both motor control and cognitive dysfunction, and effective rehabilitation requires keeping patients engaged and motivated. We introduce a wearable multimodal system based on force myography, electromyography, and inertial sensing with two associated serious games for stroke rehabilitation of twelve hand movements related to activities of daily living and the Fugl Meyer Assessment.Methods: In the ‘Find the Sheep’ serious game, patients performed corresponding hand movements to select the correct sheep card, and in the ‘Best Salesman’ serious game, patients performed corresponding hand movements to grab specific food and drink items in a store. A multi-sensor fusion model was developed for movement classification via linear discriminant analysis. Ten stroke patients with mild to moderate motor impairments (Brunnstrom Stage for Hand II-VI) performed validation testing, and effectiveness was evaluated by movement classification accuracy and qualitative patient questionnaires.Results: Classification accuracy for twelve movements using combined force myography, electromyography, and inertial sensing was 81.0%, and accuracies for using electromyography, force myography, or inertial sensing alone were 69.6%, 63.2%, and 47.8%, respectively. All patients reported that they were more enthusiastic about rehabilitation while playing serious games than conventional rehabilitation, and a majority reported the wearable multimodal-based system was easier to wear than a sensorized data glove. Significance: Results showed that multi-sensor fusion could improve hand gesture classification accuracy for stroke patients and demonstrated that the proposed wearable multimodal-serious game system could potentially facilitate upper extremity rehabilitation and cognitive training after stroke.

Inertial and viscous flywheel sensing of nanoparticles

Rotational dynamics often challenge physical intuition while enabling unique realizations, from the rotor of a gyroscope that maintains its orientation regardless of the outer gimbals, to a tennis racket that rotates around its handle when tossed face-up in the air. In the context of inertial sensing, which can measure mass with atomic precision, rotational dynamics are normally considered a complication hindering measurement interpretation. Here, we exploit the rotational dynamics of a microfluidic device to develop a modality in inertial sensing. Combining theory with experiments, we show that this modality measures the volume of a rigid particle while normally being insensitive to its density. Paradoxically, particle density only emerges when fluid viscosity becomes dominant over inertia. We explain this paradox via a viscosity-driven, hydrodynamic coupling between the fluid and the particle that activates the rotational inertia of the particle, converting it into a ‘viscous flywheel’. This modality now enables the simultaneous measurement of particle volume and mass in fluid, using a single, high-throughput measurement.

Industrial robot programming by demonstration using stereoscopic vision and inertial sensing

Purpose The purpose of this paper is to present a programming by demonstration (PbD) system based on 3D stereoscopic vision and inertial sensing that provides a cost-effective pose tracking system, even during error-prone situations, such as camera occlusions. Design/methodology/approach The proposed PbD system is based on the 6D Mimic innovative solution, whose six degrees of freedom marker hardware had to be revised and restructured to accommodate an IMU sensor. Additionally, a new software pipeline was designed to include this new sensing device, seeking the improvement of the overall system’s robustness in stereoscopic vision occlusion situations. Findings The IMU component and the new software pipeline allow the 6D Mimic system to successfully maintain the pose tracking when the main tracking tool, i.e. the stereoscopic vision, fails. Therefore, the system improves in terms of reliability, robustness, and accuracy which were verified by real experiments. Practical implications Based on this proposal, the 6D Mimic system reaches a reliable and low-cost PbD methodology. Therefore, the robot can accurately replicate, on an industrial scale, the artisan level performance of highly skilled shop-floor operators. Originality/value To the best of the authors’ knowledge, the sensor fusion between stereoscopic images and IMU applied to robot PbD is a novel approach. The system is entirely designed aiming to reduce costs and taking advantage of an offline processing step for data analysis, filtering and fusion, enhancing the reliability of the PbD system.