Single Camera-Based Remote Physical Therapy: Verification on a Large Video Dataset

In recent years, several systems have been developed to capture human motion in real-time using common RGB cameras. This approach has great potential to become widespread among the general public as it allows the remote evaluation of exercise at no additional cost. The concept of using these systems in rehabilitation in the home environment has been discussed, but no work has addressed the practical problem of detecting basic body parts under different sensing conditions on a large scale. In this study, we evaluate the ability of the OpenPose pose estimation algorithm to perform keypoint detection of anatomical landmarks under different conditions. We infer the quality of detection based on the keypoint confidence values reported by the OpenPose. We used more than two thousand unique exercises for the evaluation. We focus on the influence of the camera view and the influence of the position of the trainees, which are essential in terms of the use for home exercise. Our results show that the position of the trainee has the greatest effect, in the following increasing order of suitability across all camera views: lying position, position on the knees, sitting position, and standing position. On the other hand, the effect of the camera view was only marginal, showing that the side view is having slightly worse results. The results might also indicate that the quality of detection of lower body joints is lower across all conditions than the quality of detection of upper body joints. In this practical overview, we present the possibilities and limitations of current camera-based systems in telerehabilitation.

Integrated Temperature and Position Sensors in a Shape-Memory Driven Soft Actuator for Closed-Loop Control

Soft actuators are a promising option for the advancing fields of human-machine interaction and dexterous robots in complex environments. Shape memory alloy wire actuators can be integrated into fiber rubber composites for highly deformable structures. For autonomous, closed-loop control of such systems, additional integrated sensors are necessary. In this work, a soft actuator is presented that incorporates fiber-based actuators and sensors to monitor both deformation and temperature. The soft actuator showed considerable deformation around two solid body joints, which was then compared to the sensor signals, and their correlation was analyzed. Both, the actuator as well as the sensor materials were processed by braiding and tailored fiber placement before molding with silicone rubber. Finally, the novel fiber-rubber composite material was used to implement closed-loop control of the actuator with a maximum error of 0.5°.

Bionic design and analysis of a multi-posture wheelchair

Posture transformation is an essential function for multi-posture wheelchairs. To improve the natural motion in posture transformation that is a popular problem in the design of multi-posture wheelchairs because the current wheelchair's posture transformation mechanism cannot remain consistent between the rotation center of the wheelchair and the rotation center of the human body joints. This paper proposes a sitting–standing–lying three-posture bionic transformation mechanism for a smart wheelchair. A human–wheelchair coupling model is described and analyzed according to the biomechanical characteristics of the posture transformation of human beings and their functional requirements. The configuration of the transformation mechanism is chosen by comparing the trails of the wheelchair rotation centers and the corresponding human joint rotation centers. The kinematics of the optimized configuration are discussed in detail to obtain the most bionic motion performance using the multivariable nonlinear constraint optimization algorithm. Finally, the mechanism is designed, and its posture transformation performance is simulated and verified using Adams (Automatic Dynamic Analysis of Mechanical Systems) software.

Automated and accurate assessment for postural abnormalities in patients with Parkinson’s disease based on Kinect and machine learning

Background Automated and accurate assessment for postural abnormalities is necessary to monitor the clinical progress of Parkinson’s disease (PD). The combination of depth camera and machine learning makes this purpose possible. Methods Kinect was used to collect the postural images from 70 PD patients. The collected images were processed to extract three-dimensional body joints, which were then converted to two-dimensional body joints to obtain eight quantified coronal and sagittal features (F1-F8) of the trunk. The decision tree classifier was carried out over a data set established by the collected features and the corresponding doctors’ MDS-UPDRS-III 3.13 (the 13th item of the third part of Movement Disorder Society-Sponsored Revision of the Unified Parkinson’s Disease Rating Scale) scores. An objective function was implanted to further improve the human–machine consistency. Results The automated grading of postural abnormalities for PD patients was realized with only six selected features. The intraclass correlation coefficient (ICC) between the machine’s and doctors’ score was 0.940 (95%CI, 0.905–0.962), meaning the machine was highly consistent with the doctors’ judgement. Besides, the decision tree classifier performed outstandingly, reaching 90.0% of accuracy, 95.7% of specificity and 89.1% of sensitivity in rating postural severity. Conclusions We developed an intelligent evaluation system to provide accurate and automated assessment of trunk postural abnormalities in PD patients. This study demonstrates the practicability of our proposed method in the clinical scenario to help making the medical decision about PD.

Tunable stiffness in fish robotics: mechanisms and advantages

One of the emerging themes of fish-inspired robotics is flexibility. Adding flexibility to the body, joints, or fins of fish-inspired robots can significantly improve thrust and/or efficiency during locomotion. However, the optimal stiffness depends on variables such as swimming speed, so there is no one “best” stiffness that maximizes efficiency in all conditions. Fish are thought to solve this problem by using muscular activity to tune their body and fin stiffness in real-time. Inspired by fish, some recent robots sport polymer actuators, adjustable leaf springs, or artificial tendons that tune stiffness mechanically. Models and water channel tests are providing a theoretical framework for stiffness-tuning strategies that devices can implement. The strategies can be thought of as analogous to car transmissions, which allow users to improve efficiency by tuning gear ratio with driving speed. We provide an overview of the latest discoveries about 1) the propulsive benefits of flexibility, particularly tunable flexibility, and 2) the mechanisms and strategies that fish and fish-inspired robots use to tune stiffness while swimming.

An Ergonomic Assessment of Different Postures and Children Risk during Evacuations

Crawling is recommended for avoiding high heat and toxic fumes and for obtaining more breathable air during evacuations. Few studies have evaluated the effects of crawling on physical joints and velocity, especially in children. Based on motion capture technology, this study proposes a novel method of using wearable sensors to collect exposure (e.g., mean duration, frequency) on children’s joints to objectively quantify the impacts of different locomotion methods on physical characteristics. An on-site experiment was conducted in a kindergarten with 28 children (13 boys and 15 girls) of different ages (4–6 years old) who traveled up to 22 m in three different postures: upright walking (UW), stoop walking (SW), and knee and hand crawling (KHC). The results showed that: (1) The level of joint fatigue for KHC was heavier than bipedal walking (p < 0.05), which was evidenced by higher mean duration and frequency. There was no significant difference between UW and SW (p > 0.05). (2) The physical characteristics of the children in the different postures observed in this study were different (p < 0.05). The ankle was more fatigued than other joints during bipedal walking. Unlike infants, the wrists and hips of the children became fatigued while crawling. The key actions flexion/extension are more likely to induce joint fatigue vs. other actions. (3) Crawling velocity was significantly slower than the bipedal velocities, and UW was 10.6% faster than SW (p < 0.05). The bipedal walking velocity started to decrease after the children had travelled up to 13 m, while the KHC velocity started to decrease after traveling up to 11.6 m. (4) In a severe fire, the adoption of SW is suggested, as the evacuees can both evacuate quickly and avoid overworking their joints. (5) There were no significant differences in the age (p > 0.05) and gender (p > 0.05) of the children on the joints in any of the three postures. To conclude, KHC causes more damage to body joints compared to bipedal walking, as evidenced by higher exposure (mean duration, frequency), whereas UW and SW are similar in terms of the level of joint fatigue. The above findings are expected to provide a useful reference for future applications in the children’s risk assessment and in the prevention design of buildings.

Recognition of Manual Welding Positions from Depth Hole Image Remotely Sensed by RGB-D Camera

The proportion of welding work in total man-hours required for shipbuilding processes has been perceived to be significant, and welding man-hours are greatly affected by working posture. Continuous research has been conducted to identify the posture in welding by utilizing the relationship between man-hours and working posture. However, the results that reflect the effect of the welding posture on man-hours are not available. Although studies on posture recognition based on depth image analysis are being positively reviewed, welding operation has difficulties in image interpretation because an external obstacle caused by arcs exists. Therefore, any obstacle element must be removed in advance. This study proposes a method to acquire work postures using a low-cost RGB-D camera and recognize the welding position through image analysis. It removes obstacles that appear as depth holes in the depth image and restores the removed part to the desired state. The welder’s body joints are extracted, and a convolution neural network is used to determine the corresponding welding position. The restored image showed significantly improved recognition accuracy. The proposed method acquires, analyzes, and automates the recognition of welding positions in real-time. It can be applied to all areas where image interpretation is difficult due to obstacles.

Path Analysis for Hybrid Rigid–Flexible Mechanisms

Hybrid rigid–flexible mechanisms are a type of compliant mechanism that combines rigid and flexible elements, being that their mobility is due to rigid-body joints and the relative flexibility of bendable rods. Two of the modeling methods of flexible rods are the Cosserat rod model and its simplification, the Kirchhoff rod model. Both of them present a system of differential equations that must be solved in conjunction with the boundary constraints of the rod, leading to a boundary value problem (BVP). In this work, two methods to solve this BVP are applied to analyze the influence of external loads in the movement of hybrid compliant mechanisms. First, a shooting method (SM) is used to integrate directly the shape of the flexible rod and the forces that appear in it. Then, an integration with elliptic integrals (EI) is carried out to solve the workspace of the compliant element, considering its buckling mode. Applying both methods, an algorithm that obtains the locus of all possible trajectories of the mechanism’s coupler point, and detects the buckling mode change, is developed. This algorithm also allows calculating all possible circuits of the mechanism. Thus, the performance of this method within the path analysis of mechanisms is demonstrated.

Multi-Level Graph Encoding with Structural-Collaborative Relation Learning for Skeleton-Based Person Re-Identification

Skeleton-based person re-identification (Re-ID) is an emerging open topic providing great value for safety-critical applications. Existing methods typically extract hand-crafted features or model skeleton dynamics from the trajectory of body joints, while they rarely explore valuable relation information contained in body structure or motion. To fully explore body relations, we construct graphs to model human skeletons from different levels, and for the first time propose a Multi-level Graph encoding approach with Structural-Collaborative Relation learning (MG-SCR) to encode discriminative graph features for person Re-ID. Specifically, considering that structurally-connected body components are highly correlated in a skeleton, we first propose a multi-head structural relation layer to learn different relations of neighbor body-component nodes in graphs, which helps aggregate key correlative features for effective node representations. Second, inspired by the fact that body-component collaboration in walking usually carries recognizable patterns, we propose a cross-level collaborative relation layer to infer collaboration between different level components, so as to capture more discriminative skeleton graph features. Finally, to enhance graph dynamics encoding, we propose a novel self-supervised sparse sequential prediction task for model pre-training, which facilitates encoding high-level graph semantics for person Re-ID. MG-SCR outperforms state-of-the-art skeleton-based methods, and it achieves superior performance to many multi-modal methods that utilize extra RGB or depth features. Our codes are available at https://github.com/Kali-Hac/MG-SCR.