Experimental Evaluation on Haptic Feedback Accuracy by Using Two Self-Made Haptic Devices and One Additional Interface in Robotic Teleoperation

The goal of haptic feedback in robotic teleoperation is to enable users to accurately feel the interaction force measured at the slave side and precisely understand what is happening in the slave environment. The accuracy of the feedback force describing the error between the actual feedback force felt by a user at the master side and the measured interaction force at the slave side is the key performance indicator for haptic display in robotic teleoperation. In this paper, we evaluate the haptic feedback accuracy in robotic teleoperation via experimental method. A special interface iHandle and two haptic devices, iGrasp-T and iGrasp-R, designed for robotic teleoperation are developed for experimental evaluation. The device iHandle integrates a high-performance force sensor and a micro attitude and heading reference system which can be used to identify human upper limb motor abilities, such as posture maintenance and force application. When a user is asked to grasp the iHandle and maintain a fixed position and posture, the fluctuation value of hand posture is measured to be between 2 and 8 degrees. Based on the experimental results, human hand tremble as input noise sensed by the haptic device is found to be a major reason that results in the noise of output force from haptic device if the spring-damping model is used to render feedback force. Therefore, haptic rendering algorithms should be independent of hand motion information to avoid input noise from human hand to the haptic control loop in teleoperation. Moreover, the iHandle can be fixed at the end effector of haptic devices; iGrasp-T or iGrasp-R, to measure the output force/torque from iGrasp-T or iGrasp-Rand to the user. Experimental results show that the accuracy of the output force from haptic device iGrasp-T is approximately 0.92 N, and using the force sensor in the iHandle can compensate for the output force inaccuracy of device iGrasp-T to 0.1 N. Using a force sensor as the feedback link to form a closed-loop feedback force control system is an effective way to improve the accuracy of feedback force and guarantee high-fidelity of feedback forces at the master side in robotic teleoperation.

Flexible Strain-Sensitive Silicone-CNT Sensor for Human Motion Detection

This article describes the manufacturing technology of biocompatible flexible strain-sensitive sensor based on Ecoflex silicone and multi-walled carbon nanotubes (MWCNT). The sensor demonstrates resistive behavior. Structural, electrical, and mechanical characteristics are compared. It is shown that laser radiation significantly reduces the resistance of the material. Through laser radiation, electrically conductive networks of MWCNT are formed in a silicone matrix. The developed sensor demonstrates highly sensitive characteristics: gauge factor at 100% elongation −4.9, gauge factor at 90° bending −0.9%/deg, stretchability up to 725%, tensile strength 0.7 MPa, modulus of elasticity at 100% 46 kPa, and the temperature coefficient of resistance in the range of 30–40 °С is −2 × 10−3. There is a linear sensor response (with 1 ms response time) with a low hysteresis of ≤3%. An electronic unit for reading and processing sensor signals based on the ATXMEGA8E5-AU microcontroller has been developed. The unit was set to operate the sensor in the range of electrical resistance 5–150 kOhm. The Bluetooth module made it possible to transfer the received data to a personal computer. Currently, in the field of wearable technologies and health monitoring, a vital need is the development of flexible sensors attached to the human body to track various indicators. By integrating the sensor with the joints of the human hand, effective movement sensing has been demonstrated.

EEG characteristic investigation of the sixth-finger motor imagery and optimal channel selection for classification

Objective. Supernumerary Robotic Limbs (SRL) are body augmentation robotic devices by adding extra limbs or fingers to the human body different from the traditional wearable robotic devices such as prosthesis and exoskeleton. We proposed a novel MI (Motor imagery)-based BCI paradigm based on the sixth-finger which imagines controlling the extra finger movements. The goal of this work is to investigate the EEG characteristics and the application potential of MI-based BCI systems based on the new imagination paradigm (the sixth finger MI). Approach. 14 subjects participated in the experiment involving the sixth finger MI tasks and rest state. Event-related spectral perturbation (ERSP) was adopted to analyse EEG spatial features and key-channel time-frequency features. Common spatial patterns (CSP) were used for feature extraction and classification was implemented by support vector machine (SVM). A genetic algorithm (GA) was used to select combinations of EEG channels that maximized classification accuracy and verified EEG patterns based on the sixth finger MI. And we conducted a longitudinal 4-week EEG control experiment based on the new paradigm. Main results. ERD (event-related desynchronization) was found in the supplementary motor area (SMA) and primary motor area (M1) with a faint contralateral dominance. Unlike traditional MI based on the human hand, ERD was also found in frontal lobe. GA results showed that the distribution of the optimal 8-channel is similar to EEG topographical distributions, nearing parietal and frontal lobe. And the classification accuracy based on the optimal 8-channel (the highest accuracy of 80% and mean accuracy of 70%) was significantly better than that based on the random 8-channel (p<0.01). Significance. This work provided a new paradigm for MI-based MI system and verified its feasibility, widened the control bandwidth of the BCI system.

Identify Finger Rotation Angles with ArUco Markers and Action Cameras

Measuring the motions of human hand joints is often a challenge due to the high number of degrees of freedom. In this study, we proposed a hand tracking system utilizing action cameras and ArUco markers to continuously measure the rotation angles of hand joints. Three methods were developed to estimate the joint rotation angles. The pos-based method transforms marker positions to a reference coordinate system (RCS) and extracts a hand skeleton to identify the rotation angles. Similarly, the orient-x-based method calculates the rotation angles from the transformed x-orientations of the detected markers in the RCS. In contrast, the orient-mat-based method first identifies the rotation angles in each camera coordinate system using the detected orientations, and then, synthesizes the results regarding each joint. Experiment results indicated that the repeatability errors with one camera regarding different marker sizes were around 2.64 to 27.56 degrees and 0.60 to 2.36 degrees using the marker positions and orientations respectively. When multiple cameras were employed to measure the joint rotation angles, the angles measured by using the three methods were comparable with that measured by a goniometer. Despite larger deviations occurred when using the pos-based method. Further analysis indicated that the results of using the orient-mat-based method can describe more types of joint rotations, and the effectiveness of this method was verified by capturing hand movements of several participants. Thus it is recommended for measuring joint rotation angles in practical setups.

The Precision of the Human Hand: Variability in Pinch Strength and Manual Dexterity

Changes in hand morphology throughout human evolution have facilitated the use of forceful pad-to-pad precision grips, contributing to the development of fine motor movement and dexterous manipulation typical of modern humans. Today, variation in human hand function may be affected by demographic and/or lifestyle factors, but these remain largely unexplored. We measured pinch grip strength and dexterity in a heterogeneous cross-sectional sample of human participants (n = 556) to test for the potential effects of sex, age, hand asymmetries, hand morphology, and frequently practiced manual activities across the lifespan. We found a significant effect of sex on pinch strength, dexterity, and different directional asymmetries, with the practice of manual musical instruments, significantly increasing female dexterity for both hands. Males and females with wider hands were also stronger, but not more precise, than those with longer hands, while the thumb-index ratio had no effect. Hand dominance asymmetry further had a significant effect on dexterity but not on pinch strength. These results indicate that different patterns of hand asymmetries and hand function are influenced in part by life experiences, improving our understanding of the link between hand form and function and offering a referential context for interpreting the evolution of human dexterity.

Dimension investigation to pneumatic network bending soft actuators for soft robotic applications

Soft actuators have been investigated for robots that interact with people, such as industrial robots, entertainment robots, and medical robots. Although soft actuators have been utilised in several applications, design tools that can assist in the effective and systematic design of actuators are needed. This paper focused on the most common soft actuators for bending motion, the Pneumatic Networks bending actuators. Paper presented a survey on the effects of changing the dimensions on the soft actuator and its cross-section shape on the soft actuator flexibility and the forces generated at different applied pressures. This survey can be used to optimize the dimension ratio for the soft actuator and the cross-section shape. Furthermore, this paper analyzed the possible reasons for the dimension change effect. The performance of the bending soft actuator was evaluated using ABAQUS/CAE software simulation models to provide quantitative insights into the actuators' behaviours. Thus, this paper provided a lot of insights that can be used to guide and accelerate the soft actuator design process to create strong and flexible Pneumatic Networks bending actuators. Using the paper insights outputs, a soft gripper was designed that can grasp many complex objects without needing any modification in the gripper shape. To show the proposed actuators' capacity to do complicated movements and expand their applications, a completely soft hand was created that can mimic the mobility of the human hand as nearly as possible, and this ability was verified using hand sign language settings.

The Influence of the Abduction Joints of Four Fingers to Grasp: Experimental and Simulated Verification

The thumb is the most important finger of the human hand and has a great influence on grasp manipulations. However, the extent to which joints other than the thumb joints affect the grasp, and thus, which joints should be included in a prosthetic hand, remains an open issue. In this paper, we focus on the metacarpophalangeal joints of the four fingers, except the thumb, which can generate flexion/extension and abduction/adduction motions. The contribution of these joints to grasping was evaluated in four aspects: grasp size, grasp force, grasp quality and grasp success rate. Six subjects participated in experiments with respect to the maximum grasp size and grasp force. The results show that possessing abduction mobility of the metacarpophalangeal joints can increase the grasp size by 4.67 ± 1.93 mm and the grasp force by 5.27 ± 4.25 N. Then, the grasping quality and success rate were tested in a simulation platform and using a robotic hand, respectively. The results show that grasp quality was promoted by 76.7% in the simulated environment with abduction mobility compared to without abduction mobility, whereas the grasp success rate was promoted by 68.3%. We believe that the results of this work can benefit the understanding of hand function and prosthetic hand design.

Biorealistic Control of Hand Prosthesis Augments Functional Performance of Individuals With Amputation

The human hand has compliant properties arising from muscle biomechanics and neural reflexes, which are absent in conventional prosthetic hands. We recently proved the feasibility to restore neuromuscular reflex control (NRC) to prosthetic hands using real-time computing neuromorphic chips. Here we show that restored NRC augments the ability of individuals with forearm amputation to complete grasping tasks, including standard Box and Blocks Test (BBT), Golf Balls Test (GBT), and Potato Chips Test (PCT). The latter two were more challenging, but novel to prosthesis tests. Performance of a biorealistic controller (BC) with restored NRC was compared to that of a proportional linear feedback (PLF) controller. Eleven individuals with forearm amputation were divided into two groups: one with experience of myocontrol of a prosthetic hand and another without any. Controller performances were evaluated by success rate, failure (drop/break) rate in each grasping task. In controller property tests, biorealistic control achieved a better compliant property with a 23.2% wider range of stiffness adjustment than that of PLF control. In functional grasping tests, participants could control prosthetic hands more rapidly and steadily with neuromuscular reflex. For participants with myocontrol experience, biorealistic control yielded 20.4, 39.4, and 195.2% improvements in BBT, GBT, and PCT, respectively, compared to PLF control. Interestingly, greater improvements were achieved by participants without any myocontrol experience for BBT, GBT, and PCT at 27.4, 48.9, and 344.3%, respectively. The functional gain of biorealistic control over conventional control was more dramatic in more difficult grasp tasks of GBT and PCT, demonstrating the advantage of NRC. Results support the hypothesis that restoring neuromuscular reflex in hand prosthesis can improve neural motor compatibility to human sensorimotor system, hence enabling individuals with amputation to perform delicate grasps that are not tested with conventional prosthetic hands.

A Human Gesture Mapping Method to Control a Multi‐Functional Hand for Robot‐Assisted Laparoscopic Surgery: The MUSHA Case

This work presents a novel technique to control multi-functional hand for robot-assisted laparoscopic surgery. We tested the technique using the MUSHA multi-functional hand, a robot-aided minimally invasive surgery tool with more degrees of freedom than the standard commercial end-effector of the da Vinci robot. Extra degrees of freedom require the development of a proper control strategy to guarantee high performance and avoid an increasing complexity of control consoles. However, developing reliable control algorithms while reducing the control side’s mechanical complexity is still an open challenge. In the proposed solution, we present a control strategy that projects the human hand motions into the robot actuation space. The human hand motions are tracked by a LeapMotion camera and mapped into the actuation space of the virtualized end-effector. The effectiveness of the proposed method was evaluated in a twofold manner. Firstly, we verified the Lyapunov stability of the algorithm, then an user study with 10 subjects assessed the intuitiveness and usability of the system.