scholarly journals Robotic teleoperation systems using a wearable multimodal fusion device

2017 ◽  
Vol 14 (4) ◽  
pp. 172988141771705 ◽  
Author(s):  
Bin Fang ◽  
Fuchun Sun ◽  
Huaping Liu ◽  
Di Guo ◽  
Wendan Chen ◽  
...  
Keyword(s):  
Low Cost ◽  
Magnetic Measurement ◽  
Multimodal Fusion ◽  
Robotic Arm ◽  
Robotic Hand ◽  
Fusion Algorithm ◽  
Human Machine Interaction ◽  
Fusion Device ◽  
Teleoperation Systems ◽  
Robotic Teleoperation

Teleoperation is of great importance in the area of robotics especially when people’s presence at the robot working space is unavailable. It provides an alternative to employ human intelligence in the control of the robot remotely. We establish robotic teleoperation systems with a wearable multimodal fusion device. The device is integrated with 18 low-cost inertial and magnetic measurement units, which cover all segments of the arm and hand. The multimodal fusion algorithm based on extended Kalman filter is deduced to determine the orientations and positions of each segment. Then, the robotic teleoperation systems using the proposed device are designed. The novel teleoperation schemes can be applied for 11DOF robotic arm–hand system and 10DOF robotic arm–hand system, in which the operator’s fingers are used for robotic hand teleoperation, and the arms with palm are used for robotic arm teleoperation. Meanwhile, the proposed robotic teleoperation systems are fully realized with a user-friendly human–machine interaction interface. Finally, a series of experiments are conducted with our robotic teleoperation system successfully.

Integrated System of Mixed Virtual Reality Based on Data Glove CyberGlove II and Robotic Arm MechaTE Robot

2014 ◽  
Vol 611 ◽  
pp. 239-244 ◽  
Author(s):  
Juraj Kováč ◽  
František Ďurovský ◽  
Jozef Varga
Keyword(s):  
Virtual Reality ◽  
Mixed Reality ◽  
Low Cost ◽  
Integrated System ◽  
Robotic Arm ◽  
Robot Programming ◽  
Robotic Hand ◽  
Data Glove ◽  
Modeling Software ◽  
Mechanical Hands

Proposed paper describes development of CyberGlove II - MechaTE low-cost robotic hand interface intended for future use in virtual and mixed reality robot programming. The main goal is to explore possibilities and gain programing experience in controlling mechanical hands by means of data gloves and its interconnection to virtual reality modeling software. First part of paper describes recent progress in using virtual reality for purposes of intuitive robot programming; second part includes an overview of recent development of mechanical hands construction, as well as currently available data gloves. Last part provides details about CyberGlove – MechaTE interface and its potential for methods of intuitive robot programming in virtual or mixed reality environments.

scholarly journals A Deep Learning Approach for Robotic Arm Control using Brain-Computer Interface

2020 ◽  
Vol 14 ◽  
Keyword(s):  
Deep Learning ◽  
Real Time ◽  
Brain Computer Interface ◽  
Low Cost ◽  
Hand Movement ◽  
Substantial Improvement ◽  
Electrode System ◽  
Computer Interface ◽  
Robotic Arm ◽  
Robotic Hand

Brain-Computer Interface (BCI) is atechnology that enables a human to communicate with anexternal stratagem to achieve the desired result. This paperpresents a Motor Imagery (MI) – Electroencephalography(EEG) signal based robotic hand movements of lifting anddropping of an external robotic arm. The MI-EEG signalswere extracted using a 3-channel electrode system with theAD8232 amplifier. The electrodes were placed on threelocations, namely, C3, C4, and right mastoid. Signalprocessing methods namely, Butterworth filter and Sym-9Wavelet Packet Decomposition (WPD) were applied on theextracted EEG signals to de-noise the raw EEG signal.Statistical features like entropy, variance, standarddeviation, covariance, and spectral centroid were extractedfrom the de-noised signals. The statistical features werethen applied to train a Multi-Layer Perceptron (MLP) -Deep Neural Network (DNN) to classify the hand movementinto two classes; ‘No Hand Movement’ and ’HandMovement’. The resultant k-fold cross-validated accuracyachieved was 85.41% and other classification metrics, suchas precision, recall sensitivity, specificity, and F1 Score werealso calculated. The trained model was interfaced withArduino to move the robotic arm according to the classpredicted by the DNN model in a real-time environment.The proposed end to end low-cost deep learning frameworkprovides a substantial improvement in real-time BCI.

scholarly journals Multimodal Fusion Algorithm and Reinforcement Learning-Based Dialog System in Human-Machine Interaction

2020 ◽  
Vol 12 (4) ◽  
pp. 1016-1046
Author(s):  
Hanif Fakhrurroja ◽  
◽  
Carmadi Machbub ◽  
Ary Setijadi Prihatmanto ◽  
Ayu Purwarianti ◽  
...  
Keyword(s):  
Face Detection ◽  
User Satisfaction ◽  
System Development ◽  
Multimodal Fusion ◽  
Dialogue System ◽  
Fusion Algorithm ◽  
Human Machine Interaction ◽  
Interaction System ◽  
Dialog System ◽  
Machine Interaction

Studies on human-machine interaction system show positive results on system development accuracy. However, there are problems, especially using certain input modalities such as speech, gesture, face detection, and skeleton tracking. These problems include how to design an interface system for a machine to contextualize the existing conversations. Other problems include activating the system using various modalities, right multimodal fusion methods, machine understanding of human intentions, and methods for developing knowledge. This study developed a method of human-machine interaction system. It involved several stages, including a multimodal activation system, methods for recognizing speech modalities, gestures, face detection and skeleton tracking, multimodal fusion strategies, understanding human intent and Indonesian dialogue systems, as well as machine knowledge development methods and the right response. The research contributes to an easier and more natural humanmachine interaction system using multimodal fusion-based systems. The average accuracy rate of multimodal activation, testing dialogue system using Indonesian, gesture recognition interaction, and multimodal fusion is 87.42%, 92.11%, 93.54% and 93%, respectively. The level of user satisfaction towards the multimodal recognition-based human-machine interaction system developed was 95%. According to 76.2% of users, this interaction system was natural, while 79.4% agreed that the machine responded well to their wishes.

Low-Cost Data Glove for Robotic Hand-Forearm Teleoperation Systems Applications

2021 ◽  
Author(s):  
Lukas Gabriel Dias Gomes ◽  
Adriel Luiz Marques ◽  
Laura Ribeiro
Keyword(s):  
Low Cost ◽  
Cost Data ◽  
Robotic Hand ◽  
Data Glove ◽  
Teleoperation Systems

A novel data glove using inertial and magnetic sensors for motion capture and robotic arm-hand teleoperation

2017 ◽  
Vol 44 (2) ◽  
pp. 155-165 ◽  
Author(s):  
Bin Fang ◽  
Fuchun Sun ◽  
Huaping Liu ◽  
Di Guo
Keyword(s):  
Motion Capture ◽  
Low Cost ◽  
Magnetic Measurement ◽  
Calibration Method ◽  
Magnetic Sensors ◽  
Robotic Arm ◽  
Input Device ◽  
Content Type ◽  
Data Glove ◽  
Accuracy Of Measurements

Purpose The purpose of this paper is to present a novel data glove which can capture the motion of the arm and hand by inertial and magnetic sensors. The proposed data glove is used to provide the information of the gestures and teleoperate the robotic arm-hand. Design/methodology/approach The data glove comprises 18 low-cost inertial and magnetic measurement units (IMMUs) which not only make up the drawbacks of traditional data glove that only captures the incomplete gesture information but also provide a novel scheme of the robotic arm-hand teleoperation. The IMMUs are compact and small enough to wear on the upper arm, forearm, palm and fingers. The calibration method is proposed to improve the accuracy of measurements of units, and the orientations of each IMMU are estimated by a two-step optimal filter. The kinematic models of the arm, hand and fingers are integrated into the entire system to capture the motion gesture. A positon algorithm is also deduced to compute the positions of fingertips. With the proposed data glove, the robotic arm-hand can be teleoperated by the human arm, palm and fingers, thus establishing a novel robotic arm-hand teleoperation scheme. Findings Experimental results show that the proposed data glove can accurately and fully capture the fine gesture. Using the proposed data glove as the multiple input device has also proved to be a suitable teleoperating robotic arm-hand system. Originality/value Integrated with 18 low-cost and miniature IMMUs, the proposed data glove can give more information of the gesture than existing devices. Meanwhile, the proposed algorithms for motion capture determine the superior results. Furthermore, the accurately captured gestures can efficiently facilitate a novel teleoperation scheme to teleoperate the robotic arm-hand.

scholarly journals Development and Evaluation of a Low-Drift Inertial Sensor-Based System for Analysis of Alpine Skiing Performance

Sensors ◽  
2021 ◽  
Vol 21 (7) ◽  
pp. 2480
Author(s):  
Isidoro Ruiz-García ◽  
Ismael Navarro-Marchal ◽  
Javier Ocaña-Wilhelmi ◽  
Alberto J. Palma ◽  
Pablo J. Gómez-López ◽  
...  
Keyword(s):  
Data Fusion ◽  
Reference System ◽  
Inertial Sensor ◽  
Low Cost ◽  
Sampling Rate ◽  
Sensor Data ◽  
Fusion Algorithm ◽  
Improve Technique ◽  
Worst Case ◽  
Sensor Data Fusion

In skiing it is important to know how the skier accelerates and inclines the skis during the turn to avoid injuries and improve technique. The purpose of this pilot study with three participants was to develop and evaluate a compact, wireless, and low-cost system for detecting the inclination and acceleration of skis in the field based on inertial measurement units (IMU). To that end, a commercial IMU board was placed on each ski behind the skier boot. With the use of an attitude and heading reference system algorithm included in the sensor board, the orientation and attitude data of the skis were obtained (roll, pitch, and yaw) by IMU sensor data fusion. Results demonstrate that the proposed IMU-based system can provide reliable low-drifted data up to 11 min of continuous usage in the worst case. Inertial angle data from the IMU-based system were compared with the data collected by a video-based 3D-kinematic reference system to evaluate its operation in terms of data correlation and system performance. Correlation coefficients between 0.889 (roll) and 0.991 (yaw) were obtained. Mean biases from −1.13° (roll) to 0.44° (yaw) and 95% limits of agreements from 2.87° (yaw) to 6.27° (roll) were calculated for the 1-min trials. Although low mean biases were achieved, some limitations arose in the system precision for pitch and roll estimations that could be due to the low sampling rate allowed by the sensor data fusion algorithm and the initial zeroing of the gyroscope.

Cerebral Pulse Controlled Robotic Hand for Effective Paralytic Movements

2021 ◽  
Vol 9 (8) ◽  
pp. 2201-2204
Author(s):  
Abhay Patil
Keyword(s):  
Basic Assumption ◽  
Brain Computer Interface ◽  
Computer Interface ◽  
Robotic Arm ◽  
Human Hand ◽  
Robotic Hand ◽  
Handicapped People ◽  
Alpha Beta

Abstract: There are roughly 21 million handicapped people in India, which is comparable to 2.2% of the complete populace. These people are affected by various neuromuscular problems. To empower them to articulate their thoughts, one can supply them with elective and augmentative correspondence. For this, a Brain-Computer Interface framework (BCI) has been assembled to manage this specific need. The basic assumption of the venture reports the plan, working just as a testing impersonation of a man's arm which is intended to be powerfully just as kinematically exact. The conveyed gadget attempts to take after the movement of the human hand by investigating the signs delivered by cerebrum waves. The cerebrum waves are really detected by sensors in the Neurosky headset and produce alpha, beta, and gamma signals. Then, at that point, this sign is examined by the microcontroller and is then acquired onto the engineered hand by means of servo engines. A patient that experiences an amputee underneath the elbow can acquire from this specific biomechanical arm. Keywords: Brainwaves, Brain Computer Interface, Arduino, EEG sensor, Neurosky Mindwave Headset, Robotic arm

scholarly journals Preliminary System and Hardware Design for Guardian, a Low-cost Automatic Pill Dispenser with Pneumatic Technology

2018 ◽  
Author(s):  
Kaiwen Zhang
Keyword(s):  
Development Process ◽  
Low Cost ◽  
Hardware Design ◽  
Cylindrical Body ◽  
The Elderly ◽  
Robotic Arm ◽  
Core Technology ◽  
Pneumatic Technology

This paper presents on-going progress on Guardian, a low-cost automatic pill dispenser aimed to help the elderly community to take their medication on time. The device is composed of a cylindrical body with a pneumatically powered system and rotating robotic arm in the center column as its core technology. This information in the paper is meant to record the development process that led to the filing of a provisional patent USPTO 15964875 (Application Number).

Sign in / Sign up

Export Citation Format

Share Document