Sensor Fusion of Leap Motion Controller and Flex Sensors using Kalman Filter for Human Finger Tracking

2017 ◽  
Vol 6 (3) ◽  
pp. 178
Author(s):  
Godwin Ponraj Joseph Vedhagiri ◽  
Hongliang Ren
Keyword(s):  
Sensor Fusion ◽  
Human Hand ◽  
Leap Motion ◽  
Human Beings ◽  
Motion Controller ◽  
Hand Motion ◽  
Health Care Design ◽  
Higher Dimensional ◽  
Human Finger ◽  
Multiple Domains

<span>In our daily life, we, human beings use our hands in various ways for most of our day-to-day activities. Tracking the position, orientation and articulation of human hands has a variety of applications including gesture recognition, robotics, medicine and health care, design and manufacturing, art and entertainment across multiple domains. However, it is an equally complex and challenging task due to several factors like higher dimensional data from hand motion, higher speed of operation, self-occlusion, etc. This paper puts forth a novel method for tracking the finger tips of human hand using two distinct sensors and combining their data by sensor fusion technique.</span>

A Motion Capture Realization Method for Assisting in Product Design via Parametric Surfaces

2019 ◽  
Author(s):  
Miri Weiss Cohen ◽  
Daniele Regazzoni
Keyword(s):  
Motion Capture ◽  
Feature Recognition ◽  
Relevant Information ◽  
Sensor Data ◽  
Human Hand ◽  
Leap Motion ◽  
Parametric Surfaces ◽  
Motion Controller ◽  
Leap Motion Controller ◽  
Design Characteristics

Abstract This work proposes a human computer interface system using a motion capture device, for assisting in CAD modeling and designing. The leap motion controller input data serves as an interactive tool which is transformed to surface representation of NURBS surfaces. Acquiring the sensor data is done by analyzing the images using a feature recognition module which in this work was updated and enhanced. Joints of hands and fingers are sufficed, and define recognized 3D image of a human hand. To use relevant information from the Leap Motion device, it is mandatory to interpret and map the input sensor data into 3D software coordinate system. This is done by implementing various transformations and a normalization procedures. Methods corresponding between these representation are developed in this work, to reduce noise thus providing accuracy. The DOF provided by the definition of the NURBS parametric surfaces and the Leap Motion Controller provide flexible design characteristics.

scholarly journals Hand Motion Capture from a 3D Leap Motion Controller for a Musculoskeletal Dynamic Simulation

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1199
Author(s):  
Robin Fonk ◽  
Sean Schneeweiss ◽  
Ulrich Simon ◽  
Lucas Engelhardt
Keyword(s):  
Motion Tracking ◽  
Inverse Dynamics ◽  
Hand Movement ◽  
Hand Tracking ◽  
Leap Motion ◽  
Motion Controller ◽  
Hand Motion ◽  
Motion Data ◽  
Leap Motion Controller ◽  
Inverse Dynamics Analysis

The AnyBody Modeling System™ (AMS) is a musculoskeletal software simulation solution using inverse dynamics analysis. It enables the determination of muscle and joint forces for a given bodily motion. The recording of the individual movement and the transfer into the AMS is a complex and protracted process. Researches indicated that the contactless, visual Leap Motion Controller (LMC) provides clinically meaningful motion data for hand tracking. Therefore, the aim of this study was to integrate the LMC hand motion data into the AMS in order to improve the process of recording a hand movement. A Python-based interface between the LMC and the AMS, termed ROSE Motion, was developed. This solution records and saves the data of the movement as Biovision Hierarchy (BVH) data and AnyScript vector files that are imported into the AMS simulation. Setting simulation parameters, initiating the calculation automatically, and fetching results is implemented by using the AnyPyTools library from AnyBody. The proposed tool offers a rapid and easy-to-use recording solution for elbow, hand, and finger movements. Features include animation, cutting/editing, exporting the motion, and remote controlling the AMS for the analysis and presentation of musculoskeletal simulation results. Comparing the motion tracking results with previous studies, covering problems when using the LMC limit the correctness of the motion data. However, fast experimental setup and intuitive and rapid motion data editing strengthen the use of marker less systems as the herein presented compared to marker based motion capturing.

scholarly journals Research Strategy: A Constructive Play for Anatomy Learning System Based on Human Finger Gestures on Holographic Display

2021 ◽  
Vol 5 (5) ◽  
pp. 120-128
Author(s):  
Ahmad Affandi Supli ◽  
Heng Yu Ping ◽  
Norma Liyana Binti Omar ◽  
Wong Yun Yi
Keyword(s):  
Development Stage ◽  
Research Strategy ◽  
Learning System ◽  
Advanced Technology ◽  
Human Anatomy ◽  
Leap Motion ◽  
Motion Controller ◽  
Holographic Display ◽  
Human Finger ◽  
Constructive Play

Human anatomy is a biology field that studies human body which consists of intricate and complex piece of engineering in which every assembly has an important role. This subject is considered to be very complex and thus need an advanced technology to help users learning this course more effectively. In this study, we propose and report our research strategy and progress to build a constructive play on human anatomy system based on finger motion gesture of Leap Motion controller (LMC). This LMC device can detect hand gestures and fingers’ motion and translate it into interaction input. Then, we utilize holographic display to portray our 3D human anatomy as its output. In detail, the research strategy of this paper consists of research plan, general framework and general architecture of the developed system. Then, we also present our current development of constructive anatomy learning system. In this future, we will discuss in more detail about the development stage.

scholarly journals Feasibility of interactive gesture control of a robotic microscope

2015 ◽  
Vol 1 (1) ◽  
pp. 164-167 ◽  
Author(s):  
Sven-Thomas Antoni ◽  
Christian Sonnenburg ◽  
Thore Saathoff ◽  
Alexander Schlaefer
Keyword(s):  
Near Field ◽  
Field Of View ◽  
Leap Motion ◽  
Motion Controller ◽  
Hand Motion ◽  
Clinical Scenarios ◽  
Gesture Control ◽  
Gesture Tracking ◽  
Robotic Devices ◽  
Reliable Interaction

AbstractRobotic devices become increasingly available in the clinics. One example are motorized surgical microscopes. While there are different scenarios on how to use the devices for autonomous tasks, simple and reliable interaction with the device is a key for acceptance by surgeons. We study, how gesture tracking can be integrated within the setup of a robotic microscope. In our setup, a Leap Motion Controller is used to track hand motion and adjust the field of view accordingly. We demonstrate with a survey that moving the field of view over a specified course is possible even for untrained subjects. Our results indicate that touch-less interaction with robots carrying small, near field gesture sensors is feasible and can be of use in clinical scenarios, where robotic devices are used in direct proximity of patient and physicians.

scholarly journals Feasibility and Performance Validation of a Leap Motion Controller for Upper Limb Rehabilitation

Robotics ◽  
2021 ◽  
Vol 10 (4) ◽  
pp. 130
Author(s):  
Marcus R. S. B. de Souza ◽  
Rogério S. Gonçalves ◽  
Giuseppe Carbone
Keyword(s):  
Upper Limb ◽  
Industrial Robot ◽  
Human Hand ◽  
Leap Motion ◽  
Tracking Accuracy ◽  
Motion Controller ◽  
Upper Limb Rehabilitation ◽  
Leap Motion Controller ◽  
Limb Rehabilitation ◽  
Accuracy And Repeatability

The leap motion controller is a commercial low-cost marker-less optical sensor that can track the motion of a human hand by recording various parameters. Upper limb rehabilitation therapy is the treatment of people having upper limb impairments, whose recovery is achieved through continuous motion exercises. However, the repetitive nature of these exercises can be interpreted as boring or discouraging while patient motivation plays a key role in their recovery. Thus, serious games have been widely used in therapies for motivating patients and making the therapeutic process more enjoyable. This paper explores the feasibility, accuracy, and repeatability of a leap motion controller (LMC) to be applied in combination with a serious game for upper limb rehabilitation. Experimental feasibility tests are carried out by using an industrial robot that replicates the upper limb motions and is tracked by using an LMC. The results suggest a satisfactory performance in terms of tracking accuracy although some limitations are identified and discussed in terms of measurable workspace.

scholarly journals A Systematic Review of EMG Applications for the Characterization of Forearm and Hand Muscle Activity during Activities of Daily Living: Results, Challenges, and Open Issues

Sensors ◽  
2021 ◽  
Vol 21 (9) ◽  
pp. 3035
Author(s):  
Néstor J. Jarque-Bou ◽  
Joaquín L. Sancho-Bru ◽  
Margarita Vergara
Keyword(s):  
Activities Of Daily Living ◽  
Musculoskeletal System ◽  
Muscle Activity ◽  
Daily Living ◽  
Muscle Activation ◽  
Human Hand ◽  
Hand Motion ◽  
Biomechanical Models ◽  
Rehabilitation Protocols ◽  
Open Issues

The role of the hand is crucial for the performance of activities of daily living, thereby ensuring a full and autonomous life. Its motion is controlled by a complex musculoskeletal system of approximately 38 muscles. Therefore, measuring and interpreting the muscle activation signals that drive hand motion is of great importance in many scientific domains, such as neuroscience, rehabilitation, physiotherapy, robotics, prosthetics, and biomechanics. Electromyography (EMG) can be used to carry out the neuromuscular characterization, but it is cumbersome because of the complexity of the musculoskeletal system of the forearm and hand. This paper reviews the main studies in which EMG has been applied to characterize the muscle activity of the forearm and hand during activities of daily living, with special attention to muscle synergies, which are thought to be used by the nervous system to simplify the control of the numerous muscles by actuating them in task-relevant subgroups. The state of the art of the current results are presented, which may help to guide and foster progress in many scientific domains. Furthermore, the most important challenges and open issues are identified in order to achieve a better understanding of human hand behavior, improve rehabilitation protocols, more intuitive control of prostheses, and more realistic biomechanical models.

scholarly journals Leap Motion Controller Video Game-Based Therapy for Upper Extremity Motor Recovery in Patients with Central Nervous System Diseases. A Systematic Review with Meta-Analysis

Sensors ◽  
2021 ◽  
Vol 21 (6) ◽  
pp. 2065
Author(s):  
Irene Cortés-Pérez ◽  
Noelia Zagalaz-Anula ◽  
Desirée Montoro-Cárdenas ◽  
Rafael Lomas-Vega ◽  
Esteban Obrero-Gaitán ◽  
...  
Keyword(s):  
Systematic Review ◽  
Central Nervous System ◽  
Nervous System ◽  
Motor Function ◽  
Video Game ◽  
Meta Analysis ◽  
Leap Motion ◽  
Motion Controller ◽  
Motor Dexterity ◽  
Quality Evidence

Leap Motion Controller (LMC) is a virtual reality device that can be used in the rehabilitation of central nervous system disease (CNSD) motor impairments. This review aimed to evaluate the effect of video game-based therapy with LMC on the recovery of upper extremity (UE) motor function in patients with CNSD. A systematic review with meta-analysis was performed in PubMed Medline, Web of Science, Scopus, CINAHL, and PEDro. We included five randomized controlled trials (RCTs) of patients with CNSD in which LMC was used as experimental therapy compared to conventional therapy (CT) to restore UE motor function. Pooled effects were estimated with Cohen’s standardized mean difference (SMD) and its 95% confidence interval (95% CI). At first, in patients with stroke, LMC showed low-quality evidence of a large effect on UE mobility (SMD = 0.96; 95% CI = 0.47, 1.45). In combination with CT, LMC showed very low-quality evidence of a large effect on UE mobility (SMD = 1.34; 95% CI = 0.49, 2.19) and the UE mobility-oriented task (SMD = 1.26; 95% CI = 0.42, 2.10). Second, in patients with non-acute CNSD (cerebral palsy, multiple sclerosis, and Parkinson’s disease), LMC showed low-quality evidence of a medium effect on grip strength (GS) (SMD = 0.47; 95% CI = 0.03, 0.90) and on gross motor dexterity (GMD) (SMD = 0.73; 95% CI = 0.28, 1.17) in the most affected UE. In combination with CT, LMC showed very low-quality evidence of a high effect in the most affected UE on GMD (SMD = 0.80; 95% CI = 0.06, 1.15) and fine motor dexterity (FMD) (SMD = 0.82; 95% CI = 0.07, 1.57). In stroke, LMC improved UE mobility and UE mobility-oriented tasks, and in non-acute CNSD, LMC improved the GS and GMD of the most affected UE and FMD when it was used with CT.

Sign in / Sign up

Export Citation Format

Share Document