In-situ identification and recognition of multi-hand gestures using optimized deep residual network

2021 ◽  
pp. 1-15
Author(s):  
S. Rubin Bose ◽  
V. Sathiesh Kumar
Keyword(s):  
Real Time ◽  
Action Recognition ◽  
Performance Metrics ◽  
Class Imbalance ◽  
Residual Network ◽  
Hand Detection ◽  
Hand Gestures ◽  
Human Machine Interaction ◽  
The Real ◽  
Hand Action

The real-time perception of hand gestures in a deprived environment is a demanding machine vision task. The hand recognition operations are more strenuous with different illumination conditions and varying backgrounds. Robust recognition and classification are the vital steps to support effective human-machine interaction (HMI), virtual reality, etc. In this paper, the real-time hand action recognition is performed by using an optimized Deep Residual Network model. It incorporates a RetinaNet model for hand detection and a Depthwise Separable Convolutional (DSC) layer for precise hand gesture recognition. The proposed model overcomes the class imbalance problems encountered by the conventional single-stage hand action recognition algorithms. The integrated DSC layer reduces the computational parameters and enhances the recognition speed. The model utilizes a ResNet-101 CNN architecture as a Feature extractor. The model is trained and evaluated on the MITI-HD dataset and compared with the benchmark datasets (NUSHP-II, Senz-3D). The network achieved a higher Precision and Recall value for an IoU value of 0.5. It is realized that the RetinaNet-DSC model using ResNet-101 backbone network obtained higher Precision (99.21 %for AP0.5, 96.80%for AP0.75) for MITI-HD Dataset. Higher performance metrics are obtained for a value of γ= 2 and α= 0.25. The SGD with a momentum optimizer outperformed the other optimizers (Adam, RMSprop) for the datasets considered in the studies. The prediction time of the optimized deep residual network is 82 ms.

scholarly journals Method for detection of unsafe actions in power field based on edge computing architecture

2021 ◽  
Vol 10 (1) ◽  
Author(s):  
Yanfang Yin ◽  
Jinjiao Lin ◽  
Nongliang Sun ◽  
Qigang Zhu ◽  
Shuaishuai Zhang ◽  
...  
Keyword(s):  
Electric Power ◽  
Real Time ◽  
Action Recognition ◽  
Electric Power Industry ◽  
Edge Computing ◽  
Classification Rule ◽  
Computing Architecture ◽  
The Real ◽  
Time Performance ◽  
Temporal Features

AbstractDue to the high risk factors in the electric power industry, the safety of power system can be improved by using the surveillance system to predict and warn the operators’ nonstandard and unsafe actions in real time. In this paper, aiming at the real-time and accuracy requirements in video intelligent surveillance, a method based on edge computing architecture is proposed to judge unsafe actions of electric power operations in time. In this method, the service of unsafe actions judgment is deployed to the edge cloud, which improves the real-time performance. In order to identify the action being executed, the end-to-end action recognition model proposed in this paper uses the Temporal Convolutional Neural Network (TCN) to extract local temporal features and a Gate Recurrent Unit (GRU) layer to extract global temporal features, which increases the accuracy of action fragment recognition. The result of action recognition is combined with the result of equipment target recognition based on the yolov3 model, and the classification rule is used to determine whether the current action is safe. Experiments show that the proposed method has better real-time performance, and the proposed action cognition is verified on the MSRAction Dataset, which improves the recognition accuracy of action segments. At the same time, the judgment results of unsafe actions also prove the effectiveness of the proposed method.

Development of an integrated real time dispatching system

2014 ◽  
Vol 25 (7) ◽  
pp. 980-997 ◽  
Author(s):  
Yanting Ni ◽  
Yuchen Li ◽  
Jin Yao ◽  
Jingmin Li
Keyword(s):  
Real Time ◽  
Information Integration ◽  
Production Scheduling ◽  
Performance Metrics ◽  
Dynamic Scheduling ◽  
Shop Floor ◽  
Content Type ◽  
The Real ◽  
Work In Process ◽  
Dispatching Algorithm

Purpose – In a complex semiconductor manufacturing system (SMS) environment, the implementation of dynamic production scheduling and dispatching strategies is critical for SMS distributed collaborative manufacturing events to make quick and correct decisions. The purpose of this paper is to assist manufacturers in achieving the real time dispatching and obtaining integrated optimization for shop floor production scheduling. Design/methodology/approach – In this paper, an integrated model is designed under assemble to order environment and a framework of a real time dispatching (IRTD) system for production scheduling control is presented accordingly. Both of the scheduling and ordering performances are integrated into the days of inventory based dispatching algorithm, which can deal with the multiple indicators of dynamic scheduling and ordering in this system to generate the “optimal” dispatching policies. Subsequently, the platform of IRTD system is realized with four modules function embedded. Findings – The proposed IRTD system is designed to compare the previous constant work in process method in the experiment, which shows the better performance achievement of the IRTD system for shop floor production dynamic scheduling and order control. The presented framework and algorithm can facilitate real time dispatching information integration to obtain performance metrics in terms of reliability, availability, and maintainability. Research limitations/implications – The presented system can be further developed to generic factory manufacturing with the presented logic and architecture proliferation. Originality/value – The IRTD system can integrate the real time customer demand and work in process information, based on which manufacturers can make correct and timely decisions in solving dispatching strategies and ordering selection within an integrated information system.

scholarly journals The Manipulation of Real-Time Kinect-Based Robotic Arm Using Double-Hand Gestures

2020 ◽  
Vol 2020 ◽  
pp. 1-9
Author(s):  
Chern-Sheng Lin ◽  
Pei-Chi Chen ◽  
Yu-Ching Pan ◽  
Che-Ming Chang ◽  
Kuo-Liang Huang
Keyword(s):  
Real Time ◽  
Human Body ◽  
Relevant Information ◽  
Robotic Arm ◽  
Depth Sensor ◽  
Hand Gestures ◽  
The Real ◽  
Left Hand ◽  
Right Hand ◽  
The Right

This study focused on utilizing the Kinect depth sensor to track double-hand gestures and control a real-time robotic arm. The control system is mainly composed of the microprocessor, a color camera, the depth sensor, and the robotic arm. The Kinect depth sensor was used to take photos of the human body to analyze the skeleton of a human body and obtain the relevant information. Such information was used to identify the gestures of the left hand and the left palm of the user. The gesture of left hand was used as an input command device. The gesture of the right hand was used for imitation movement teaching of robotic arm. From the depth sensor, the real-time images of the human body and the deep information of each joint were collected and converted to the relative positions of the robotic arm. Combining forward kinematics and inverse kinematics and D-H link, the gesture information of the right hand was calculated, which was converted via coordinates into each angle of the motor of the robotic arm. From the color camera, when the left palm was not detected, the user could simply use the right hand to control the action and movement of the real-time robotic arm. When the left palm was detected and 5 fingertips were identified, it meant the start of recording the real-time imitation movement of the robotic arm by the right hand. When 0 fingertip was identified, it meant the stoppage of the above recording. When 2 fingertips were identified, the user could not only control the real-time robotic arm but also repeat the recorded actions.

scholarly journals A Multiday Evaluation of Real-Time Intramuscular EMG Usability with ANN

Sensors ◽  
2020 ◽  
Vol 20 (12) ◽  
pp. 3385
Author(s):  
Asim Waris ◽  
Muhammad Zia ur Rehman ◽  
Imran Khan Niazi ◽  
Mads Jochumsen ◽  
Kevin Englehart ◽  
...  
Keyword(s):  
Real Time ◽  
Performance Metrics ◽  
Completion Rate ◽  
Test Scheme ◽  
Real Time Control ◽  
The Real ◽  
Time Control ◽  
Recent Developments ◽  
Path Efficiency ◽  
Artificial Neural Network Ann

Recent developments in implantable technology, such as high-density recordings, wireless transmission of signals to a prosthetic hand, may pave the way for intramuscular electromyography (iEMG)-based myoelectric control in the future. This study aimed to investigate the real-time control performance of iEMG over time. A novel protocol was developed to quantify the robustness of the real-time performance parameters. Intramuscular wires were used to record EMG signals, which were kept inside the muscles for five consecutive days. Tests were performed on multiple days using Fitts’ law. Throughput, completion rate, path efficiency and overshoot were evaluated as performance metrics using three train/test strategies. Each train/test scheme was categorized on the basis of data quantity and the time difference between training and testing data. An artificial neural network (ANN) classifier was trained and tested on (i) data from the same day (WDT), (ii) data collected from the previous day and tested on present-day (BDT) and (iii) trained on all previous days including the present day and tested on present-day (CDT). It was found that the completion rate (91.6 ± 3.6%) of CDT was significantly better (p < 0.01) than BDT (74.02 ± 5.8%) and WDT (88.16 ± 3.6%). For BDT, on average, the first session of each day was significantly better (p < 0.01) than the second and third sessions for completion rate (77.9 ± 14.0%) and path efficiency (88.9 ± 16.9%). Subjects demonstrated the ability to achieve targets successfully with wire electrodes. Results also suggest that time variations in the iEMG signal can be catered by concatenating the data over several days. This scheme can be helpful in attaining stable and robust performance.

scholarly journals Method for Detection of Unsafe Actions in Power Field Based on Edge Computing Architecture

2021 ◽  
Author(s):  
Yanfang Yin ◽  
Jinjiao Lin ◽  
Nongliang Sun ◽  
Qigang Zhu ◽  
Shuaishuai Zhang ◽  
...  
Keyword(s):  
Electric Power ◽  
Real Time ◽  
Action Recognition ◽  
Electric Power Industry ◽  
Edge Computing ◽  
Classification Rule ◽  
Computing Architecture ◽  
The Real ◽  
Time Performance ◽  
Temporal Features

Abstract Due to the high risk factors in the electric power industry, the safety of power system can be improved by using the surveillance system to predict and warn the operators' nonstandard and unsafe actions by real time. In this paper, aiming at the real-time and accuracy requirements in video intelligent surveillance, a method based on edge computing architecture is proposed to judge unsafe actions of electric power operations in time. In this method, the service of unsafe actions judgment is deployed to the edge cloud, which improves the real-time performance. In order to identify the action being executing, the end-to-end action recognition model proposed in this paper uses the Temporal Convolutional Neural Network (TCN) to extract local temporal features and a Gate Recurrent Unit (GRU) layer to extract global temporal features, which increases the accuracy of action fragment recognition. The result of action recognition is combined with the result of equipment target recognition based on yolov3 model, and the classification rule is used to determine whether the current action is safe.Experiments show that the proposed method has better real-time performance,and the proposed action cgnition is verified on the MSRAction Dataset,which improves the recognition accuracy of action segments.At the same time,the judgment results of unsafe actions also prove the effectiveness of the proposed method.

Virtual manipulations for draping

2015 ◽  
Vol 27 (3) ◽  
pp. 417-433 ◽  
Author(s):  
Yuko Mesuda ◽  
Shigeru Inui ◽  
Yosuke Horiba
Keyword(s):  
Real Time ◽  
Real World ◽  
Design Methodology ◽  
Hand Gestures ◽  
Content Type ◽  
Key Technology ◽  
The Real ◽  
Hand Motion ◽  
Hand Trajectory ◽  
Clothing Design

Purpose – Draping is one method used in clothing design. It is important to virtualize draping in real time, and virtual cloth handling is a key technology for this purpose. A mouse is often used for real-time cloth handling in many studies. However, gesture manipulation is more realistic than movements using the mouse. The purpose of this paper is to demonstrate virtual cloth manipulation using hand gestures in the real world. Design/methodology/approach – In this study, the authors demonstrate three types of manipulation: moving, cutting, and attaching. The user’s hand coordinates are obtained with a Kinect, and the cloth model is manipulated by them. The cloth model is moved based on the position of the hand coordinates. The cloth model is cut along a cut line calculated from the hand coordinates. In attaching the cloth model, it is mapped to a dummy model and then part of the cloth model is fixed and another part is released. Findings – This method can move the cloth model according to the motion of the hands. The authors have succeeded in cutting the cloth model based on the hand trajectory. The cloth model can be attached to the dummy model and its form is changed along the dummy model shape. Originality/value – Cloth handling in many studies is based on indirect manipulation using a mouse. In this study, the cloth model is manipulated according to hand motion in the real world in real time.

The Real-Time Neural Genesis of a Punchline

2014 ◽  
Author(s):  
Irving Biederman ◽  
Ori Amir
Keyword(s):  
Real Time ◽  
The Real

Real-Time System for Sending Data from Three Clients to One Server with MATLAB

2015 ◽  
Vol 2 (1) ◽  
pp. 35-41
Author(s):  
Rivan Risdaryanto ◽  
Houtman P. Siregar ◽  
Dedy Loebis
Keyword(s):  
Information System ◽  
Real Time ◽  
Time System ◽  
Real Time System ◽  
The Real ◽  
Performance Time ◽  
Data Process ◽  
Effectiveness And Efficiency

The real-time system is now used on many fields, such as telecommunication, military, information system, evenmedical to get information quickly, on time and accurate. Needless to say, a real-time system will always considerthe performance time. In our application, we define the time target/deadline, so that the system should execute thewhole tasks under predefined deadline. However, if the system failed to finish the tasks, it will lead to fatal failure.In other words, if the system cannot be executed on time, it will affect the subsequent tasks. In this paper, wepropose a real-time system for sending data to find effectiveness and efficiency. Sending data process will beconstructed in MATLAB and sending data process has a time target as when data will send.

Sign in / Sign up

Export Citation Format

Share Document