scholarly journals A Mixed Perception Approach for Safe Human-Robot Collaboration in Industrial Automation

2020 ◽  
Author(s):  
Fatemeh Mohammadi Amin ◽  
Maryam Rezayati ◽  
Hans Wernher van de Venn ◽  
Hossein Karimpour
Keyword(s):  
Action Recognition ◽  
Manufacturing Systems ◽  
Detection System ◽  
Low Cost ◽  
Human Action ◽  
Physical Contact ◽  
Contact Detection ◽  
Human Beings ◽  
Industrial Automation ◽  
Human Robot Collaboration

Digital enabled manufacturing systems require high level of automation for fast and low-cost production but should also present flexibility and adaptiveness to varying and dynamic conditions in their environment, including the presence of human beings. This issue is addressed in this work by implementing a reliable system for real-time safe human-robot collaboration based upon the combination of human action and contact type detection systems. Two datasets containing contact and vision data are collected by using different volunteers. The action recognition system classifies human actions using the skeleton representation of the latter when entering the shared workspace and the contact detection system distinguishes between intentional and incidental interactions if a physical contact between human and robot takes place. Two different deep learning networks are used for human action recognition and contact detection which in combination, lead to the enhancement of human safety and an increase of the level of robot awareness about human intentions. The results show a promising path for future AI-driven solutions in safe and productive human–robot collaboration (HRC) in industrial automation.

scholarly journals A Mixed-Perception Approach for Safe Human–Robot Collaboration in Industrial Automation

Sensors ◽  
2020 ◽  
Vol 20 (21) ◽  
pp. 6347
Author(s):  
Fatemeh Mohammadi Amin ◽  
Maryam Rezayati ◽  
Hans Wernher van de Venn ◽  
Hossein Karimpour
Keyword(s):  
Action Recognition ◽  
Manufacturing Systems ◽  
Physical Contact ◽  
Contact Detection ◽  
Human Beings ◽  
Industrial Automation ◽  
Human Actions ◽  
Shared Workspace ◽  
Human Safety ◽  
Human Robot Collaboration

Digital-enabled manufacturing systems require a high level of automation for fast and low-cost production but should also present flexibility and adaptiveness to varying and dynamic conditions in their environment, including the presence of human beings; however, this presence of workers in the shared workspace with robots decreases the productivity, as the robot is not aware about the human position and intention, which leads to concerns about human safety. This issue is addressed in this work by designing a reliable safety monitoring system for collaborative robots (cobots). The main idea here is to significantly enhance safety using a combination of recognition of human actions using visual perception and at the same time interpreting physical human–robot contact by tactile perception. Two datasets containing contact and vision data are collected by using different volunteers. The action recognition system classifies human actions using the skeleton representation of the latter when entering the shared workspace and the contact detection system distinguishes between intentional and incidental interactions if physical contact between human and cobot takes place. Two different deep learning networks are used for human action recognition and contact detection, which in combination, are expected to lead to the enhancement of human safety and an increase in the level of cobot perception about human intentions. The results show a promising path for future AI-driven solutions in safe and productive human–robot collaboration (HRC) in industrial automation.

scholarly journals Low-Cost Embedded System Using Convolutional Neural Networks-Based Spatiotemporal Feature Map for Real-Time Human Action Recognition

2021 ◽  
Vol 11 (11) ◽  
pp. 4940
Author(s):  
Jinsoo Kim ◽  
Jeongho Cho
Keyword(s):  
Embedded System ◽  
Real Time ◽  
Action Recognition ◽  
Processing Speed ◽  
Recognition Accuracy ◽  
Low Cost ◽  
Human Action Recognition ◽  
Human Action ◽  
Video Data ◽  
Feature Maps

The field of research related to video data has difficulty in extracting not only spatial but also temporal features and human action recognition (HAR) is a representative field of research that applies convolutional neural network (CNN) to video data. The performance for action recognition has improved, but owing to the complexity of the model, some still limitations to operation in real-time persist. Therefore, a lightweight CNN-based single-stream HAR model that can operate in real-time is proposed. The proposed model extracts spatial feature maps by applying CNN to the images that develop the video and uses the frame change rate of sequential images as time information. Spatial feature maps are weighted-averaged by frame change, transformed into spatiotemporal features, and input into multilayer perceptrons, which have a relatively lower complexity than other HAR models; thus, our method has high utility in a single embedded system connected to CCTV. The results of evaluating action recognition accuracy and data processing speed through challenging action recognition benchmark UCF-101 showed higher action recognition accuracy than the HAR model using long short-term memory with a small amount of video frames and confirmed the real-time operational possibility through fast data processing speed. In addition, the performance of the proposed weighted mean-based HAR model was verified by testing it in Jetson NANO to confirm the possibility of using it in low-cost GPU-based embedded systems.

scholarly journals Human Actions and Hand Gesture Recognition with Deep Learning

2019 ◽  
Vol 9 (2) ◽  
pp. 1250-1253
Keyword(s):  
Deep Learning ◽  
Action Recognition ◽  
Gesture Recognition ◽  
Human Activities ◽  
Low Cost ◽  
Human Action Recognition ◽  
Human Action ◽  
Hand Gesture Recognition ◽  
Hand Gesture ◽  
Characteristic Features

Over recent times, deep learning has been challenged extensively to automatically read and interpret characteristic features from large volumes of data. Human Action Recognition (HAR) has been experimented with variety of techniques like wearable devices, mobile devices etc., but they can cause unnecessary discomfort to people especially elderly and child. Since it is very vital to monitor the movements of elderly and children in unattended scenarios, thus, HAR is focused. A smart human action recognition method to automatically identify the human activities from skeletal joint motions and combines the competencies are focused. We can also intimate the near ones about the status of the people. Also, it is a low-cost method and has high accuracy. Thus, this provides a way to help the senior citizens and children from any kind of mishaps and health issues. Hand gesture recognition is also discussed along with human activities using deep learning.

Security robot for the prevention of workplace violence using the non-linear adaptive heuristic mathematical model

Work ◽  
2021 ◽  
pp. 1-9
Author(s):  
Hai Tao ◽  
Md Arafatur Rahman ◽  
Ahmed AL-Saffar ◽  
Renrui Zhang ◽  
Sinan Q Salih ◽  
...  
Keyword(s):  
Mathematical Model ◽  
Workplace Violence ◽  
Physical Contact ◽  
Optimal Decision ◽  
Human Being ◽  
Human Beings ◽  
Wide Range ◽  
Non Linear ◽  
Adaptive Heuristic ◽  
Human Robot Collaboration

BACKGROUND: Nowadays, workplace violence is found to be a mental health hazard and considered a crucial topic. The collaboration between robots and humans is increasing with the growth of Industry 4.0. Therefore, the first problem that must be solved is human-machine security. Ensuring the safety of human beings is one of the main aspects of human-robotic interaction. This is not just about preventing collisions within a shared space among human beings and robots; it includes all possible means of harm for an individual, from physical contact to unpleasant or dangerous psychological effects. OBJECTIVE: In this paper, Non-linear Adaptive Heuristic Mathematical Model (NAHMM) has been proposed for the prevention of workplace violence using security Human-Robot Collaboration (HRC). Human-Robot Collaboration (HRC) is an area of research with a wide range of up-demands, future scenarios, and potential economic influence. HRC is an interdisciplinary field of research that encompasses cognitive sciences, classical robotics, and psychology. RESULTS: The robot can thus make the optimal decision between actions that expose its capabilities to the human being and take the best steps given the knowledge that is currently available to the human being. Further, the ideal policy can be measured carefully under certain observability assumptions. CONCLUSION: The system is shown on a collaborative robot and is compared to a state of the art security system. The device is experimentally demonstrated. The new system is being evaluated qualitatively and quantitatively.

scholarly journals Real-Time Human Action Recognition with a Low-Cost RGB Camera and Mobile Robot Platform

Sensors ◽  
2020 ◽  
Vol 20 (10) ◽  
pp. 2886 ◽  
Author(s):  
Junwoo Lee ◽  
Bummo Ahn
Keyword(s):  
Mobile Robot ◽  
Real Time ◽  
Action Recognition ◽  
Low Cost ◽  
Human Action Recognition ◽  
Human Action ◽  
Research Area ◽  
Training Dataset ◽  
Kinect Camera ◽  
Robot Platform

Human action recognition is an important research area in the field of computer vision that can be applied in surveillance, assisted living, and robotic systems interacting with people. Although various approaches have been widely used, recent studies have mainly focused on deep-learning networks using Kinect camera that can easily generate data on skeleton joints using depth data, and have achieved satisfactory performances. However, their models are deep and complex to achieve a higher recognition score; therefore, they cannot be applied to a mobile robot platform using a Kinect camera. To overcome these limitations, we suggest a method to classify human actions in real-time using a single RGB camera, which can be applied to the mobile robot platform as well. We integrated two open-source libraries, i.e., OpenPose and 3D-baseline, to extract skeleton joints on RGB images, and classified the actions using convolutional neural networks. Finally, we set up the mobile robot platform including an NVIDIA JETSON XAVIER embedded board and tracking algorithm to monitor a person continuously. We achieved an accuracy of 70% on the NTU-RGBD training dataset, and the whole process was performed on an average of 15 frames per second (FPS) on an embedded board system.

Human Action Recognition Based on a Spatio-Temporal Video Autoencoder

2020 ◽  
Vol 34 (11) ◽  
pp. 2040001
Author(s):  
Anderson Carlos Sousa e Santos ◽  
Helio Pedrini
Keyword(s):  
Action Recognition ◽  
Low Cost ◽  
Human Action Recognition ◽  
Human Action ◽  
Video Sequences ◽  
Computer Vision Applications ◽  
Recognition Systems ◽  
Spatio Temporal ◽  
Human Operators ◽  
Accuracy Rates

Due to rapid advances in the development of surveillance cameras with high sampling rates, low cost, small size and high resolution, video-based action recognition systems have become more commonly used in various computer vision applications. Human operators can be supported with the aid of such systems to detect events of interest in video sequences, improving recognition results and reducing failure cases. In this work, we propose and evaluate a method to learn two-dimensional (2D) representations from video sequences based on an autoencoder framework. Spatial and temporal information is explored through a multi-stream convolutional neural network in the context of human action recognition. Experimental results on the challenging UCF101 and HMDB51 datasets demonstrate that our representation is capable of achieving competitive accuracy rates when compared to other approaches available in the literature.

Human-Like Daily Action Recognition Model

2005 ◽  
Vol 17 (6) ◽  
pp. 672-680 ◽  
Author(s):  
Taketoshi Mori ◽  
◽  
Kousuke Tsujioka
Keyword(s):  
Action Recognition ◽  
Human Action Recognition ◽  
Human Action ◽  
Recognition Algorithm ◽  
The Body ◽  
Human Being ◽  
Human Beings ◽  
Recognition Model ◽  
Recognition Ability ◽  
Partial View

This paper proposes a human-like action recognition model. When the model is implemented as a system, the system recognizes human actions similarly to human beings recognize. The recognition algorithm is constructed taking account of the following characteristics of human action recognition: simultaneous recognition, priority between actions, judgement fuzziness, multiple judge conditions for one action, and recognition ability from partial view of the body. The experiments based on a comparison with completed questionnaires demonstrated that the system recognizes human action the way like a human being does. Results ensure natural understanding of human action by a system, which leads to smooth communication between computer systems and human beings.

scholarly journals Surge of the Innovative Quest in the first lockdown period due to the Pandemic Effect

2021 ◽  
Vol 9 (4) ◽  
pp. 34-43
Author(s):  
Ishita Ghosh
Keyword(s):  
Review Paper ◽  
Low Cost ◽  
Cost Effective ◽  
Physical Contact ◽  
Effective Vaccine ◽  
Successful Implementation ◽  
Human Beings ◽  
To Come ◽  
The One ◽  
Effective Drugs

When the entire world is reeling under the COVID 19 pandemic effect and the tensed human race is struggling to return back to the normalcy of life, the one thing which has become very active is the grey cells of our brain. The pandemic effect has cut down our physical limits due to the movement constraints. But it is thankfully unable to restrict the ticking of the grey cells of the human brain. As is said, “Necessity is the mother of the invention”. Sure enough!! We can be extremely pleased to know that the innovative surge in science and technology continues unabated in this lockdown period. The prime requirement of the pandemic effect is social distancing, less physical contact and keeping ourselves away from infection by corona virus. Keeping this necessity in mind, the doctors, the engineers, the researchers as well as the students’ community are keeping themselves busy in pumping out the solutions to the currently faced problems. The outputs include automatic masks machines, low cost PPE’s, automatic wash basins, suitable ventilators, sanitizer tunnels etc. This review paper looks into the innovative surge already made and what more can be churned out for the effective social safety in this tensed pandemic effect. The most awaited news as of now is the successful implementation of an effective vaccine and cost effective drugs which can help the human beings breathe easy. The pandemic effect has also showed us the way for a cleaner and greener nature. It is now a challenge to the intellectual world to come up with inexpensive, innovative and smart solutions which will make our beautiful planet safer, greener, cleaner and worthier to live in.

scholarly journals A Lightweight Hierarchical Model with Frame-Level Joints Adaptive Graph Convolution for Skeleton-Based Action Recognition

2021 ◽  
Vol 2021 ◽  
pp. 1-13
Author(s):  
Yujian Jiang ◽  
Xue Yang ◽  
Jingyu Liu ◽  
Junming Zhang
Keyword(s):  
Action Recognition ◽  
Feature Fusion ◽  
Low Cost ◽  
Real Life ◽  
Model Performance ◽  
Human Action Recognition ◽  
Human Action ◽  
Embedded Devices ◽  
Convolutional Network ◽  
Action Sequence

In skeleton-based human action recognition methods, human behaviours can be analysed through temporal and spatial changes in the human skeleton. Skeletons are not limited by clothing changes, lighting conditions, or complex backgrounds. This recognition method is robust and has aroused great interest; however, many existing studies used deep-layer networks with large numbers of required parameters to improve the model performance and thus lost the advantage of less computation of skeleton data. It is difficult to deploy previously established models to real-life applications based on low-cost embedded devices. To obtain a model with fewer parameters and a higher accuracy, this study designed a lightweight frame-level joints adaptive graph convolutional network (FLAGCN) model to solve skeleton-based action recognition tasks. Compared with the classical 2s-AGCN model, the new model obtained a higher precision with 1/8 of the parameters and 1/9 of the floating-point operations (FLOPs). Our proposed network characterises three main improvements. First, a previous feature-fusion method replaces the multistream network and reduces the number of required parameters. Second, at the spatial level, two kinds of graph convolution methods capture different aspects of human action information. A frame-level graph convolution constructs a human topological structure for each data frame, whereas an adjacency graph convolution captures the characteristics of the adjacent joints. Third, the model proposed in this study hierarchically extracts different levels of action sequence features, making the model clear and easy to understand; further, it reduces the depth of the model and the number of parameters. A large number of experiments on the NTU RGB + D 60 and 120 data sets show that this method has the advantages of few required parameters, low computational costs, and fast speeds. It also has a simple structure and training process that make it easy to deploy in real-time recognition systems based on low-cost embedded devices.

Sign in / Sign up

Export Citation Format

Share Document