scholarly journals Comparison between Recurrent Networks and Temporal Convolutional Networks Approaches for Skeleton-Based Action Recognition

Sensors ◽  
2021 ◽  
Vol 21 (6) ◽  
pp. 2051
Author(s):  
Mihai Nan ◽  
Mihai Trăscău ◽  
Adina Magda Florea ◽  
Cezar Cătălin Iacob
Keyword(s):  
Action Recognition ◽  
State Of The Art ◽  
Human Action Recognition ◽  
Human Action ◽  
Assistive Robotics ◽  
Recurrent Networks ◽  
Human Machine Interaction ◽  
Convolutional Networks ◽  
Machine Interaction ◽  
Spatial And Temporal Characteristics

Action recognition plays an important role in various applications such as video monitoring, automatic video indexing, crowd analysis, human-machine interaction, smart homes and personal assistive robotics. In this paper, we propose improvements to some methods for human action recognition from videos that work with data represented in the form of skeleton poses. These methods are based on the most widely used techniques for this problem—Graph Convolutional Networks (GCNs), Temporal Convolutional Networks (TCNs) and Recurrent Neural Networks (RNNs). Initially, the paper explores and compares different ways to extract the most relevant spatial and temporal characteristics for a sequence of frames describing an action. Based on this comparative analysis, we show how a TCN type unit can be extended to work even on the characteristics extracted from the spatial domain. To validate our approach, we test it against a benchmark often used for human action recognition problems and we show that our solution obtains comparable results to the state-of-the-art, but with a significant increase in the inference speed.

scholarly journals Skeleton based Human Action Recognition using a Structured-Tree Neural Network

2020 ◽  
Vol 5 (8) ◽  
pp. 849-854
Author(s):  
Muhammad Sajid Khan ◽  
Andrew Ware ◽  
Misha Karim ◽  
Nisar Bahoo ◽  
Muhammad Junaid Khalid
Keyword(s):  
Neural Network ◽  
Action Recognition ◽  
Research Effort ◽  
Human Action Recognition ◽  
Human Action ◽  
Human Machine Interaction ◽  
Human Movements ◽  
Significant Research ◽  
Structure Lead ◽  
Machine Interaction

The ability for automated technologies to correctly identify a human’s actions provides considerable scope for systems that make use of human-machine interaction. Thus, automatic3D Human Action Recognition is an area that has seen significant research effort. In work described here, a human’s everyday 3D actions recorded in the NTU RGB+D dataset are identified using a novel structured-tree neural network. The nodes of the tree represent the skeleton joints, with the spine joint being represented by the root. The connection between a child node and its parent is known as the incoming edge while the reciprocal connection is known as the outgoing edge. The uses of tree structure lead to a system that intuitively maps to human movements. The classifier uses the change in displacement of joints and change in the angles between incoming and outgoing edges as features for classification of the actions performed

scholarly journals Distinct Two-Stream Convolutional Networks for Human Action Recognition in Videos Using Segment-Based Temporal Modeling

Data ◽  
2020 ◽  
Vol 5 (4) ◽  
pp. 104
Author(s):  
Ashok Sarabu ◽  
Ajit Kumar Santra
Keyword(s):  
Action Recognition ◽  
Data Augmentation ◽  
Main Idea ◽  
Human Action Recognition ◽  
Human Action ◽  
Great Success ◽  
Temporal Modeling ◽  
Convolutional Networks ◽  
Temporal Features ◽  
Augmentation Techniques

The Two-stream convolution neural network (CNN) has proven a great success in action recognition in videos. The main idea is to train the two CNNs in order to learn spatial and temporal features separately, and two scores are combined to obtain final scores. In the literature, we observed that most of the methods use similar CNNs for two streams. In this paper, we design a two-stream CNN architecture with different CNNs for the two streams to learn spatial and temporal features. Temporal Segment Networks (TSN) is applied in order to retrieve long-range temporal features, and to differentiate the similar type of sub-action in videos. Data augmentation techniques are employed to prevent over-fitting. Advanced cross-modal pre-training is discussed and introduced to the proposed architecture in order to enhance the accuracy of action recognition. The proposed two-stream model is evaluated on two challenging action recognition datasets: HMDB-51 and UCF-101. The findings of the proposed architecture shows the significant performance increase and it outperforms the existing methods.

scholarly journals Feature Fusion of Deep Spatial Features and Handcrafted Spatiotemporal Features for Human Action Recognition

Sensors ◽  
2019 ◽  
Vol 19 (7) ◽  
pp. 1599 ◽  
Author(s):  
Md Uddin ◽  
Young-Koo Lee
Keyword(s):  
Action Recognition ◽  
State Of The Art ◽  
Human Action Recognition ◽  
Human Action ◽  
Support Vector ◽  
Feature Descriptor ◽  
Weber’S Law ◽  
Weber's Law ◽  
Spatiotemporal Features ◽  
Spatial Features

Human action recognition plays a significant part in the research community due to its emerging applications. A variety of approaches have been proposed to resolve this problem, however, several issues still need to be addressed. In action recognition, effectively extracting and aggregating the spatial-temporal information plays a vital role to describe a video. In this research, we propose a novel approach to recognize human actions by considering both deep spatial features and handcrafted spatiotemporal features. Firstly, we extract the deep spatial features by employing a state-of-the-art deep convolutional network, namely Inception-Resnet-v2. Secondly, we introduce a novel handcrafted feature descriptor, namely Weber’s law based Volume Local Gradient Ternary Pattern (WVLGTP), which brings out the spatiotemporal features. It also considers the shape information by using gradient operation. Furthermore, Weber’s law based threshold value and the ternary pattern based on an adaptive local threshold is presented to effectively handle the noisy center pixel value. Besides, a multi-resolution approach for WVLGTP based on an averaging scheme is also presented. Afterward, both these extracted features are concatenated and feed to the Support Vector Machine to perform the classification. Lastly, the extensive experimental analysis shows that our proposed method outperforms state-of-the-art approaches in terms of accuracy.

scholarly journals Human Action Recognition Using Improved Salient Dense Trajectories

2016 ◽  
Vol 2016 ◽  
pp. 1-11 ◽  
Author(s):  
Qingwu Li ◽  
Haisu Cheng ◽  
Yan Zhou ◽  
Guanying Huo
Keyword(s):  
Action Recognition ◽  
State Of The Art ◽  
Human Action Recognition ◽  
Human Action ◽  
Interest Points ◽  
Dense Trajectories ◽  
Dense Trajectory ◽  
Sparse Coefficient ◽  
Active Research ◽  
Motion Saliency

Human action recognition in videos is a topic of active research in computer vision. Dense trajectory (DT) features were shown to be efficient for representing videos in state-of-the-art approaches. In this paper, we present a more effective approach of video representation using improved salient dense trajectories: first, detecting the motion salient region and extracting the dense trajectories by tracking interest points in each spatial scale separately and then refining the dense trajectories via the analysis of the motion saliency. Then, we compute several descriptors (i.e., trajectory displacement, HOG, HOF, and MBH) in the spatiotemporal volume aligned with the trajectories. Finally, in order to represent the videos better, we optimize the framework of bag-of-words according to the motion salient intensity distribution and the idea of sparse coefficient reconstruction. Our architecture is trained and evaluated on the four standard video actions datasets of KTH, UCF sports, HMDB51, and UCF50, and the experimental results show that our approach performs competitively comparing with the state-of-the-art results.

Human Action Recognition Using Spatio-Temporal Multiplier Network and Attentive Correlated Temporal Feature

2021 ◽  
Author(s):  
C. Indhumathi ◽  
V. Murugan ◽  
G. Muthulakshmii
Keyword(s):  
Action Recognition ◽  
State Of The Art ◽  
Human Action Recognition ◽  
Human Action ◽  
Regional Correlation ◽  
Temporal Features ◽  
Adaptive Motion ◽  
Spatio Temporal ◽  
Inter Frame ◽  
Temporal Feature

Nowadays, action recognition has gained more attention from the computer vision community. Normally for recognizing human actions, spatial and temporal features are extracted. Two-stream convolutional neural network is used commonly for human action recognition in videos. In this paper, Adaptive motion Attentive Correlated Temporal Feature (ACTF) is used for temporal feature extractor. The temporal average pooling in inter-frame is used for extracting the inter-frame regional correlation feature and mean feature. This proposed method has better accuracy of 96.9% for UCF101 and 74.6% for HMDB51 datasets, respectively, which are higher than the other state-of-the-art methods.

scholarly journals Dual Attention-Guided Multiscale Dynamic Aggregate Graph Convolutional Networks for Skeleton-Based Human Action Recognition

Symmetry ◽  
2020 ◽  
Vol 12 (10) ◽  
pp. 1589
Author(s):  
Zeyuan Hu ◽  
Eung-Joo Lee
Keyword(s):  
Action Recognition ◽  
Human Action Recognition ◽  
Human Action ◽  
Great Success ◽  
Semantic Features ◽  
Convolutional Networks ◽  
Temporal Correlations ◽  
Semantic Relevance ◽  
High Level ◽  
Relationship Of

Traditional convolution neural networks have achieved great success in human action recognition. However, it is challenging to establish effective associations between different human bone nodes to capture detailed information. In this paper, we propose a dual attention-guided multiscale dynamic aggregate graph convolution neural network (DAG-GCN) for skeleton-based human action recognition. Our goal is to explore the best correlation and determine high-level semantic features. First, a multiscale dynamic aggregate GCN module is used to capture important semantic information and to establish dependence relationships for different bone nodes. Second, the higher level semantic feature is further refined, and the semantic relevance is emphasized through a dual attention guidance module. In addition, we exploit the relationship of joints hierarchically and the spatial temporal correlations through two modules. Experiments with the DAG-GCN method result in good performance on the NTU-60-RGB+D and NTU-120-RGB+D datasets. The accuracy is 95.76% and 90.01%, respectively, for the cross (X)-View and X-Subon the NTU60dataset.

scholarly journals A Review on Computer Vision-Based Methods for Human Action Recognition

2020 ◽  
Vol 6 (6) ◽  
pp. 46
Author(s):  
Mahmoud Al-Faris ◽  
John Chiverton ◽  
David Ndzi ◽  
Ahmed Isam Ahmed
Keyword(s):  
Health Care ◽  
Computer Vision ◽  
Action Recognition ◽  
State Of The Art ◽  
Human Action Recognition ◽  
Human Action ◽  
Generative Models ◽  
Future Research ◽  
Learning Technology ◽  
Recognition Systems

Human action recognition targets recognising different actions from a sequence of observations and different environmental conditions. A wide different applications is applicable to vision based action recognition research. This can include video surveillance, tracking, health care, and human–computer interaction. However, accurate and effective vision based recognition systems continue to be a big challenging area of research in the field of computer vision. This review introduces the most recent human action recognition systems and provides the advances of state-of-the-art methods. To this end, the direction of this research is sorted out from hand-crafted representation based methods including holistic and local representation methods with various sources of data, to a deep learning technology including discriminative and generative models and multi-modality based methods. Next, the most common datasets of human action recognition are presented. This review introduces several analyses, comparisons and recommendations that help to find out the direction of future research.

scholarly journals Deep Learning-Based Action Recognition Using 3D Skeleton Joints Information

Inventions ◽  
2020 ◽  
Vol 5 (3) ◽  
pp. 49
Author(s):  
Nusrat Tasnim ◽  
Md. Mahbubul Islam ◽  
Joong-Hwan Baek
Keyword(s):  
Action Recognition ◽  
Large Scale ◽  
Dimensional Space ◽  
Human Action Recognition ◽  
Human Action ◽  
Human Machine Interaction ◽  
Human Actions ◽  
3 Dimensional ◽  
3D Skeleton ◽  
Color Depth

Human action recognition has turned into one of the most attractive and demanding fields of research in computer vision and pattern recognition for facilitating easy, smart, and comfortable ways of human-machine interaction. With the witnessing of massive improvements to research in recent years, several methods have been suggested for the discrimination of different types of human actions using color, depth, inertial, and skeleton information. Despite having several action identification methods using different modalities, classifying human actions using skeleton joints information in 3-dimensional space is still a challenging problem. In this paper, we conceive an efficacious method for action recognition using 3D skeleton data. First, large-scale 3D skeleton joints information was analyzed and accomplished some meaningful pre-processing. Then, a simple straight-forward deep convolutional neural network (DCNN) was designed for the classification of the desired actions in order to evaluate the effectiveness and embonpoint of the proposed system. We also conducted prior DCNN models such as ResNet18 and MobileNetV2, which outperform existing systems using human skeleton joints information.

Sign in / Sign up

Export Citation Format

Share Document