scholarly journals Spatio-Temporal Features in Action Recognition Using 3D Skeletal Joints

Sensors ◽  
2019 ◽  
Vol 19 (2) ◽  
pp. 423 ◽  
Author(s):  
Mihai Trăscău ◽  
Mihai Nan ◽  
Adina Florea
Keyword(s):  
Action Recognition ◽  
Assisted Living ◽  
State Of The Art ◽  
Ambient Assisted Living ◽  
Depth Image ◽  
Depth Images ◽  
Temporal Features ◽  
Improve Accuracy ◽  
Spatio Temporal ◽  
Temporal Dimensions

Robust action recognition methods lie at the cornerstone of Ambient Assisted Living (AAL) systems employing optical devices. Using 3D skeleton joints extracted from depth images taken with time-of-flight (ToF) cameras has been a popular solution for accomplishing these tasks. Though seemingly scarce in terms of information availability compared to its RGB or depth image counterparts, the skeletal representation has proven to be effective in the task of action recognition. This paper explores different interpretations of both the spatial and the temporal dimensions of a sequence of frames describing an action. We show that rather intuitive approaches, often borrowed from other computer vision tasks, can improve accuracy. We report results based on these modifications and propose an architecture that uses temporal convolutions with results comparable to the state of the art.

scholarly journals Multi-Term Attention Networks for Skeleton-Based Action Recognition

2020 ◽  
Vol 10 (15) ◽  
pp. 5326
Author(s):  
Xiaolei Diao ◽  
Xiaoqiang Li ◽  
Chen Huang
Keyword(s):  
Neural Network ◽  
Time Scales ◽  
Action Recognition ◽  
State Of The Art ◽  
Attention Networks ◽  
Weighted Fusion ◽  
Temporal Features ◽  
Benchmark Datasets ◽  
Spatio Temporal ◽  
Different Time Scales

The same action takes different time in different cases. This difference will affect the accuracy of action recognition to a certain extent. We propose an end-to-end deep neural network called “Multi-Term Attention Networks” (MTANs), which solves the above problem by extracting temporal features with different time scales. The network consists of a Multi-Term Attention Recurrent Neural Network (MTA-RNN) and a Spatio-Temporal Convolutional Neural Network (ST-CNN). In MTA-RNN, a method for fusing multi-term temporal features are proposed to extract the temporal dependence of different time scales, and the weighted fusion temporal feature is recalibrated by the attention mechanism. Ablation research proves that this network has powerful spatio-temporal dynamic modeling capabilities for actions with different time scales. We perform extensive experiments on four challenging benchmark datasets, including the NTU RGB+D dataset, UT-Kinect dataset, Northwestern-UCLA dataset, and UWA3DII dataset. Our method achieves better results than the state-of-the-art benchmarks, which demonstrates the effectiveness of MTANs.

Human Action Recognition Using Spatio-Temporal Multiplier Network and Attentive Correlated Temporal Feature

2021 ◽  
Author(s):  
C. Indhumathi ◽  
V. Murugan ◽  
G. Muthulakshmii
Keyword(s):  
Action Recognition ◽  
State Of The Art ◽  
Human Action Recognition ◽  
Human Action ◽  
Regional Correlation ◽  
Temporal Features ◽  
Adaptive Motion ◽  
Spatio Temporal ◽  
Inter Frame ◽  
Temporal Feature

Nowadays, action recognition has gained more attention from the computer vision community. Normally for recognizing human actions, spatial and temporal features are extracted. Two-stream convolutional neural network is used commonly for human action recognition in videos. In this paper, Adaptive motion Attentive Correlated Temporal Feature (ACTF) is used for temporal feature extractor. The temporal average pooling in inter-frame is used for extracting the inter-frame regional correlation feature and mean feature. This proposed method has better accuracy of 96.9% for UCF101 and 74.6% for HMDB51 datasets, respectively, which are higher than the other state-of-the-art methods.

scholarly journals Improved Action Recognition with Separable Spatio-Temporal Attention Using Alternative Skeletal and Video Pre-Processing

Sensors ◽  
2021 ◽  
Vol 21 (3) ◽  
pp. 1005
Author(s):  
Pau Climent-Pérez ◽  
Francisco Florez-Revuelta
Keyword(s):  
Activities Of Daily Living ◽  
Action Recognition ◽  
Assisted Living ◽  
Daily Living ◽  
State Of The Art ◽  
Temporal Attention ◽  
Attention Network ◽  
Full Activity ◽  
Potential Benefits ◽  
Spatio Temporal

The potential benefits of recognising activities of daily living from video for active and assisted living have yet to be fully untapped. These technologies can be used for behaviour understanding, and lifelogging for caregivers and end users alike. The recent publication of realistic datasets for this purpose, such as the Toyota Smarthomes dataset, calls for pushing forward the efforts to improve action recognition. Using the separable spatio-temporal attention network proposed in the literature, this paper introduces a view-invariant normalisation of skeletal pose data and full activity crops for RGB data, which improve the baseline results by 9.5% (on the cross-subject experiments), outperforming state-of-the-art techniques in this field when using the original unmodified skeletal data in dataset. Our code and data are available online.

scholarly journals A Low-Dimensional Radial Silhouette-Based Feature for Fast Human Action Recognition Fusing Multiple Views

2014 ◽  
Vol 2014 ◽  
pp. 1-11 ◽  
Author(s):  
Alexandros Andre Chaaraoui ◽  
Francisco Flórez-Revuelta
Keyword(s):  
Real Time ◽  
Action Recognition ◽  
Assisted Living ◽  
Learning Algorithm ◽  
Ambient Assisted Living ◽  
Human Action Recognition ◽  
Human Action ◽  
Sequence Matching ◽  
Low Dimensional ◽  
Video Frequency

This paper presents a novel silhouette-based feature for vision-based human action recognition, which relies on the contour of the silhouette and a radial scheme. Its low-dimensionality and ease of extraction result in an outstanding proficiency for real-time scenarios. This feature is used in a learning algorithm that by means of model fusion of multiple camera streams builds a bag of key poses, which serves as a dictionary of known poses and allows converting the training sequences into sequences of key poses. These are used in order to perform action recognition by means of a sequence matching algorithm. Experimentation on three different datasets returns high and stable recognition rates. To the best of our knowledge, this paper presents the highest results so far on the MuHAVi-MAS dataset. Real-time suitability is given, since the method easily performs above video frequency. Therefore, the related requirements that applications as ambient-assisted living services impose are successfully fulfilled.

scholarly journals Improved Collaborative Representation Classifier Based on l2-Regularized for Human Action Recognition

2017 ◽  
Vol 2017 ◽  
pp. 1-6
Author(s):  
Shirui Huo ◽  
Tianrui Hu ◽  
Ce Li
Keyword(s):  
Action Recognition ◽  
Human Action Recognition ◽  
Test Sample ◽  
Human Action ◽  
Superior Performance ◽  
Depth Image ◽  
Collaborative Representation ◽  
Depth Images ◽  
Spatiotemporal Information ◽  
Depth Motion Maps

Human action recognition is an important recent challenging task. Projecting depth images onto three depth motion maps (DMMs) and extracting deep convolutional neural network (DCNN) features are discriminant descriptor features to characterize the spatiotemporal information of a specific action from a sequence of depth images. In this paper, a unified improved collaborative representation framework is proposed in which the probability that a test sample belongs to the collaborative subspace of all classes can be well defined and calculated. The improved collaborative representation classifier (ICRC) based on l2-regularized for human action recognition is presented to maximize the likelihood that a test sample belongs to each class, then theoretical investigation into ICRC shows that it obtains a final classification by computing the likelihood for each class. Coupled with the DMMs and DCNN features, experiments on depth image-based action recognition, including MSRAction3D and MSRGesture3D datasets, demonstrate that the proposed approach successfully using a distance-based representation classifier achieves superior performance over the state-of-the-art methods, including SRC, CRC, and SVM.

Sign in / Sign up

Export Citation Format

Share Document