Action recognition from depth sequence using depth motion maps-based local ternary patterns and CNN

Background & Objective: Identification of human actions from video has gathered much attention in past few years. Most of the computer vision tasks such as Health Care Activity Detection, Suspicious Activity detection, Human Computer Interactions etc. are based on the principle of activity detection. Automatic labelling of activity from videos frames is known as activity detection. Motivation of this work is to use most out of the data generated from sensors and use them for recognition of classes. Recognition of actions from videos sequences is a growing field with the upcoming trends of deep neural networks. Automatic learning capability of Convolutional Neural Network (CNN) make them good choice as compared to traditional handcrafted based approaches. With the increasing demand of RGB-D sensors combination of RGB and depth data is in great demand. This work comprises of the use of dynamic images generated from RGB combined with depth map for action recognition purpose. We have experimented our approach on pre trained VGG-F model using MSR Daily activity dataset and UTD MHAD Dataset. We achieve state of the art results. To support our research, we have calculated different parameters apart from accuracy such as precision, F score, recall. Conclusion: Accordingly, the investigation confirms improvement in term of accuracy, precision, F-Score and Recall. The proposed model is 4 Stream model is prone to occlusion, used in real time and also the data from the RGB-D sensor is fully utilized.

Download Full-text

Exploring 3D Human Action Recognition Using STACOG on Multi-View Depth Motion Maps Sequences

Sensors ◽

10.3390/s21113642 ◽

2021 ◽

Vol 21 (11) ◽

pp. 3642

Author(s):

Mohammad Farhad Bulbul ◽

Sadiya Tabussum ◽

Hazrat Ali ◽

Wenli Zheng ◽

Mi Young Lee ◽

...

Keyword(s):

Action Recognition ◽

Depth Map ◽

Human Action Recognition ◽

Human Action ◽

Collaborative Representation ◽

Auto Correlation ◽

Time Operation ◽

Real Time Operation ◽

Benchmark Datasets ◽

Depth Motion Maps

This paper proposes an action recognition framework for depth map sequences using the 3D Space-Time Auto-Correlation of Gradients (STACOG) algorithm. First, each depth map sequence is split into two sets of sub-sequences of two different frame lengths individually. Second, a number of Depth Motion Maps (DMMs) sequences from every set are generated and are fed into STACOG to find an auto-correlation feature vector. For two distinct sets of sub-sequences, two auto-correlation feature vectors are obtained and applied gradually to L2-regularized Collaborative Representation Classifier (L2-CRC) for computing a pair of sets of residual values. Next, the Logarithmic Opinion Pool (LOGP) rule is used to combine the two different outcomes of L2-CRC and to allocate an action label of the depth map sequence. Finally, our proposed framework is evaluated on three benchmark datasets named MSR-action 3D dataset, DHA dataset, and UTD-MHAD dataset. We compare the experimental results of our proposed framework with state-of-the-art approaches to prove the effectiveness of the proposed framework. The computational efficiency of the framework is also analyzed for all the datasets to check whether it is suitable for real-time operation or not.

Download Full-text

Human action recognition using Adaptive Hierarchical Depth Motion Maps and Gabor filter

2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) ◽

10.1109/icassp.2017.7952393 ◽

2017 ◽

Cited By ~ 11

Author(s):

Hong Liu ◽

Qinqin He ◽

Mengyuan Liu

Keyword(s):

Action Recognition ◽

Gabor Filter ◽

Human Action Recognition ◽

Human Action ◽

Depth Motion Maps

Download Full-text

Improved Collaborative Representation Classifier Based on l2-Regularized for Human Action Recognition

Journal of Electrical and Computer Engineering ◽

10.1155/2017/8191537 ◽

2017 ◽

Vol 2017 ◽

pp. 1-6

Author(s):

Shirui Huo ◽

Tianrui Hu ◽

Ce Li

Keyword(s):

Action Recognition ◽

Human Action Recognition ◽

Test Sample ◽

Human Action ◽

Superior Performance ◽

Depth Image ◽

Collaborative Representation ◽

Depth Images ◽

Spatiotemporal Information ◽

Depth Motion Maps

Human action recognition is an important recent challenging task. Projecting depth images onto three depth motion maps (DMMs) and extracting deep convolutional neural network (DCNN) features are discriminant descriptor features to characterize the spatiotemporal information of a specific action from a sequence of depth images. In this paper, a unified improved collaborative representation framework is proposed in which the probability that a test sample belongs to the collaborative subspace of all classes can be well defined and calculated. The improved collaborative representation classifier (ICRC) based on l2-regularized for human action recognition is presented to maximize the likelihood that a test sample belongs to each class, then theoretical investigation into ICRC shows that it obtains a final classification by computing the likelihood for each class. Coupled with the DMMs and DCNN features, experiments on depth image-based action recognition, including MSRAction3D and MSRGesture3D datasets, demonstrate that the proposed approach successfully using a distance-based representation classifier achieves superior performance over the state-of-the-art methods, including SRC, CRC, and SVM.

Download Full-text