Fusing Dynamic Images and Depth Motion Maps for Action Recognition in Surveillance Systems

2019 ◽  
Vol 09 ◽  
Author(s):  
Rajat Khurana ◽  
Alok Kumar Singh Kushwaha
Keyword(s):  
Action Recognition ◽  
Depth Map ◽  
Good Choice ◽  
Surveillance Systems ◽  
Activity Detection ◽  
Learning Capability ◽  
Proposed Model ◽  
Care Activity ◽  
Increasing Demand ◽  
Depth Motion Maps

Background & Objective: Identification of human actions from video has gathered much attention in past few years. Most of the computer vision tasks such as Health Care Activity Detection, Suspicious Activity detection, Human Computer Interactions etc. are based on the principle of activity detection. Automatic labelling of activity from videos frames is known as activity detection. Motivation of this work is to use most out of the data generated from sensors and use them for recognition of classes. Recognition of actions from videos sequences is a growing field with the upcoming trends of deep neural networks. Automatic learning capability of Convolutional Neural Network (CNN) make them good choice as compared to traditional handcrafted based approaches. With the increasing demand of RGB-D sensors combination of RGB and depth data is in great demand. This work comprises of the use of dynamic images generated from RGB combined with depth map for action recognition purpose. We have experimented our approach on pre trained VGG-F model using MSR Daily activity dataset and UTD MHAD Dataset. We achieve state of the art results. To support our research, we have calculated different parameters apart from accuracy such as precision, F score, recall. Conclusion: Accordingly, the investigation confirms improvement in term of accuracy, precision, F-Score and Recall. The proposed model is 4 Stream model is prone to occlusion, used in real time and also the data from the RGB-D sensor is fully utilized.

scholarly journals Exploring 3D Human Action Recognition Using STACOG on Multi-View Depth Motion Maps Sequences

Sensors ◽  
2021 ◽  
Vol 21 (11) ◽  
pp. 3642
Author(s):  
Mohammad Farhad Bulbul ◽  
Sadiya Tabussum ◽  
Hazrat Ali ◽  
Wenli Zheng ◽  
Mi Young Lee ◽  
...  
Keyword(s):  
Action Recognition ◽  
Depth Map ◽  
Human Action Recognition ◽  
Human Action ◽  
Collaborative Representation ◽  
Auto Correlation ◽  
Time Operation ◽  
Real Time Operation ◽  
Benchmark Datasets ◽  
Depth Motion Maps

This paper proposes an action recognition framework for depth map sequences using the 3D Space-Time Auto-Correlation of Gradients (STACOG) algorithm. First, each depth map sequence is split into two sets of sub-sequences of two different frame lengths individually. Second, a number of Depth Motion Maps (DMMs) sequences from every set are generated and are fed into STACOG to find an auto-correlation feature vector. For two distinct sets of sub-sequences, two auto-correlation feature vectors are obtained and applied gradually to L2-regularized Collaborative Representation Classifier (L2-CRC) for computing a pair of sets of residual values. Next, the Logarithmic Opinion Pool (LOGP) rule is used to combine the two different outcomes of L2-CRC and to allocate an action label of the depth map sequence. Finally, our proposed framework is evaluated on three benchmark datasets named MSR-action 3D dataset, DHA dataset, and UTD-MHAD dataset. We compare the experimental results of our proposed framework with state-of-the-art approaches to prove the effectiveness of the proposed framework. The computational efficiency of the framework is also analyzed for all the datasets to check whether it is suitable for real-time operation or not.

scholarly journals Conditional Deep 3D-Convolutional Generative Adversarial Nets for RGB-D Generation

2021 ◽  
Vol 2021 ◽  
pp. 1-8
Author(s):  
Richa Sharma ◽  
Manoj Sharma ◽  
Ankit Shukla ◽  
Santanu Chaudhury
Keyword(s):  
Object Tracking ◽  
Action Recognition ◽  
Synthetic Data ◽  
Depth Map ◽  
Generative Adversarial Networks ◽  
Data Generation ◽  
Adversarial Networks ◽  
Proposed Model ◽  
Spatio Temporal ◽  
Class Labels

Generation of synthetic data is a challenging task. There are only a few significant works on RGB video generation and no pertinent works on RGB-D data generation. In the present work, we focus our attention on synthesizing RGB-D data which can further be used as dataset for various applications like object tracking, gesture recognition, and action recognition. This paper has put forward a proposal for a novel architecture that uses conditional deep 3D-convolutional generative adversarial networks to synthesize RGB-D data by exploiting 3D spatio-temporal convolutional framework. The proposed architecture can be used to generate virtually unlimited data. In this work, we have presented the architecture to generate RGB-D data conditioned on class labels. In the architecture, two parallel paths were used, one to generate RGB data and the second to synthesize depth map. The output from the two parallel paths is combined to generate RGB-D data. The proposed model is used for video generation at 30 fps (frames per second). The frame referred here is an RGB-D with the spatial resolution of 512 × 512.

Automatic Badminton Action Recognition Using RGB-D Sensor

2014 ◽  
Vol 1042 ◽  
pp. 89-93 ◽  
Author(s):  
H.Y. Ting ◽  
K.S. Sim ◽  
F.S. Abas
Keyword(s):  
Action Recognition ◽  
Support Vector Machine Classifier ◽  
Recognition Accuracy ◽  
Depth Map ◽  
Microsoft Kinect ◽  
Experimental Result ◽  
Support Vector ◽  
Validation Test ◽  
And Gender ◽  
Microsoft Kinect Sensor

This paper presents a method to recognize badminton action from depth map sequences acquired by Microsoft Kinect sensor. Badminton is one of Malaysia’s most popular, but there is still lack of research on action recognition focusing on this sport. In this research, bone orientation details of badminton players are computed and extracted in order to form a bag of quaternions feature vectors. After conversion to log-covariance matrix, the system is trained and the badminton actions are classified by a support vector machine classifier. Our experimental dataset of depth map sequences composed of 300 badminton action samples of 10 badminton actions performed by six badminton players. The dataset varies in terms of human body size, clothes, speed, and gender. Experimental result has shown that nearly 92% of average recognition accuracy (ARA) was achieved in inter-class leave one sample out cross validation test. At the same time, 86% of ARA was achieved in inter-class cross subject validation test.

scholarly journals Improved Collaborative Representation Classifier Based on l2-Regularized for Human Action Recognition

2017 ◽  
Vol 2017 ◽  
pp. 1-6
Author(s):  
Shirui Huo ◽  
Tianrui Hu ◽  
Ce Li
Keyword(s):  
Action Recognition ◽  
Human Action Recognition ◽  
Test Sample ◽  
Human Action ◽  
Superior Performance ◽  
Depth Image ◽  
Collaborative Representation ◽  
Depth Images ◽  
Spatiotemporal Information ◽  
Depth Motion Maps

Human action recognition is an important recent challenging task. Projecting depth images onto three depth motion maps (DMMs) and extracting deep convolutional neural network (DCNN) features are discriminant descriptor features to characterize the spatiotemporal information of a specific action from a sequence of depth images. In this paper, a unified improved collaborative representation framework is proposed in which the probability that a test sample belongs to the collaborative subspace of all classes can be well defined and calculated. The improved collaborative representation classifier (ICRC) based on l2-regularized for human action recognition is presented to maximize the likelihood that a test sample belongs to each class, then theoretical investigation into ICRC shows that it obtains a final classification by computing the likelihood for each class. Coupled with the DMMs and DCNN features, experiments on depth image-based action recognition, including MSRAction3D and MSRGesture3D datasets, demonstrate that the proposed approach successfully using a distance-based representation classifier achieves superior performance over the state-of-the-art methods, including SRC, CRC, and SVM.

Sign in / Sign up

Export Citation Format

Share Document