Multi-View Hierarchical Bidirectional Recurrent Neural Network for Depth Video Sequence Based Action Recognition

2018 ◽  
Vol 32 (10) ◽  
pp. 1850033 ◽  
Author(s):  
Xueping Liu ◽  
Yibo Li ◽  
Qingjun Wang
Keyword(s):  
Action Recognition ◽  
Video Sequence ◽  
Single Layer ◽  
Human Action Recognition ◽  
Research Direction ◽  
Human Action ◽  
Depth Image ◽  
Classification Framework ◽  
Depth Video ◽  
High Computational Efficiency

Human action recognition based on depth video sequence is an important research direction in the field of computer vision. The present study proposed a classification framework based on hierarchical multi-view to resolve depth video sequence-based action recognition. Herein, considering the distinguishing feature of 3D human action space, we project the 3D human action image to three coordinate planes, so that the 3D depth image is converted to three 2D images, and then feed them to three subnets, respectively. With the increase of the number of layers, the representations of subnets are hierarchically fused to be the inputs of next layers. The final representations of the depth video sequence are fed into a single layer perceptron, and the final result is decided by the time accumulated through the output of the perceptron. We compare with other methods on two publicly available datasets, and we also verify the proposed method through the human action database acquired by our Kinect system. Our experimental results demonstrate that our model has high computational efficiency and achieves the performance of state-of-the-art method.

scholarly journals Multi-feature consultation model for human action recognition in depth video sequence

2018 ◽  
Vol 2018 (16) ◽  
pp. 1498-1502
Author(s):  
Xueping Liu ◽  
Yibo Li ◽  
Xiaoming Li ◽  
Can Tian ◽  
Yueqi Yang
Keyword(s):  
Action Recognition ◽  
Video Sequence ◽  
Human Action Recognition ◽  
Human Action ◽  
Consultation Model ◽  
Depth Video

scholarly journals 2D Information Space Based Action Recognition

2019 ◽  
Vol 9 (1S) ◽  
pp. 295-298
Keyword(s):  
Convex Hull ◽  
Action Recognition ◽  
Video Sequence ◽  
Feature Vector ◽  
Human Action Recognition ◽  
Human Action ◽  
New Approach ◽  
Human Object ◽  
Background Data ◽  
2D Data

Video based human action recognition has attained more attraction from the researchers and it predominates in the field of computer vision and pattern recognition. In this paper we deliver a new approach to suppress the background data and to extract 2D data of foreground human object of the video sequence. A combination of convex hull area, convex hull perimeter, solidity and eccentricity is used to represent the feature vector. Experiments are conducted on Weizmann video dataset to assess how the system is doing. The discriminative nature of the feature vectors assures accuracy in action recognition.

scholarly journals Using a Multilearner to Fuse Multimodal Features for Human Action Recognition

2020 ◽  
Vol 2020 ◽  
pp. 1-18
Author(s):  
Chao Tang ◽  
Huosheng Hu ◽  
Wenjian Wang ◽  
Wei Li ◽  
Hua Peng ◽  
...  
Keyword(s):  
Action Recognition ◽  
Feature Fusion ◽  
Human Action Recognition ◽  
Human Action ◽  
Human Motion ◽  
Image Features ◽  
Depth Image ◽  
Good Ability ◽  
Multimodal Features ◽  
Feature Based

The representation and selection of action features directly affect the recognition effect of human action recognition methods. Single feature is often affected by human appearance, environment, camera settings, and other factors. Aiming at the problem that the existing multimodal feature fusion methods cannot effectively measure the contribution of different features, this paper proposed a human action recognition method based on RGB-D image features, which makes full use of the multimodal information provided by RGB-D sensors to extract effective human action features. In this paper, three kinds of human action features with different modal information are proposed: RGB-HOG feature based on RGB image information, which has good geometric scale invariance; D-STIP feature based on depth image, which maintains the dynamic characteristics of human motion and has local invariance; and S-JRPF feature-based skeleton information, which has good ability to describe motion space structure. At the same time, multiple K-nearest neighbor classifiers with better generalization ability are used to integrate decision-making classification. The experimental results show that the algorithm achieves ideal recognition results on the public G3D and CAD60 datasets.

scholarly journals Improved Collaborative Representation Classifier Based on l2-Regularized for Human Action Recognition

2017 ◽  
Vol 2017 ◽  
pp. 1-6
Author(s):  
Shirui Huo ◽  
Tianrui Hu ◽  
Ce Li
Keyword(s):  
Action Recognition ◽  
Human Action Recognition ◽  
Test Sample ◽  
Human Action ◽  
Superior Performance ◽  
Depth Image ◽  
Collaborative Representation ◽  
Depth Images ◽  
Spatiotemporal Information ◽  
Depth Motion Maps

Human action recognition is an important recent challenging task. Projecting depth images onto three depth motion maps (DMMs) and extracting deep convolutional neural network (DCNN) features are discriminant descriptor features to characterize the spatiotemporal information of a specific action from a sequence of depth images. In this paper, a unified improved collaborative representation framework is proposed in which the probability that a test sample belongs to the collaborative subspace of all classes can be well defined and calculated. The improved collaborative representation classifier (ICRC) based on l2-regularized for human action recognition is presented to maximize the likelihood that a test sample belongs to each class, then theoretical investigation into ICRC shows that it obtains a final classification by computing the likelihood for each class. Coupled with the DMMs and DCNN features, experiments on depth image-based action recognition, including MSRAction3D and MSRGesture3D datasets, demonstrate that the proposed approach successfully using a distance-based representation classifier achieves superior performance over the state-of-the-art methods, including SRC, CRC, and SVM.

scholarly journals Hierarchical dynamic depth projected difference images–based action recognition in videos with convolutional neural networks

2019 ◽  
Vol 16 (1) ◽  
pp. 172988141882509 ◽  
Author(s):  
Hanbo Wu ◽  
Xin Ma ◽  
Yibin Li
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Action Recognition ◽  
Human Action Recognition ◽  
Human Action ◽  
Temporal Information ◽  
Superior Performance ◽  
Video Sequences ◽  
Depth Video ◽  
Difference Images

Temporal information plays a significant role in video-based human action recognition. How to effectively extract the spatial–temporal characteristics of actions in videos has always been a challenging problem. Most existing methods acquire spatial and temporal cues in videos individually. In this article, we propose a new effective representation for depth video sequences, called hierarchical dynamic depth projected difference images that can aggregate the action spatial and temporal information simultaneously at different temporal scales. We firstly project depth video sequences onto three orthogonal Cartesian views to capture the 3D shape and motion information of human actions. Hierarchical dynamic depth projected difference images are constructed with the rank pooling in each projected view to hierarchically encode the spatial–temporal motion dynamics in depth videos. Convolutional neural networks can automatically learn discriminative features from images and have been extended to video classification because of their superior performance. To verify the effectiveness of hierarchical dynamic depth projected difference images representation, we construct a hierarchical dynamic depth projected difference images–based action recognition framework where hierarchical dynamic depth projected difference images in three views are fed into three identical pretrained convolutional neural networks independently for finely retuning. We design three classification schemes in the framework and different schemes utilize different convolutional neural network layers to compare their effects on action recognition. Three views are combined to describe the actions more comprehensively in each classification scheme. The proposed framework is evaluated on three challenging public human action data sets. Experiments indicate that our method has better performance and can provide discriminative spatial–temporal information for human action recognition in depth videos.

A Flexible High-Level Fusion for an Accurate Human Action Recognition System

2020 ◽  
Vol 29 (12) ◽  
pp. 2050190
Author(s):  
Amel Ben Mahjoub ◽  
Mohamed Atri
Keyword(s):  
Action Recognition ◽  
Short Term Memory ◽  
Recognition Rate ◽  
Human Action Recognition ◽  
Local Binary Patterns ◽  
Low Complexity ◽  
Recognition System ◽  
Human Action ◽  
Collaborative Representation ◽  
Depth Video

Action recognition is a very effective method of computer vision areas. In the last few years, there has been a growing interest in Deep learning networks as the Long Short–Term Memory (LSTM) architectures due to their efficiency in long-term time sequence processing. In the light of these recent events in deep neural networks, there is now considerable concern about the development of an accurate action recognition approach with low complexity. This paper aims to introduce a method for learning depth activity videos based on the LSTM and the classification fusion. The first step consists in extracting compact depth video features. We start with the calculation of Depth Motion Maps (DMM) from each sequence. Then we encode and concatenate contour and texture DMM characteristics using the histogram-of-oriented-gradient and local-binary-patterns descriptors. The second step is the depth video classification based on the naive Bayes fusion approach. Training three classifiers, which are the collaborative representation classifier, the kernel-based extreme learning machine and the LSTM, is done separately to get classification scores. Finally, we fuse the classification score outputs of all classifiers with the naive Bayesian method to get a final predicted label. Our proposed method achieves a significant improvement in the recognition rate compared to previous work that has used Kinect v2 and UTD-MHAD human action datasets.

Sign in / Sign up

Export Citation Format

Share Document