scholarly journals A Review on Computer Vision-Based Methods for Human Action Recognition

2020 ◽  
Vol 6 (6) ◽  
pp. 46
Author(s):  
Mahmoud Al-Faris ◽  
John Chiverton ◽  
David Ndzi ◽  
Ahmed Isam Ahmed
Keyword(s):  
Health Care ◽  
Computer Vision ◽  
Action Recognition ◽  
State Of The Art ◽  
Human Action Recognition ◽  
Human Action ◽  
Generative Models ◽  
Future Research ◽  
Learning Technology ◽  
Recognition Systems

Human action recognition targets recognising different actions from a sequence of observations and different environmental conditions. A wide different applications is applicable to vision based action recognition research. This can include video surveillance, tracking, health care, and human–computer interaction. However, accurate and effective vision based recognition systems continue to be a big challenging area of research in the field of computer vision. This review introduces the most recent human action recognition systems and provides the advances of state-of-the-art methods. To this end, the direction of this research is sorted out from hand-crafted representation based methods including holistic and local representation methods with various sources of data, to a deep learning technology including discriminative and generative models and multi-modality based methods. Next, the most common datasets of human action recognition are presented. This review introduces several analyses, comparisons and recommendations that help to find out the direction of future research.

scholarly journals Feature Fusion of Deep Spatial Features and Handcrafted Spatiotemporal Features for Human Action Recognition

Sensors ◽  
2019 ◽  
Vol 19 (7) ◽  
pp. 1599 ◽  
Author(s):  
Md Uddin ◽  
Young-Koo Lee
Keyword(s):  
Action Recognition ◽  
State Of The Art ◽  
Human Action Recognition ◽  
Human Action ◽  
Support Vector ◽  
Feature Descriptor ◽  
Weber’S Law ◽  
Weber's Law ◽  
Spatiotemporal Features ◽  
Spatial Features

Human action recognition plays a significant part in the research community due to its emerging applications. A variety of approaches have been proposed to resolve this problem, however, several issues still need to be addressed. In action recognition, effectively extracting and aggregating the spatial-temporal information plays a vital role to describe a video. In this research, we propose a novel approach to recognize human actions by considering both deep spatial features and handcrafted spatiotemporal features. Firstly, we extract the deep spatial features by employing a state-of-the-art deep convolutional network, namely Inception-Resnet-v2. Secondly, we introduce a novel handcrafted feature descriptor, namely Weber’s law based Volume Local Gradient Ternary Pattern (WVLGTP), which brings out the spatiotemporal features. It also considers the shape information by using gradient operation. Furthermore, Weber’s law based threshold value and the ternary pattern based on an adaptive local threshold is presented to effectively handle the noisy center pixel value. Besides, a multi-resolution approach for WVLGTP based on an averaging scheme is also presented. Afterward, both these extracted features are concatenated and feed to the Support Vector Machine to perform the classification. Lastly, the extensive experimental analysis shows that our proposed method outperforms state-of-the-art approaches in terms of accuracy.

scholarly journals Human Action Recognition Using Improved Salient Dense Trajectories

2016 ◽  
Vol 2016 ◽  
pp. 1-11 ◽  
Author(s):  
Qingwu Li ◽  
Haisu Cheng ◽  
Yan Zhou ◽  
Guanying Huo
Keyword(s):  
Action Recognition ◽  
State Of The Art ◽  
Human Action Recognition ◽  
Human Action ◽  
Interest Points ◽  
Dense Trajectories ◽  
Dense Trajectory ◽  
Sparse Coefficient ◽  
Active Research ◽  
Motion Saliency

Human action recognition in videos is a topic of active research in computer vision. Dense trajectory (DT) features were shown to be efficient for representing videos in state-of-the-art approaches. In this paper, we present a more effective approach of video representation using improved salient dense trajectories: first, detecting the motion salient region and extracting the dense trajectories by tracking interest points in each spatial scale separately and then refining the dense trajectories via the analysis of the motion saliency. Then, we compute several descriptors (i.e., trajectory displacement, HOG, HOF, and MBH) in the spatiotemporal volume aligned with the trajectories. Finally, in order to represent the videos better, we optimize the framework of bag-of-words according to the motion salient intensity distribution and the idea of sparse coefficient reconstruction. Our architecture is trained and evaluated on the four standard video actions datasets of KTH, UCF sports, HMDB51, and UCF50, and the experimental results show that our approach performs competitively comparing with the state-of-the-art results.

Human Action Recognition Using Spatio-Temporal Multiplier Network and Attentive Correlated Temporal Feature

2021 ◽  
Author(s):  
C. Indhumathi ◽  
V. Murugan ◽  
G. Muthulakshmii
Keyword(s):  
Action Recognition ◽  
State Of The Art ◽  
Human Action Recognition ◽  
Human Action ◽  
Regional Correlation ◽  
Temporal Features ◽  
Adaptive Motion ◽  
Spatio Temporal ◽  
Inter Frame ◽  
Temporal Feature

Nowadays, action recognition has gained more attention from the computer vision community. Normally for recognizing human actions, spatial and temporal features are extracted. Two-stream convolutional neural network is used commonly for human action recognition in videos. In this paper, Adaptive motion Attentive Correlated Temporal Feature (ACTF) is used for temporal feature extractor. The temporal average pooling in inter-frame is used for extracting the inter-frame regional correlation feature and mean feature. This proposed method has better accuracy of 96.9% for UCF101 and 74.6% for HMDB51 datasets, respectively, which are higher than the other state-of-the-art methods.

Human Action Recognition Based on Improved MIL

2015 ◽  
Vol 713-715 ◽  
pp. 2152-2155 ◽  
Author(s):  
Shao Ping Zhu
Keyword(s):  
Computer Vision ◽  
Action Recognition ◽  
Heuristic Algorithm ◽  
Human Action Recognition ◽  
Human Action ◽  
Image Sequences ◽  
Multiple Instance Learning ◽  
New Method ◽  
Human Actions ◽  
Human Actions Recognition

According to the problem that achieves robust human actions recognition from image sequences in computer vision, using the Iterative Querying Heuristic algorithm as a guide, a improved Multiple Instance Learning (MIL) method is proposed for human action recognition in video image sequences. Experiments show that the new method can quickly recognize human actions and achieve high recognition rates, and on the Weizmann database validate our analysis.

scholarly journals An Efficient Human Instance-Guided Framework for Video Action Recognition

Sensors ◽  
2021 ◽  
Vol 21 (24) ◽  
pp. 8309
Author(s):  
Inwoong Lee ◽  
Doyoung Kim ◽  
Dongyoon Wee ◽  
Sanghoon Lee
Keyword(s):  
Action Recognition ◽  
Temporal Dynamics ◽  
State Of The Art ◽  
Confusion Matrix ◽  
Human Action Recognition ◽  
Human Action ◽  
Stream Networks ◽  
Comparable Performance ◽  
Motion Features ◽  
Temporal Action

In recent years, human action recognition has been studied by many computer vision researchers. Recent studies have attempted to use two-stream networks using appearance and motion features, but most of these approaches focused on clip-level video action recognition. In contrast to traditional methods which generally used entire images, we propose a new human instance-level video action recognition framework. In this framework, we represent the instance-level features using human boxes and keypoints, and our action region features are used as the inputs of the temporal action head network, which makes our framework more discriminative. We also propose novel temporal action head networks consisting of various modules, which reflect various temporal dynamics well. In the experiment, the proposed models achieve comparable performance with the state-of-the-art approaches on two challenging datasets. Furthermore, we evaluate the proposed features and networks to verify the effectiveness of them. Finally, we analyze the confusion matrix and visualize the recognized actions at human instance level when there are several people.

scholarly journals EduNet: A New Video Dataset for Understanding Human Activity in the Classroom Environment

Sensors ◽  
2021 ◽  
Vol 21 (17) ◽  
pp. 5699
Author(s):  
Vijeta Sharma ◽  
Manjari Gupta ◽  
Ajai Kumar ◽  
Deepti Mishra
Keyword(s):  
Action Recognition ◽  
Human Activity ◽  
Classroom Environment ◽  
Total Duration ◽  
Human Action Recognition ◽  
Human Action ◽  
Research Area ◽  
Future Research ◽  
Student Activities ◽  
Novel Approach

Human action recognition in videos has become a popular research area in artificial intelligence (AI) technology. In the past few years, this research has accelerated in areas such as sports, daily activities, kitchen activities, etc., due to developments in the benchmarks proposed for human action recognition datasets in these areas. However, there is little research in the benchmarking datasets for human activity recognition in educational environments. Therefore, we developed a dataset of teacher and student activities to expand the research in the education domain. This paper proposes a new dataset, called EduNet, for a novel approach towards developing human action recognition datasets in classroom environments. EduNet has 20 action classes, containing around 7851 manually annotated clips extracted from YouTube videos, and recorded in an actual classroom environment. Each action category has a minimum of 200 clips, and the total duration is approximately 12 h. To the best of our knowledge, EduNet is the first dataset specially prepared for classroom monitoring for both teacher and student activities. It is also a challenging dataset of actions as it has many clips (and due to the unconstrained nature of the clips). We compared the performance of the EduNet dataset with benchmark video datasets UCF101 and HMDB51 on a standard I3D-ResNet-50 model, which resulted in 72.3% accuracy. The development of a new benchmark dataset for the education domain will benefit future research concerning classroom monitoring systems. The EduNet dataset is a collection of classroom activities from 1 to 12 standard schools.

Automatic Learning of Articulated Skeletons Based on Mean of 3D Joints for Efficient Action Recognition

2017 ◽  
Vol 31 (04) ◽  
pp. 1750008 ◽  
Author(s):  
Abdelouahid Ben Tamou ◽  
Lahoucine Ballihi ◽  
Driss Aboutajdine
Keyword(s):  
Action Recognition ◽  
Feature Vector ◽  
State Of The Art ◽  
Human Action Recognition ◽  
Human Action ◽  
Action Classification ◽  
New Approach ◽  
Temporal Aspects ◽  
The Mean ◽  
Mean Function

In this paper, we present a new approach for human action recognition using [Formula: see text] skeleton joints recovered from RGB-D cameras. We propose a descriptor based on differences of skeleton joints. This descriptor combines two characteristics including static posture and overall dynamics that encode spatial and temporal aspects. Then, we apply the mean function on these characteristics in order to form the feature vector, used as an input to Random Forest classifier for action classification. The experimental results on both datasets: MSR Action 3D dataset and MSR Daily Activity 3D dataset demonstrate that our approach is efficient and gives promising results compared to state-of-the-art approaches.

scholarly journals Human Action Recognition in Videos using a Robust CNN LSTM Approach

2020 ◽  
pp. 21-34
Author(s):  
Carlos Ismael Orozco ◽  
Eduardo Xamena ◽  
María Elena Buemi ◽  
Julio Jacobo Berlles
Keyword(s):  
Neural Network ◽  
Computer Vision ◽  
Convolutional Neural Network ◽  
Action Recognition ◽  
Human Action Recognition ◽  
Public Spaces ◽  
Human Action ◽  
Multimedia Indexing ◽  
Potential Applications ◽  
Evaluation Metric

Action recognition in videos is currently a topic of interest in the area of computer vision, due to potential applications such as: multimedia indexing, surveillance in public spaces, among others. In this paper we propose (1) Implement a CNN–LSTM architecture. First, a pre-trained VGG16 convolutional neural network extracts the features of the input video. Then, an LSTM classifies the video in a particular class. (2) Study how the number of LSTM units affects the performance of the system. To carry out the training and test phases, we used the KTH, UCF-11 and HMDB-51 datasets. (3) Evaluate the performance of our system using accuracy as evaluation metric. We obtain 93%, 91% and 47% accuracy respectively for each dataset. 

Human Action Recognition Based on a Spatio-Temporal Video Autoencoder

2020 ◽  
Vol 34 (11) ◽  
pp. 2040001
Author(s):  
Anderson Carlos Sousa e Santos ◽  
Helio Pedrini
Keyword(s):  
Action Recognition ◽  
Low Cost ◽  
Human Action Recognition ◽  
Human Action ◽  
Video Sequences ◽  
Computer Vision Applications ◽  
Recognition Systems ◽  
Spatio Temporal ◽  
Human Operators ◽  
Accuracy Rates

Due to rapid advances in the development of surveillance cameras with high sampling rates, low cost, small size and high resolution, video-based action recognition systems have become more commonly used in various computer vision applications. Human operators can be supported with the aid of such systems to detect events of interest in video sequences, improving recognition results and reducing failure cases. In this work, we propose and evaluate a method to learn two-dimensional (2D) representations from video sequences based on an autoencoder framework. Spatial and temporal information is explored through a multi-stream convolutional neural network in the context of human action recognition. Experimental results on the challenging UCF101 and HMDB51 datasets demonstrate that our representation is capable of achieving competitive accuracy rates when compared to other approaches available in the literature.

scholarly journals Comparison between Recurrent Networks and Temporal Convolutional Networks Approaches for Skeleton-Based Action Recognition

Sensors ◽  
2021 ◽  
Vol 21 (6) ◽  
pp. 2051
Author(s):  
Mihai Nan ◽  
Mihai Trăscău ◽  
Adina Magda Florea ◽  
Cezar Cătălin Iacob
Keyword(s):  
Action Recognition ◽  
State Of The Art ◽  
Human Action Recognition ◽  
Human Action ◽  
Assistive Robotics ◽  
Recurrent Networks ◽  
Human Machine Interaction ◽  
Convolutional Networks ◽  
Machine Interaction ◽  
Spatial And Temporal Characteristics

Action recognition plays an important role in various applications such as video monitoring, automatic video indexing, crowd analysis, human-machine interaction, smart homes and personal assistive robotics. In this paper, we propose improvements to some methods for human action recognition from videos that work with data represented in the form of skeleton poses. These methods are based on the most widely used techniques for this problem—Graph Convolutional Networks (GCNs), Temporal Convolutional Networks (TCNs) and Recurrent Neural Networks (RNNs). Initially, the paper explores and compares different ways to extract the most relevant spatial and temporal characteristics for a sequence of frames describing an action. Based on this comparative analysis, we show how a TCN type unit can be extended to work even on the characteristics extracted from the spatial domain. To validate our approach, we test it against a benchmark often used for human action recognition problems and we show that our solution obtains comparable results to the state-of-the-art, but with a significant increase in the inference speed.

Sign in / Sign up

Export Citation Format

Share Document