scholarly journals UCF-STAR: A Large Scale Still Image Dataset for Understanding Human Actions

2020 ◽  
Vol 34 (03) ◽  
pp. 2677-2684
Author(s):  
Marjaneh Safaei ◽  
Pooyan Balouchian ◽  
Hassan Foroosh
Keyword(s):  
Action Recognition ◽  
Human Body ◽  
Large Scale ◽  
Temporal Information ◽  
Training Data ◽  
Body Motion ◽  
Human Actions ◽  
Still Images ◽  
Still Image ◽  
Novel Approach

Action recognition in still images poses a great challenge due to (i) fewer available training data, (ii) absence of temporal information. To address the first challenge, we introduce a dataset for STill image Action Recognition (STAR), containing over $1M$ images across 50 different human body-motion action categories. UCF-STAR is the largest dataset in the literature for action recognition in still images. The key characteristics of UCF-STAR include (1) focusing on human body-motion rather than relatively static human-object interaction categories, (2) collecting images from the wild to benefit from a varied set of action representations, (3) appending multiple human-annotated labels per image rather than just the action label, and (4) inclusion of rich, structured and multi-modal set of metadata for each image. This departs from existing datasets, which typically provide single annotation in a smaller number of images and categories, with no metadata. UCF-STAR exposes the intrinsic difficulty of action recognition through its realistic scene and action complexity. To benchmark and demonstrate the benefits of UCF-STAR as a large-scale dataset, and to show the role of “latent” motion information in recognizing human actions in still images, we present a novel approach relying on predicting temporal information, yielding higher accuracy on 5 widely-used datasets.

scholarly journals Discovering Latent Class Labels for Multi-Label Learning

2020 ◽  
Author(s):  
Jun Huang ◽  
Linchuan Xu ◽  
Jing Wang ◽  
Lei Feng ◽  
Kenji Yamanishi
Keyword(s):  
Large Scale ◽  
Latent Class ◽  
Training Data ◽  
Data Sets ◽  
Robust Learning ◽  
Large Scale Data ◽  
Novel Approach ◽  
Fixed Set ◽  
Class Labels ◽  
Scale Data

Existing multi-label learning (MLL) approaches mainly assume all the labels are observed and construct classification models with a fixed set of target labels (known labels). However, in some real applications, multiple latent labels may exist outside this set and hide in the data, especially for large-scale data sets. Discovering and exploring the latent labels hidden in the data may not only find interesting knowledge but also help us to build a more robust learning model. In this paper, a novel approach named DLCL (i.e., Discovering Latent Class Labels for MLL) is proposed which can not only discover the latent labels in the training data but also predict new instances with the latent and known labels simultaneously. Extensive experiments show a competitive performance of DLCL against other state-of-the-art MLL approaches.

Clustering Approaches for Top-k Recommender Systems

2019 ◽  
Vol 28 (05) ◽  
pp. 1950019 ◽  
Author(s):  
Nicolás Torres ◽  
Marcelo Mendoza
Keyword(s):  
Recommender Systems ◽  
Recommender System ◽  
Large Scale ◽  
Empirical Studies ◽  
Cluster Structure ◽  
Parallel Architecture ◽  
Evaluation Framework ◽  
Training Data ◽  
Cluster Solution ◽  
Novel Approach

Clustering-based recommender systems bound the seek of similar users within small user clusters providing fast recommendations in large-scale datasets. Then groups can naturally be distributed into different data partitions scaling up in the number of users the recommender system can handle. Unfortunately, while the number of users and items included in a cluster solution increases, the performance in terms of precision of a clustering-based recommender system decreases. We present a novel approach that introduces a cluster-based distance function used for neighborhood computation. In our approach, clusters generated from the training data provide the basis for neighborhood selection. Then, to expand the search of relevant users, we use a novel measure that can exploit the global cluster structure to infer cluster-outside user’s distances. Empirical studies on five widely known benchmark datasets show that our proposal is very competitive in terms of precision, recall, and NDCG. However, the strongest point of our method relies on scalability, reaching speedups of 20× in a sequential computing evaluation framework and up to 100× in a parallel architecture. These results show that an efficient implementation of our cluster-based CF method can handle very large datasets providing also good results in terms of precision, avoiding the high computational costs involved in the application of more sophisticated techniques.

Graphical Models for Representation and Recognition of Human Actions

2010 ◽  
pp. 31-54 ◽  
Author(s):  
Pradeep Natarajan ◽  
Ramakant Nevatia
Keyword(s):  
Action Recognition ◽  
Graphical Models ◽  
Video Processing ◽  
Hierarchical Organization ◽  
Training Data ◽  
State Transitions ◽  
Human Actions ◽  
Low Level ◽  
Spatio Temporal ◽  
Representation Scheme

Building a system for recognition of human actions from video involves two key problems - 1) designing suitable low-level features that are both efficient to extract from videos and are capable of distinguishing between events 2) developing a suitable representation scheme that can bridge the large gap between low-level features and high-level event concepts, and also handle the uncertainty and errors inherent in any low-level video processing. Graphical models provide a natural framework for representing state transitions in events and also the spatio-temporal constraints between the actors and events. Hidden Markov models(HMMs) have been widely used in several action recognition applications but the basic representation has three key deficiencies: These include unrealistic models for the duration of a sub-event, not encoding interactions among multiple agents directly and not modeling the inherent hierarchical organization of these activities. Several extensions have been proposed to address one or more of these issues and have been successfully applied in various gesture and action recognition domains. More recently, conditionalrandomfields (CRF) are becoming increasingly popular since they allow complex potential functions for modeling observations and state transitions, and also produce superior performance to HMMs when sufficient training data is available. The authors will first review the various extension of these graphical models, then present the theory of inference and learning in them and finally discuss their applications in various domains.

scholarly journals Image Analysis Using Human Body Geometry and Size Proportion Science for Action Classification

2020 ◽  
Vol 10 (16) ◽  
pp. 5453
Author(s):  
Syed Muhammad Saqlain ◽  
Anwar Ghani ◽  
Imran Khan ◽  
Shahbaz Ahmed Khan Ghayyur ◽  
Shahaboddin Shamshirband  ◽  
...  
Keyword(s):  
Human Body ◽  
Commission Error ◽  
Language Recognition ◽  
Human Communication ◽  
Sign Language Recognition ◽  
Step Size ◽  
Human Actions ◽  
Still Images ◽  
Data Object ◽  
Morphological Image Processing

Gestures are one of the basic modes of human communication and are usually used to represent different actions. Automatic recognition of these actions forms the basis for solving more complex problems like human behavior analysis, video surveillance, event detection, and sign language recognition, etc. Action recognition from images is a challenging task as the key information like temporal data, object trajectory, and optical flow are not available in still images. While measuring the size of different regions of the human body i.e., step size, arms span, length of the arm, forearm, and hand, etc., provides valuable clues for identification of the human actions. In this article, a framework for classification of the human actions is presented where humans are detected and localized through faster region-convolutional neural networks followed by morphological image processing techniques. Furthermore, geometric features from human blob are extracted and incorporated into the classification rules for the six human actions i.e., standing, walking, single-hand side wave, single-hand top wave, both hands side wave, and both hands top wave. The performance of the proposed technique has been evaluated using precision, recall, omission error, and commission error. The proposed technique has been comparatively analyzed in terms of overall accuracy with existing approaches showing that it performs well in contrast to its counterparts.

Recognition of Inter-Class Variation of Human Actions in Sports Video

2021 ◽  
Author(s):  
Akila.K
Keyword(s):  
Action Recognition ◽  
Wavelet Packet ◽  
Human Action Recognition ◽  
Recognition System ◽  
Human Action ◽  
Human Actions ◽  
Linear Discriminant ◽  
Novel Approach ◽  
Current Events ◽  
Rule Set

Abstract Background: Human action recognition encompasses a scope for an automatic analysis of current events from video and has varied applications in multi-various fields. Recognizing and understanding of human actions from videos still remains a difficult downside as a result of the massive variations in human look, posture and body size inside identical category.Objective: This paper focuses on a specific issue related to inter-class variation in Human Action Recognition.Approach: To discriminate the human actions among the category, a novel approach which is based on wavelet packet transformation for feature extraction. As we are concentrating on classifying similar actions non-linearity among the features are analyzed and discriminated by Deterministic Normalized - Linear Discriminant Analysis (DN-LDA). However the major part of the recognition system relays on classification part and the dynamic feeds are classified by Hidden Markov Model at the final stage based on rule set..Conclusion: Experiments results have shown that the proposed approach is discriminative for similar human action recognition and well adapted to the inter-class variation

scholarly journals Self-Supervised Video Representation Learning with Space-Time Cubic Puzzles

2019 ◽  
Vol 33 ◽  
pp. 8545-8552 ◽  
Author(s):  
Dahun Kim ◽  
Donghyeon Cho ◽  
In So Kweon
Keyword(s):  
Action Recognition ◽  
Large Scale ◽  
State Of The Art ◽  
Representation Learning ◽  
Space Time ◽  
Temporal Relation ◽  
Still Images ◽  
Video Frames ◽  
Spatio Temporal ◽  
The Cost

Self-supervised tasks such as colorization, inpainting and zigsaw puzzle have been utilized for visual representation learning for still images, when the number of labeled images is limited or absent at all. Recently, this worthwhile stream of study extends to video domain where the cost of human labeling is even more expensive. However, the most of existing methods are still based on 2D CNN architectures that can not directly capture spatio-temporal information for video applications. In this paper, we introduce a new self-supervised task called as Space-Time Cubic Puzzles to train 3D CNNs using large scale video dataset. This task requires a network to arrange permuted 3D spatio-temporal crops. By completing Space-Time Cubic Puzzles, the network learns both spatial appearance and temporal relation of video frames, which is our final goal. In experiments, we demonstrate that our learned 3D representation is well transferred to action recognition tasks, and outperforms state-of-the-art 2D CNN-based competitors on UCF101 and HMDB51 datasets.

scholarly journals Deep Learning-Based Action Recognition Using 3D Skeleton Joints Information

Inventions ◽  
2020 ◽  
Vol 5 (3) ◽  
pp. 49
Author(s):  
Nusrat Tasnim ◽  
Md. Mahbubul Islam ◽  
Joong-Hwan Baek
Keyword(s):  
Action Recognition ◽  
Large Scale ◽  
Dimensional Space ◽  
Human Action Recognition ◽  
Human Action ◽  
Human Machine Interaction ◽  
Human Actions ◽  
3 Dimensional ◽  
3D Skeleton ◽  
Color Depth

Human action recognition has turned into one of the most attractive and demanding fields of research in computer vision and pattern recognition for facilitating easy, smart, and comfortable ways of human-machine interaction. With the witnessing of massive improvements to research in recent years, several methods have been suggested for the discrimination of different types of human actions using color, depth, inertial, and skeleton information. Despite having several action identification methods using different modalities, classifying human actions using skeleton joints information in 3-dimensional space is still a challenging problem. In this paper, we conceive an efficacious method for action recognition using 3D skeleton data. First, large-scale 3D skeleton joints information was analyzed and accomplished some meaningful pre-processing. Then, a simple straight-forward deep convolutional neural network (DCNN) was designed for the classification of the desired actions in order to evaluate the effectiveness and embonpoint of the proposed system. We also conducted prior DCNN models such as ResNet18 and MobileNetV2, which outperform existing systems using human skeleton joints information.

scholarly journals GAS-GCN: Gated Action-Specific Graph Convolutional Networks for Skeleton-Based Action Recognition

Sensors ◽  
2020 ◽  
Vol 20 (12) ◽  
pp. 3499 ◽  
Author(s):  
Wensong Chan ◽  
Zhiqiang Tian ◽  
Yang Wu
Keyword(s):  
Action Recognition ◽  
Human Body ◽  
Adjacency Matrix ◽  
Large Scale ◽  
Redundant Information ◽  
Temporal Dimension ◽  
Convolutional Networks ◽  
Effective Operation ◽  
Public Datasets ◽  
Body Joints

Skeleton-based action recognition has achieved great advances with the development of graph convolutional networks (GCNs). Many existing GCNs-based models only use the fixed hand-crafted adjacency matrix to describe the connections between human body joints. This omits the important implicit connections between joints, which contain discriminative information for different actions. In this paper, we propose an action-specific graph convolutional module, which is able to extract the implicit connections and properly balance them for each action. In addition, to filter out the useless and redundant information in the temporal dimension, we propose a simple yet effective operation named gated temporal convolution. These two major novelties ensure the superiority of our proposed method, as demonstrated on three large-scale public datasets: NTU-RGB + D, Kinetics, and NTU-RGB + D 120, and also shown in the detailed ablation studies.

scholarly journals Teacher-Student Networks with Multiple Decoders for Solving Math Word Problem

2020 ◽  
Author(s):  
Jipeng Zhang ◽  
Roy Ka-Wei Lee ◽  
Ee-Peng Lim ◽  
Wei Qin ◽  
Lei Wang ◽  
...  
Keyword(s):  
Word Problem ◽  
Large Scale ◽  
State Of The Art ◽  
Training Data ◽  
Teacher Student ◽  
Candidate Solution ◽  
Novel Approach ◽  
Art Works ◽  
Standard Solution ◽  
Final Answer

Math word problem (MWP) is challenging due to the limitation in training data where only one “standard” solution is available. MWP models often simply fit this solution rather than truly understand or solve the problem. The generalization of models (to diverse word scenarios) is thus limited. To address this problem, this paper proposes a novel approach, TSN-MD, by leveraging the teacher network to integrate the knowledge of equivalent solution expressions and then to regularize the learning behavior of the student network. In addition, we introduce the multiple-decoder student network to generate multiple candidate solution expressions by which the final answer is voted. In experiments, we conduct extensive comparisons and ablative studies on two large-scale MWP benchmarks, and show that using TSN-MD can surpass the state-of-the-art works by a large margin. More intriguingly, the visualization results demonstrate that TSN-MD not only produces correct final answers but also generates diverse equivalent expressions of the solution.

Detecting Human Actions by 3D Deformable Parts Models

2014 ◽  
Vol 981 ◽  
pp. 331-334
Author(s):  
Ming Yang ◽  
Yong Yang
Keyword(s):  
Object Detection ◽  
Action Recognition ◽  
High Performance ◽  
Human Action Recognition ◽  
Human Action ◽  
Human Actions ◽  
Still Images ◽  
Histograms Of Oriented Gradients ◽  
Deformable Part Models ◽  
Unified Method

In this paper, we introduce the high performance Deformable part models from object detection into human action recognition and localization and propose a unified method to detect action in video sequences. The Deformable part models have attracted intensive attention in the field of object detection. We generalize the approach from 2D still images to 3D spatiotemporal volumes. The human actions are described by 3D histograms of oriented gradients based features. Different poses are presented by mixture of models on different resolutions. The model autonomously selects the most discriminative 3D parts and learns their anchor positions related to the root. Empirical results on several video datasets prove the efficacy of our proposed method on both action recognition and localization.

Sign in / Sign up

Export Citation Format

Share Document