scholarly journals Turbo Learning Framework for Human-Object Interactions Recognition and Human Pose Estimation

2019 ◽  
Vol 33 ◽  
pp. 898-905 ◽  
Author(s):  
Wei Feng ◽  
Wentao Liu ◽  
Tong Li ◽  
Jing Peng ◽  
Chen Qian ◽  
...  
Keyword(s):  
Pose Estimation ◽  
Message Passing ◽  
Closed Loop ◽  
State Of The Art ◽  
Human Action ◽  
Complementary Information ◽  
Learning Framework ◽  
Human Object ◽  
Human Pose ◽  
Object Interactions

Human-object interactions (HOI) recognition and pose estimation are two closely related tasks. Human pose is an essential cue for recognizing actions and localizing the interacted objects. Meanwhile, human action and their interacted objects’ localizations provide guidance for pose estimation. In this paper, we propose a turbo learning framework to perform HOI recognition and pose estimation simultaneously. First, two modules are designed to enforce message passing between the tasks, i.e. pose aware HOI recognition module and HOI guided pose estimation module. Then, these two modules form a closed loop to utilize the complementary information iteratively, which can be trained in an end-to-end manner. The proposed method achieves the state-of-the-art performance on two public benchmarks including Verbs in COCO (V-COCO) and HICO-DET datasets.

scholarly journals Detecting human—object interaction with multi-level pairwise feature network

2020 ◽  
Author(s):  
Hanchao Liu ◽  
Tai-Jiang Mu ◽  
Xiaolei Huang
Keyword(s):  
Spatial Configuration ◽  
State Of The Art ◽  
Body Part ◽  
Human Action ◽  
Image Understanding ◽  
Semantic Level ◽  
Object Interaction ◽  
Human Object ◽  
Multi Level ◽  
Object Interactions

Abstract Human–object interaction (HOI) detection is crucial for human-centric image understanding which aims to infer ⟨human, action, object⟩ triplets within an image. Recent studies often exploit visual features and the spatial configuration of a human–object pair in order to learn the action linking the human and object in the pair. We argue that such a paradigm of pairwise feature extraction and action inference can be applied not only at the whole human and object instance level, but also at the part level at which a body part interacts with an object, and at the semantic level by considering the semantic label of an object along with human appearance and human–object spatial configuration, to infer the action. We thus propose a multi-level pairwise feature network (PFNet) for detecting human–object interactions. The network consists of three parallel streams to characterize HOI utilizing pairwise features at the above three levels; the three streams are finally fused to give the action prediction. Extensive experiments show that our proposed PFNet outperforms other state-of-the-art methods on the V-COCO dataset and achieves comparable results to the state-of-the-art on the HICO-DET dataset.

scholarly journals DGCN: Dynamic Graph Convolutional Network for Efficient Multi-Person Pose Estimation

2020 ◽  
Vol 34 (07) ◽  
pp. 11924-11931
Author(s):  
Zhongwei Qiu ◽  
Kai Qiu ◽  
Jianlong Fu ◽  
Dongmei Fu
Keyword(s):  
Pose Estimation ◽  
State Of The Art ◽  
Semantic Relations ◽  
Dynamic Graphs ◽  
Dynamic Graph ◽  
Convolutional Network ◽  
Bottom Up ◽  
Multi Level ◽  
Human Pose ◽  
Relative Gains

Multi-person pose estimation aims to detect human keypoints from images with multiple persons. Bottom-up methods for multi-person pose estimation have attracted extensive attention, owing to the good balance between efficiency and accuracy. Recent bottom-up methods usually follow the principle of keypoints localization and grouping, where relations between keypoints are the keys to group keypoints. These relations spontaneously construct a graph of keypoints, where the edges represent the relations between two nodes (i.e., keypoints). Existing bottom-up methods mainly define relations by empirically picking out edges from this graph, while omitting edges that may contain useful semantic relations. In this paper, we propose a novel Dynamic Graph Convolutional Module (DGCM) to model rich relations in the keypoints graph. Specifically, we take into account all relations (all edges of the graph) and construct dynamic graphs to tolerate large variations of human pose. The DGCM is quite lightweight, which allows it to be stacked like a pyramid architecture and learn structural relations from multi-level features. Our network with single DGCM based on ResNet-50 achieves relative gains of 3.2% and 4.8% over state-of-the-art bottom-up methods on COCO keypoints and MPII dataset, respectively.

scholarly journals Group Sparse Regression-Based Learning Model for Real-Time Depth-Based Human Action Prediction

2018 ◽  
Vol 2018 ◽  
pp. 1-7 ◽  
Author(s):  
Meng Li ◽  
Liang Yan ◽  
Qianying Wang
Keyword(s):  
Real Time ◽  
Human Action Recognition ◽  
Human Action ◽  
Group Sparsity ◽  
Human Actions ◽  
Depth Data ◽  
Human Object ◽  
Benchmark Datasets ◽  
Object Interactions ◽  
Sparse Set

This paper addresses the problem of predicting human actions in depth videos. Due to the complex spatiotemporal structure of human actions, it is difficult to infer ongoing human actions before they are fully executed. To handle this challenging issue, we first propose two new depth-based features called pairwise relative joint orientations (PRJOs) and depth patch motion maps (DPMMs) to represent the relative movements between each pair of joints and human-object interactions, respectively. The two proposed depth-based features are suitable for recognizing and predicting human actions in real-time fashion. Then, we propose a regression-based learning approach with a group sparsity inducing regularizer to learn action predictor based on the combination of PRJOs and DPMMs for a sparse set of joints. Experimental results on benchmark datasets have demonstrated that our proposed approach significantly outperforms existing methods for real-time human action recognition and prediction from depth data.

Recent Advances in 3D Human Pose Estimation: From Optimization to Implementation and Beyond

2022 ◽  
Author(s):  
Jielu Yan ◽  
MingLiang Zhou ◽  
Jinli Pan ◽  
Meng Yin ◽  
Bin Fang
Keyword(s):  
Real Time ◽  
Pose Estimation ◽  
Recent Progress ◽  
State Of The Art ◽  
The State ◽  
Human Pose Estimation ◽  
Estimation Techniques ◽  
Comprehensive Survey ◽  
Human Pose ◽  
3D Human Pose Estimation

3D human pose estimation describes estimating 3D articulation structure of a person from an image or a video. The technology has massive potential because it can enable tracking people and analyzing motion in real time. Recently, much research has been conducted to optimize human pose estimation, but few works have focused on reviewing 3D human pose estimation. In this paper, we offer a comprehensive survey of the state-of-the-art methods for 3D human pose estimation, referred to as pose estimation solutions, implementations on images or videos that contain different numbers of people and advanced 3D human pose estimation techniques. Furthermore, different kinds of algorithms are further subdivided into sub-categories and compared in light of different methodologies. To the best of our knowledge, this is the first such comprehensive survey of the recent progress of 3D human pose estimation and will hopefully facilitate the completion, refinement and applications of 3D human pose estimation.

scholarly journals Dynamic Pose Estimation Using Multiple RGB-D Cameras

Sensors ◽  
2018 ◽  
Vol 18 (11) ◽  
pp. 3865 ◽  
Author(s):  
Sungjin Hong ◽  
Yejin Kim
Keyword(s):  
Pose Estimation ◽  
Extreme Points ◽  
Graph Model ◽  
Human Action ◽  
Body Parts ◽  
Physical Constraints ◽  
Depth Images ◽  
Motion Sequence ◽  
Human Pose ◽  
Hands And Feet

Human poses are difficult to estimate due to the complicated body structure and the self-occlusion problem. In this paper, we introduce a marker-less system for human pose estimation by detecting and tracking key body parts, namely the head, hands, and feet. Given color and depth images captured by multiple red, green, blue, and depth (RGB-D) cameras, our system constructs a graph model with segmented regions from each camera and detects the key body parts as a set of extreme points based on accumulative geodesic distances in the graph. During the search process, local detection using a supervised learning model is utilized to match local body features. A final set of extreme points is selected with a voting scheme and tracked with physical constraints from the unified data received from the multiple cameras. During the tracking process, a Kalman filter-based method is introduced to reduce positional noises and to recover from a failure of tracking extremes. Our system shows an average of 87% accuracy against the commercial system, which outperforms the previous multi-Kinects system, and can be applied to recognize a human action or to synthesize a motion sequence from a few key poses using a small set of extremes as input data.

scholarly journals EfficientPose: Scalable single-person pose estimation

2020 ◽  
Author(s):  
Daniel Groos ◽  
Heri Ramampiaro ◽  
Espen AF Ihlen
Keyword(s):  
Pose Estimation ◽  
Network Architecture ◽  
State Of The Art ◽  
Computational Cost ◽  
Real Life ◽  
Low Complexity ◽  
Human Pose Estimation ◽  
Efficient Detection ◽  
Single Person ◽  
Human Pose

Abstract Single-person human pose estimation facilitates markerless movement analysis in sports, as well as in clinical applications. Still, state-of-the-art models for human pose estimation generally do not meet the requirements of real-life applications. The proliferation of deep learning techniques has resulted in the development of many advanced approaches. However, with the progresses in the field, more complex and inefficient models have also been introduced, which have caused tremendous increases in computational demands. To cope with these complexity and inefficiency challenges, we propose a novel convolutional neural network architecture, called EfficientPose, which exploits recently proposed EfficientNets in order to deliver efficient and scalable single-person pose estimation. EfficientPose is a family of models harnessing an effective multi-scale feature extractor and computationally efficient detection blocks using mobile inverted bottleneck convolutions, while at the same time ensuring that the precision of the pose configurations is still improved. Due to its low complexity and efficiency, EfficientPose enables real-world applications on edge devices by limiting the memory footprint and computational cost. The results from our experiments, using the challenging MPII single-person benchmark, show that the proposed EfficientPose models substantially outperform the widely-used OpenPose model both in terms of accuracy and computational efficiency. In particular, our top-performing model achieves state-of-the-art accuracy on single-person MPII, with low-complexity ConvNets.

scholarly journals Using DeepLabCut for 3D markerless pose estimation across species and behaviors

2018 ◽  
Author(s):  
Tanmay Nath ◽  
Alexander Mathis ◽  
An Chi Chen ◽  
Amir Patel ◽  
Matthias Bethge ◽  
...  
Keyword(s):  
Pose Estimation ◽  
User Interfaces ◽  
Deep Neural Network ◽  
State Of The Art ◽  
Estimation Algorithm ◽  
Graphical User Interfaces ◽  
Training Data ◽  
Behavioral Tracking ◽  
Human Pose ◽  
Python Package

Noninvasive behavioral tracking of animals during experiments is crucial to many scientific pursuits. Extracting the poses of animals without using markers is often essential for measuring behavioral effects in biomechanics, genetics, ethology & neuroscience. Yet, extracting detailed poses without markers in dynamically changing backgrounds has been challenging. We recently introduced an open source toolbox called DeepLabCut that builds on a state-of-the-art human pose estimation algorithm to allow a user to train a deep neural network using limited training data to precisely track user-defined features that matches human labeling accuracy. Here, with this paper we provide an updated toolbox that is self contained within a Python package that includes new features such as graphical user interfaces and active-learning based network refinement. Lastly, we provide a step-by-step guide for using DeepLabCut.

Sign in / Sign up

Export Citation Format

Share Document