MEMe: A Mutually Enhanced Modeling Method for Efficient and Effective Human Pose Estimation

In this paper, a mutually enhanced modeling method (MEMe) is presented for human pose estimation, which focuses on enhancing lightweight model performance, but with low complexity. To obtain higher accuracy, a traditional model scale is largely expanded with heavy deployment difficulties. However, for a more lightweight model, there is a large performance gap compared to the former; thus, an urgent need for a way to fill it. Therefore, we propose a MEMe to reconstruct a lightweight baseline model, EffBase transferred intuitively from EfficientDet, into the efficient and effective pose (EEffPose) net, which contains three mutually enhanced modules: the Enhanced EffNet (EEffNet) backbone, the total fusion neck (TFNeck), and the final attention head (FAHead). Extensive experiments on COCO and MPII benchmarks show that our MEMe-based models reach state-of-the-art performances, with limited parameters. Specifically, in the same conditions, our EEffPose-P0 with 256 × 192 can use only 8.98 M parameters to achieve 75.4 AP on the COCO val set, which outperforms HRNet-W48, but with only 14% of its parameters.

Download Full-text

EfficientPose: Scalable single-person pose estimation

Applied Intelligence ◽

10.1007/s10489-020-01918-7 ◽

2020 ◽

Author(s):

Daniel Groos ◽

Heri Ramampiaro ◽

Espen AF Ihlen

Keyword(s):

Pose Estimation ◽

Network Architecture ◽

State Of The Art ◽

Computational Cost ◽

Real Life ◽

Low Complexity ◽

Human Pose Estimation ◽

Efficient Detection ◽

Single Person ◽

Human Pose

Abstract Single-person human pose estimation facilitates markerless movement analysis in sports, as well as in clinical applications. Still, state-of-the-art models for human pose estimation generally do not meet the requirements of real-life applications. The proliferation of deep learning techniques has resulted in the development of many advanced approaches. However, with the progresses in the field, more complex and inefficient models have also been introduced, which have caused tremendous increases in computational demands. To cope with these complexity and inefficiency challenges, we propose a novel convolutional neural network architecture, called EfficientPose, which exploits recently proposed EfficientNets in order to deliver efficient and scalable single-person pose estimation. EfficientPose is a family of models harnessing an effective multi-scale feature extractor and computationally efficient detection blocks using mobile inverted bottleneck convolutions, while at the same time ensuring that the precision of the pose configurations is still improved. Due to its low complexity and efficiency, EfficientPose enables real-world applications on edge devices by limiting the memory footprint and computational cost. The results from our experiments, using the challenging MPII single-person benchmark, show that the proposed EfficientPose models substantially outperform the widely-used OpenPose model both in terms of accuracy and computational efficiency. In particular, our top-performing model achieves state-of-the-art accuracy on single-person MPII, with low-complexity ConvNets.

Download Full-text

Recent Advances in 3D Human Pose Estimation: From Optimization to Implementation and Beyond

International Journal of Pattern Recognition and Artificial Intelligence ◽

10.1142/s0218001422550035 ◽

2022 ◽

Author(s):

Jielu Yan ◽

MingLiang Zhou ◽

Jinli Pan ◽

Meng Yin ◽

Bin Fang

Keyword(s):

Real Time ◽

Pose Estimation ◽

Recent Progress ◽

State Of The Art ◽

The State ◽

Human Pose Estimation ◽

Estimation Techniques ◽

Comprehensive Survey ◽

Human Pose ◽

3D Human Pose Estimation

3D human pose estimation describes estimating 3D articulation structure of a person from an image or a video. The technology has massive potential because it can enable tracking people and analyzing motion in real time. Recently, much research has been conducted to optimize human pose estimation, but few works have focused on reviewing 3D human pose estimation. In this paper, we offer a comprehensive survey of the state-of-the-art methods for 3D human pose estimation, referred to as pose estimation solutions, implementations on images or videos that contain different numbers of people and advanced 3D human pose estimation techniques. Furthermore, different kinds of algorithms are further subdivided into sub-categories and compared in light of different methodologies. To the best of our knowledge, this is the first such comprehensive survey of the recent progress of 3D human pose estimation and will hopefully facilitate the completion, refinement and applications of 3D human pose estimation.

Download Full-text

Designing Compact Convolutional Filters for Lightweight Human Pose Estimation

Wireless Communications and Mobile Computing ◽

10.1155/2021/1333250 ◽

2021 ◽

Vol 2021 ◽

pp. 1-11

Author(s):

Shili Niu ◽

Weihua Ou ◽

Shihua Feng ◽

Jianping Gou ◽

Fei Long ◽

...

Keyword(s):

Pose Estimation ◽

State Of The Art ◽

Computational Cost ◽

Estimation Accuracy ◽

Human Pose Estimation ◽

Model Parameters ◽

Resource Limited ◽

Benchmark Datasets ◽

Human Pose ◽

Low Computational Cost

Existing methods for human pose estimation usually use a large intermediate tensor, leading to a high computational load, which is detrimental to resource-limited devices. To solve this problem, we propose a low computational cost pose estimation network, MobilePoseNet, which includes encoder, decoder, and parallel nonmaximum suppression operation. Specifically, we design a lightweight upsampling block instead of transposing the convolution as the decoder and use the lightweight network as our downsampling part. Then, we choose the high-resolution features as the input for upsampling to reduce the number of model parameters. Finally, we propose a parallel OKS-NMS, which significantly outperforms the conventional NMS in terms of accuracy and speed. Experimental results on the benchmark datasets show that MobilePoseNet obtains almost comparable results to state-of-the-art methods with a low compilation load. Compared to SimpleBaseline, the parameter of MobilePoseNet is only 4%, while the estimation accuracy reaches 98%.

Download Full-text

2D Human Pose Estimation: New Benchmark and State of the Art Analysis

2014 IEEE Conference on Computer Vision and Pattern Recognition ◽

10.1109/cvpr.2014.471 ◽

2014 ◽

Cited By ~ 542

Author(s):

Mykhaylo Andriluka ◽

Leonid Pishchulin ◽

Peter Gehler ◽

Bernt Schiele

Keyword(s):

Pose Estimation ◽

State Of The Art ◽

Human Pose Estimation ◽

Art Analysis ◽

Human Pose

Download Full-text

Semi-Dynamic Hypergraph Neural Network for 3D Pose Estimation

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2020/109 ◽

2020 ◽

Author(s):

Shengyuan Liu ◽

Pei Lv ◽

Yuzhen Zhang ◽

Jie Fu ◽

Junjin Cheng ◽

...

Keyword(s):

Neural Network ◽

Pose Estimation ◽

State Of The Art ◽

Human Pose Estimation ◽

Body Structure ◽

Single Image ◽

3D Pose Estimation ◽

Convolutional Networks ◽

Human Pose ◽

Fixed Tree

This paper proposes a novel Semi-Dynamic Hypergraph Neural Network (SD-HNN) to estimate 3D human pose from a single image. SD-HNN adopts hypergraph to represent the human body to effectively exploit the kinematic constrains among adjacent and non-adjacent joints. Specifically, a pose hypergraph in SD-HNN has two components. One is a static hypergraph constructed according to the conventional tree body structure. The other is the semi-dynamic hypergraph representing the dynamic kinematic constrains among different joints. These two hypergraphs are combined together to be trained in an end-to-end fashion. Unlike traditional Graph Convolutional Networks (GCNs) that are based on a fixed tree structure, the SD-HNN can deal with ambiguity in human pose estimation. Experimental results demonstrate that the proposed method achieves state-of-the-art performance both on the Human3.6M and MPI-INF-3DHP datasets.

Download Full-text

Human pose, hand and mesh estimation using deep learning: a survey

The Journal of Supercomputing ◽

10.1007/s11227-021-04184-7 ◽

2022 ◽

Author(s):

Mukhiddin Toshpulatov ◽

Wookey Lee ◽

Suan Lee ◽

Arousha Haghighian Roudsari

Keyword(s):

Computer Vision ◽

Deep Learning ◽

Pose Estimation ◽

State Of The Art ◽

Significant Problem ◽

Estimation Methods ◽

Human Pose Estimation ◽

Estimation Techniques ◽

The Past ◽

Human Pose

AbstractHuman pose estimation is one of the issues that have gained many benefits from using state-of-the-art deep learning-based models. Human pose, hand and mesh estimation is a significant problem that has attracted the attention of the computer vision community for the past few decades. A wide variety of solutions have been proposed to tackle the problem. Deep Learning-based approaches have been extensively studied in recent years and used to address several computer vision problems. However, it is sometimes hard to compare these methods due to their intrinsic difference. This paper extensively summarizes the current deep learning-based 2D and 3D human pose, hand and mesh estimation methods with a single or multi-person, single or double-stage methodology-based taxonomy. The authors aim to make every step in the deep learning-based human pose, hand and mesh estimation techniques interpretable by providing readers with a readily understandable explanation. The presented taxonomy has clearly illustrated current research on deep learning-based 2D and 3D human pose, hand and mesh estimation. Moreover, it also provided dataset and evaluation metrics for both 2D and 3DHPE approaches.

Download Full-text