Weakly-Supervised 3D Hand Pose Estimation from Monocular RGB Images

Despite recent successes in hand pose estimation from RGB images or depth maps, inherent challenges remain. RGB-based methods suffer from heavy self-occlusions and depth ambiguity. Depth sensors rely heavily on distance and can only be used indoors, thus there are many limitations to the practical application of depth-based methods. The aforementioned challenges have inspired us to combine the two modalities to offset the shortcomings of the other. In this paper, we propose a novel RGB and depth information fusion network to improve the accuracy of 3D hand pose estimation, which is called CrossFuNet. Specifically, the RGB image and the paired depth map are input into two different subnetworks, respectively. The feature maps are fused in the fusion module in which we propose a completely new approach to combine the information from the two modalities. Then, the common method is used to regress the 3D key-points by heatmaps. We validate our model on two public datasets and the results reveal that our model outperforms the state-of-the-art methods.

Download Full-text

Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraints

Computer Vision – ECCV 2020 - Lecture Notes in Computer Science ◽

10.1007/978-3-030-58520-4_13 ◽

2020 ◽

pp. 211-228

Author(s):

Adrian Spurr ◽

Umar Iqbal ◽

Pavlo Molchanov ◽

Otmar Hilliges ◽

Jan Kautz

Keyword(s):

Pose Estimation ◽

Hand Pose Estimation ◽

Weakly Supervised ◽

Biomechanical Constraints ◽

Hand Pose

Download Full-text

InterNet+: A Light Network for Hand Pose Estimation

Sensors ◽

10.3390/s21206747 ◽

2021 ◽

Vol 21 (20) ◽

pp. 6747

Author(s):

Yang Liu ◽

Jie Jiang ◽

Jiahao Sun ◽

Xianghan Wang

Keyword(s):

Performance Improvement ◽

Pose Estimation ◽

Network Architecture ◽

Activation Function ◽

Depth Information ◽

Global Features ◽

Hand Pose Estimation ◽

Feature Extractor ◽

Rgb Images ◽

Hand Pose

Hand pose estimation from RGB images has always been a difficult task, owing to the incompleteness of the depth information. Moon et al. improved the accuracy of hand pose estimation by using a new network, InterNet, through their unique design. Still, the network still has potential for improvement. Based on the architecture of MobileNet v3 and MoGA, we redesigned a feature extractor that introduced the latest achievements in the field of computer vision, such as the ACON activation function and the new attention mechanism module, etc. Using these modules effectively with our network, architecture can better extract global features from an RGB image of the hand, leading to a greater performance improvement compared to InterNet and other similar networks.

Download Full-text

Grasping Hand Pose Estimation from RGB Images Using Digital Human Model by Convolutional Neural Network

Proceedings of 3DBODY.TECH 2018 - 9th International Conference and Exhibition on 3D Body Scanning and Processing Technologies, Lugano, Switzerland, 16-17 Oct. 2018 ◽

10.15221/18.154 ◽

2018 ◽

Cited By ~ 1

Author(s):

Kentaro INO ◽

Naoto IENAGA ◽

Yuta SUGIURA ◽

Hideo SAITO ◽

Natsuki MIYATA ◽

...

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Pose Estimation ◽

Human Model ◽

Hand Pose Estimation ◽

Digital Human Model ◽

Rgb Images ◽

Digital Human ◽

Hand Pose

Download Full-text

CFAM: Estimating 3D Hand Poses from a Single RGB Image with Attention

Applied Sciences ◽

10.3390/app10020618 ◽

2020 ◽

Vol 10 (2) ◽

pp. 618

Author(s):

Xianghan Wang ◽

Jie Jiang ◽

Yanming Guo ◽

Lai Kang ◽

Yingmei Wei ◽

...

Keyword(s):

Computer Vision ◽

Pose Estimation ◽

Spatial Information ◽

Image Features ◽

Estimation Methods ◽

Feature Maps ◽

Hand Pose Estimation ◽

Rgb Images ◽

Rgb Image ◽

Hand Pose

Precise 3D hand pose estimation can be used to improve the performance of human–computer interaction (HCI). Specifically, computer-vision-based hand pose estimation can make this process more natural. Most traditional computer-vision-based hand pose estimation methods use depth images as the input, which requires complicated and expensive acquisition equipment. Estimation through a single RGB image is more convenient and less expensive. Previous methods based on RGB images utilize only 2D keypoint score maps to recover 3D hand poses but ignore the hand texture features and the underlying spatial information in the RGB image, which leads to a relatively low accuracy. To address this issue, we propose a channel fusion attention mechanism that combines 2D keypoint features and RGB image features at the channel level. In particular, the proposed method replans weights by using cascading RGB images and 2D keypoint features, which enables rational planning and the utilization of various features. Moreover, our method improves the fusion performance of different types of feature maps. Multiple contrast experiments on public datasets demonstrate that the accuracy of our proposed method is comparable to the state-of-the-art accuracy.

Download Full-text