scholarly journals 6DoF Pose Estimation of Transparent Object from a Single RGB-D Image

Sensors ◽  
2020 ◽  
Vol 20 (23) ◽  
pp. 6790
Author(s):  
Chi Xu ◽  
Jiale Chen ◽  
Mengyang Yao ◽  
Jun Zhou ◽  
Lijun Zhang ◽  
...  
Keyword(s):  
Pose Estimation ◽  
Point Cloud ◽  
State Of The Art ◽  
Transparent Material ◽  
Automatic Driving ◽  
Second Stage ◽  
Transparent Objects ◽  
Object Pose Estimation ◽  
Transparent Object ◽  
Extended Point

6DoF object pose estimation is a foundation for many important applications, such as robotic grasping, automatic driving, and so on. However, it is very challenging to estimate 6DoF pose of transparent object which is commonly seen in our daily life, because the optical characteristics of transparent material lead to significant depth error which results in false estimation. To solve this problem, a two-stage approach is proposed to estimate 6DoF pose of transparent object from a single RGB-D image. In the first stage, the influence of the depth error is eliminated by transparent segmentation, surface normal recovering, and RANSAC plane estimation. In the second stage, an extended point-cloud representation is presented to accurately and efficiently estimate object pose. As far as we know, it is the first deep learning based approach which focuses on 6DoF pose estimation of transparent objects from a single RGB-D image. Experimental results show that the proposed approach can effectively estimate 6DoF pose of transparent object, and it out-performs the state-of-the-art baselines by a large margin.

scholarly journals Coarse-to-Fine Hand–Object Pose Estimation with Interaction-Aware Graph Convolutional Network

Sensors ◽  
2021 ◽  
Vol 21 (23) ◽  
pp. 8092
Author(s):  
Maomao Zhang ◽  
Ao Li ◽  
Honglei Liu ◽  
Minghui Wang
Keyword(s):  
Object Relations ◽  
Pose Estimation ◽  
State Of The Art ◽  
Convolutional Network ◽  
Specific Relation ◽  
Key Factor ◽  
Rgb Images ◽  
Object Pose Estimation ◽  
Coarse To Fine ◽  
Rgb Image

The analysis of hand–object poses from RGB images is important for understanding and imitating human behavior and acts as a key factor in various applications. In this paper, we propose a novel coarse-to-fine two-stage framework for hand–object pose estimation, which explicitly models hand–object relations in 3D pose refinement rather than in the process of converting 2D poses to 3D poses. Specifically, in the coarse stage, 2D heatmaps of hand and object keypoints are obtained from RGB image and subsequently fed into pose regressor to derive coarse 3D poses. As for the fine stage, an interaction-aware graph convolutional network called InterGCN is introduced to perform pose refinement by fully leveraging the hand–object relations in 3D context. One major challenge in 3D pose refinement lies in the fact that relations between hand and object change dynamically according to different HOI scenarios. In response to this issue, we leverage both general and interaction-specific relation graphs to significantly enhance the capacity of the network to cover variations of HOI scenarios for successful 3D pose refinement. Extensive experiments demonstrate state-of-the-art performance of our approach on benchmark hand–object datasets.

scholarly journals From IR Images to Point Clouds to Pose: Point Cloud-Based AR Glasses Pose Estimation

2021 ◽  
Vol 7 (5) ◽  
pp. 80
Author(s):  
Ahmet Firintepe ◽  
Carolin Vey ◽  
Stylianos Asteriadis ◽  
Alain Pagani ◽  
Didier Stricker
Keyword(s):  
Pose Estimation ◽  
Point Cloud ◽  
State Of The Art ◽  
Estimation Error ◽  
Point Clouds ◽  
Intermediate Representation ◽  
Infrared Images ◽  
Estimation Algorithms ◽  
3D Point Clouds ◽  
Model Combining

In this paper, we propose two novel AR glasses pose estimation algorithms from single infrared images by using 3D point clouds as an intermediate representation. Our first approach “PointsToRotation” is based on a Deep Neural Network alone, whereas our second approach “PointsToPose” is a hybrid model combining Deep Learning and a voting-based mechanism. Our methods utilize a point cloud estimator, which we trained on multi-view infrared images in a semi-supervised manner, generating point clouds based on one image only. We generate a point cloud dataset with our point cloud estimator using the HMDPose dataset, consisting of multi-view infrared images of various AR glasses with the corresponding 6-DoF poses. In comparison to another point cloud-based 6-DoF pose estimation named CloudPose, we achieve an error reduction of around 50%. Compared to a state-of-the-art image-based method, we reduce the pose estimation error by around 96%.

scholarly journals Iterative Pose Refinement for Object Pose Estimation Based on RGBD Data

Sensors ◽  
2020 ◽  
Vol 20 (15) ◽  
pp. 4114
Author(s):  
Shao-Kang Huang ◽  
Chen-Chien Hsu ◽  
Wei-Yen Wang ◽  
Cheng-Hung Lin
Keyword(s):  
Augmented Reality ◽  
Pose Estimation ◽  
State Of The Art ◽  
Iterative Refinement ◽  
Accurate Estimation ◽  
Geometric Features ◽  
Wide Range ◽  
Final Output ◽  
Refinement Process ◽  
Object Pose Estimation

Accurate estimation of 3D object pose is highly desirable in a wide range of applications, such as robotics and augmented reality. Although significant advancement has been made for pose estimation, there is room for further improvement. Recent pose estimation systems utilize an iterative refinement process to revise the predicted pose to obtain a better final output. However, such refinement process only takes account of geometric features for pose revision during the iteration. Motivated by this approach, this paper designs a novel iterative refinement process that deals with both color and geometric features for object pose refinement. Experiments show that the proposed method is able to reach 94.74% and 93.2% in ADD(-S) metric with only 2 iterations, outperforming the state-of-the-art methods on the LINEMOD and YCB-Video datasets, respectively.

scholarly journals Maximum Sum of Evidence—An Evidence-Based Solution to Object Pose Estimation in Point Cloud Data

Sensors ◽  
2021 ◽  
Vol 21 (19) ◽  
pp. 6473
Author(s):  
Tyson Phillips ◽  
Tim D’Adamo ◽  
Peter McAree
Keyword(s):  
Pose Estimation ◽  
Point Cloud ◽  
Joint Space ◽  
Object Identification ◽  
Sensor Calibration ◽  
Evidence Based ◽  
Point Cloud Data ◽  
Cloud Data ◽  
Object Pose Estimation ◽  
2D And 3D

The capability to estimate the pose of known geometry from point cloud data is a frequently arising requirement in robotics and automation applications. This problem is directly addressed by Iterative Closest Point (ICP), however, this method has several limitations and lacks robustness. This paper makes the case for an alternative method that seeks to find the most likely solution based on available evidence. Specifically, an evidence-based metric is described that seeks to find the pose of the object that would maximise the conditional likelihood of reproducing the observed range measurements. A seedless search heuristic is also provided to find the most likely pose estimate in light of these measurements. The method is demonstrated to provide for pose estimation (2D and 3D shape poses as well as joint-space searches), object identification/classification, and platform localisation. Furthermore, the method is shown to be robust in cluttered or non-segmented point cloud data as well as being robust to measurement uncertainty and extrinsic sensor calibration.

scholarly journals DRNet: A Depth-Based Regression Network for 6D Object Pose Estimation

Sensors ◽  
2021 ◽  
Vol 21 (5) ◽  
pp. 1692
Author(s):  
Lei Jin ◽  
Xiaojuan Wang ◽  
Mingshu He ◽  
Jingyue Wang
Keyword(s):  
Pose Estimation ◽  
State Of The Art ◽  
Region Of Interest ◽  
Depth Map ◽  
Depth Information ◽  
Challenging Problem ◽  
Optimization Framework ◽  
Art Object ◽  
Object Pose Estimation ◽  
Difficult Cases

This paper focuses on 6Dof object pose estimation from a single RGB image. We tackle this challenging problem with a two-stage optimization framework. More specifically, we first introduce a translation estimation module to provide an initial translation based on an estimated depth map. Then, a pose regression module combines the ROI (Region of Interest) and the original image to predict the rotation and refine the translation. Compared with previous end-to-end methods that directly predict rotations and translations, our method can utilize depth information as weak guidance and significantly reduce the searching space for the subsequent module. Furthermore, we design a new loss function function for symmetric objects, an approach that has handled such exceptionally difficult cases in prior works. Experiments show that our model achieves state-of-the-art object pose estimation for the YCB- video dataset (Yale-CMU-Berkeley).

scholarly journals A New Multi-Person Pose Estimation Method Using the Partitioned CenterPose Network

2021 ◽  
Vol 11 (9) ◽  
pp. 4241
Author(s):  
Jiahua Wu ◽  
Hyo Jong Lee
Keyword(s):  
Pose Estimation ◽  
Human Body ◽  
State Of The Art ◽  
Estimation Method ◽  
Bottom Up ◽  
Center Point ◽  
Novel Approach ◽  
Body Joints

In bottom-up multi-person pose estimation, grouping joint candidates into the appropriately structured corresponding instance of a person is challenging. In this paper, a new bottom-up method, the Partitioned CenterPose (PCP) Network, is proposed to better cluster the detected joints. To achieve this goal, we propose a novel approach called Partition Pose Representation (PPR) which integrates the instance of a person and its body joints based on joint offset. PPR leverages information about the center of the human body and the offsets between that center point and the positions of the body’s joints to encode human poses accurately. To enhance the relationships between body joints, we divide the human body into five parts, and then, we generate a sub-PPR for each part. Based on this PPR, the PCP Network can detect people and their body joints simultaneously, then group all body joints according to joint offset. Moreover, an improved l1 loss is designed to more accurately measure joint offset. Using the COCO keypoints and CrowdPose datasets for testing, it was found that the performance of the proposed method is on par with that of existing state-of-the-art bottom-up methods in terms of accuracy and speed.

scholarly journals RobotP: A Benchmark Dataset for 6D Object Pose Estimation

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1299
Author(s):  
Honglin Yuan ◽  
Tim Hoogenkamp ◽  
Remco C. Veltkamp
Keyword(s):  
Pose Estimation ◽  
Ground Truth ◽  
3D Models ◽  
Depth Image ◽  
Great Success ◽  
Estimation Algorithms ◽  
Depth Images ◽  
Object Pose Estimation ◽  
Image Pairs ◽  
Bounding Boxes

Deep learning has achieved great success on robotic vision tasks. However, when compared with other vision-based tasks, it is difficult to collect a representative and sufficiently large training set for six-dimensional (6D) object pose estimation, due to the inherent difficulty of data collection. In this paper, we propose the RobotP dataset consisting of commonly used objects for benchmarking in 6D object pose estimation. To create the dataset, we apply a 3D reconstruction pipeline to produce high-quality depth images, ground truth poses, and 3D models for well-selected objects. Subsequently, based on the generated data, we produce object segmentation masks and two-dimensional (2D) bounding boxes automatically. To further enrich the data, we synthesize a large number of photo-realistic color-and-depth image pairs with ground truth 6D poses. Our dataset is freely distributed to research groups by the Shape Retrieval Challenge benchmark on 6D pose estimation. Based on our benchmark, different learning-based approaches are trained and tested by the unified dataset. The evaluation results indicate that there is considerable room for improvement in 6D object pose estimation, particularly for objects with dark colors, and photo-realistic images are helpful in increasing the performance of pose estimation algorithms.

Sign in / Sign up

Export Citation Format

Share Document