scholarly journals A Coarse-to-Fine Method for Estimating the Axis Pose Based on 3D Point Clouds in Robotic Cylindrical Shaft-in-Hole Assembly

Sensors ◽  
2021 ◽  
Vol 21 (12) ◽  
pp. 4064
Author(s):  
Can Li ◽  
Ping Chen ◽  
Xin Xu ◽  
Xinyu Wang ◽  
Aijun Yin
Keyword(s):  
Pose Estimation ◽  
Point Clouds ◽  
Geometric Constraints ◽  
Admittance Control ◽  
3D Vision ◽  
Axis Orientation ◽  
3D Point Clouds ◽  
Object Pose Estimation ◽  
Traditional Approaches ◽  
Coarse To Fine

In this work, we propose a novel coarse-to-fine method for object pose estimation coupled with admittance control to promote robotic shaft-in-hole assembly. Considering that traditional approaches to locate the hole by force sensing are time-consuming, we employ 3D vision to estimate the axis pose of the hole. Thus, robots can locate the target hole in both position and orientation and enable the shaft to move into the hole along the axis orientation. In our method, first, the raw point cloud of a hole is processed to acquire the keypoints. Then, a coarse axis is extracted according to the geometric constraints between the surface normals and axis. Lastly, axis refinement is performed on the coarse axis to achieve higher precision. Practical experiments verified the effectiveness of the axis pose estimation. The assembly strategy composed of axis pose estimation and admittance control was effectively applied to the robotic shaft-in-hole assembly.

A novel method for extracting skeleton of fruit tree from 3D point clouds

2020 ◽  
pp. 2050051
Author(s):  
Shenglian lu ◽  
Guo Li ◽  
Jian Wang
Keyword(s):  
Laser Scanning ◽  
Laser Scanner ◽  
Point Clouds ◽  
Fruit Trees ◽  
Fruit Tree ◽  
Tree Canopy ◽  
3D Point Clouds ◽  
Novel Method ◽  
Manual Interaction ◽  
Traditional Approaches

Tree skeleton could be useful to agronomy researchers because the skeleton describes the shape and topological structure of a tree. The phenomenon of organs’ mutual occlusion in fruit tree canopy is usually very serious, this should result in a large amount of data missing in directed laser scanning 3D point clouds from a fruit tree. However, traditional approaches can be ineffective and problematic in extracting the tree skeleton correctly when the tree point clouds contain occlusions and missing points. To overcome this limitation, we present a method for accurate and fast extracting the skeleton of fruit tree from laser scanner measured 3D point clouds. The proposed method selects the start point and endpoint of a branch from the point clouds by user’s manual interaction, then a backward searching is used to find a path from the 3D point cloud with a radius parameter as a restriction. The experimental results in several kinds of fruit trees demonstrate that our method can extract the skeleton of a leafy fruit tree with highly accuracy.

Fast template matching and pose estimation in 3D point clouds

2019 ◽  
Vol 79 ◽  
pp. 36-45 ◽  
Author(s):  
Richard Vock ◽  
Alexander Dieckmann ◽  
Sebastian Ochmann ◽  
Reinhard Klein
Keyword(s):  
Pose Estimation ◽  
Template Matching ◽  
Point Clouds ◽  
3D Point Clouds

scholarly journals Coarse-to-Fine Hand–Object Pose Estimation with Interaction-Aware Graph Convolutional Network

Sensors ◽  
2021 ◽  
Vol 21 (23) ◽  
pp. 8092
Author(s):  
Maomao Zhang ◽  
Ao Li ◽  
Honglei Liu ◽  
Minghui Wang
Keyword(s):  
Object Relations ◽  
Pose Estimation ◽  
State Of The Art ◽  
Convolutional Network ◽  
Specific Relation ◽  
Key Factor ◽  
Rgb Images ◽  
Object Pose Estimation ◽  
Coarse To Fine ◽  
Rgb Image

The analysis of hand–object poses from RGB images is important for understanding and imitating human behavior and acts as a key factor in various applications. In this paper, we propose a novel coarse-to-fine two-stage framework for hand–object pose estimation, which explicitly models hand–object relations in 3D pose refinement rather than in the process of converting 2D poses to 3D poses. Specifically, in the coarse stage, 2D heatmaps of hand and object keypoints are obtained from RGB image and subsequently fed into pose regressor to derive coarse 3D poses. As for the fine stage, an interaction-aware graph convolutional network called InterGCN is introduced to perform pose refinement by fully leveraging the hand–object relations in 3D context. One major challenge in 3D pose refinement lies in the fact that relations between hand and object change dynamically according to different HOI scenarios. In response to this issue, we leverage both general and interaction-specific relation graphs to significantly enhance the capacity of the network to cover variations of HOI scenarios for successful 3D pose refinement. Extensive experiments demonstrate state-of-the-art performance of our approach on benchmark hand–object datasets.

scholarly journals Worldwide Pose Estimation Using 3D Point Clouds

2012 ◽  
pp. 15-29 ◽  
Author(s):  
Yunpeng Li ◽  
Noah Snavely ◽  
Dan Huttenlocher ◽  
Pascal Fua
Keyword(s):  
Pose Estimation ◽  
Point Clouds ◽  
3D Point Clouds

scholarly journals Efficient 3D Objects Recognition Using Multifoveated Point Clouds

Sensors ◽  
2018 ◽  
Vol 18 (7) ◽  
pp. 2302 ◽  
Author(s):  
Fabio Oliveira ◽  
Anderson Souza ◽  
Marcelo Fernandes ◽  
Rafael Gomes ◽  
Luiz Goncalves
Keyword(s):  
Real Time ◽  
Processing Time ◽  
Feature Detection ◽  
Point Clouds ◽  
3D Vision ◽  
3D Objects ◽  
3D Point Clouds ◽  
Recognition Quality ◽  
Computationally Intensive ◽  
Current Proposal

Technological innovations in the hardware of RGB-D sensors have allowed the acquisition of 3D point clouds in real time. Consequently, various applications have arisen related to the 3D world, which are receiving increasing attention from researchers. Nevertheless, one of the main problems that remains is the demand for computationally intensive processing that required optimized approaches to deal with 3D vision modeling, especially when it is necessary to perform tasks in real time. A previously proposed multi-resolution 3D model known as foveated point clouds can be a possible solution to this problem. Nevertheless, this is a model limited to a single foveated structure with context dependent mobility. In this work, we propose a new solution for data reduction and feature detection using multifoveation in the point cloud. Nonetheless, the application of several foveated structures results in a considerable increase of processing since there are intersections between regions of distinct structures, which are processed multiple times. Towards solving this problem, the current proposal brings an approach that avoids the processing of redundant regions, which results in even more reduced processing time. Such approach can be used to identify objects in 3D point clouds, one of the key tasks for real-time applications as robotics vision, with efficient synchronization allowing the validation of the model and verification of its applicability in the context of computer vision. Experimental results demonstrate a performance gain of at least 27.21% in processing time while retaining the main features of the original, and maintaining the recognition quality rate in comparison with state-of-the-art 3D object recognition methods.

scholarly journals Review: Deep Learning on 3D Point Clouds

2020 ◽  
Vol 12 (11) ◽  
pp. 1729 ◽  
Author(s):  
Saifullahi Aminu Bello ◽  
Shangshu Yu ◽  
Cheng Wang ◽  
Jibril Muhmmad Adam ◽  
Jonathan Li
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
General Structure ◽  
Point Clouds ◽  
Autonomous Driving ◽  
Point Cloud Data ◽  
3D Vision ◽  
Cloud Data ◽  
3D Point Clouds ◽  
Learning Techniques

A point cloud is a set of points defined in a 3D metric space. Point clouds have become one of the most significant data formats for 3D representation and are gaining increased popularity as a result of the increased availability of acquisition devices, as well as seeing increased application in areas such as robotics, autonomous driving, and augmented and virtual reality. Deep learning is now the most powerful tool for data processing in computer vision and is becoming the most preferred technique for tasks such as classification, segmentation, and detection. While deep learning techniques are mainly applied to data with a structured grid, the point cloud, on the other hand, is unstructured. The unstructuredness of point clouds makes the use of deep learning for its direct processing very challenging. This paper contains a review of the recent state-of-the-art deep learning techniques, mainly focusing on raw point cloud data. The initial work on deep learning directly with raw point cloud data did not model local regions; therefore, subsequent approaches model local regions through sampling and grouping. More recently, several approaches have been proposed that not only model the local regions but also explore the correlation between points in the local regions. From the survey, we conclude that approaches that model local regions and take into account the correlation between points in the local regions perform better. Contrary to existing reviews, this paper provides a general structure for learning with raw point clouds, and various methods were compared based on the general structure. This work also introduces the popular 3D point cloud benchmark datasets and discusses the application of deep learning in popular 3D vision tasks, including classification, segmentation, and detection.

scholarly journals From IR Images to Point Clouds to Pose: Point Cloud-Based AR Glasses Pose Estimation

2021 ◽  
Vol 7 (5) ◽  
pp. 80
Author(s):  
Ahmet Firintepe ◽  
Carolin Vey ◽  
Stylianos Asteriadis ◽  
Alain Pagani ◽  
Didier Stricker
Keyword(s):  
Pose Estimation ◽  
Point Cloud ◽  
State Of The Art ◽  
Estimation Error ◽  
Point Clouds ◽  
Intermediate Representation ◽  
Infrared Images ◽  
Estimation Algorithms ◽  
3D Point Clouds ◽  
Model Combining

In this paper, we propose two novel AR glasses pose estimation algorithms from single infrared images by using 3D point clouds as an intermediate representation. Our first approach “PointsToRotation” is based on a Deep Neural Network alone, whereas our second approach “PointsToPose” is a hybrid model combining Deep Learning and a voting-based mechanism. Our methods utilize a point cloud estimator, which we trained on multi-view infrared images in a semi-supervised manner, generating point clouds based on one image only. We generate a point cloud dataset with our point cloud estimator using the HMDPose dataset, consisting of multi-view infrared images of various AR glasses with the corresponding 6-DoF poses. In comparison to another point cloud-based 6-DoF pose estimation named CloudPose, we achieve an error reduction of around 50%. Compared to a state-of-the-art image-based method, we reduce the pose estimation error by around 96%.

Sign in / Sign up

Export Citation Format

Share Document