6D pose estimation with combined deep learning and 3D vision techniques for a fast and accurate object grasping

Manipulation tasks on subsea instalments require extremely precise detection and localization of objects of interest. This problem is referred to as “pose estimation”. In this work, we present a framework for detecting and predicting 6DoF pose for relevant objects (fish-tail, gauges, and valves) on a subsea panel under varying water turbidity. A deep learning model that takes 3D vision data as an input is developed, providing a more robust 6D pose estimate. Compared to the 2D vision deep learning model, the proposed method reduces rotation and translation prediction error by (−Δ0.39∘) and translation (−Δ6.5 mm), respectively, in high turbid waters. The proposed approach is able to provide object detection as well as 6D pose estimation with an average precision of 91%. The 6D pose estimation results show 2.59∘ and 6.49 cm total average deviation in rotation and translation as compared to the ground truth data on varying unseen turbidity levels. Furthermore, our approach runs at over 16 frames per second and does not require pose refinement steps. Finally, to facilitate the training of such model we also collected and automatically annotated a new underwater 6D pose estimation dataset spanning seven levels of turbidity.

Download Full-text

Pose Estimation for Non-cooperative Spacecraft based on Deep Learning

2020 39th Chinese Control Conference (CCC) ◽

10.23919/ccc50068.2020.9189253 ◽

2020 ◽

Author(s):

Wenxiu Huan ◽

Mingmin Liu ◽

Qinglei Hu

Keyword(s):

Deep Learning ◽

Pose Estimation

Download Full-text

Deep Learning-based Hand Pose Estimation from 2D Image

202020 3rd IEEE International Conference on Knowledge Innovation and Invention (ICKII) ◽

10.1109/ickii50300.2020.9318917 ◽

2020 ◽

Author(s):

Jungpil Shin ◽

Md Abdur Rahim ◽

Okuyama Yuichi ◽

Yoichi Tomioka

Keyword(s):

Deep Learning ◽

Pose Estimation ◽

Hand Pose Estimation ◽

Hand Pose

Download Full-text

A Deep Learning Method for Target Pose Estimation of Planetary Surface

Journal of Physics Conference Series ◽

10.1088/1742-6596/1948/1/012026 ◽

2021 ◽

Vol 1948 (1) ◽

pp. 012026

Author(s):

Huang Lu ◽

Mao Xiaoyan ◽

Xing Yan

Keyword(s):

Deep Learning ◽

Pose Estimation ◽

Planetary Surface ◽

Learning Method

Download Full-text

Gait analysis for early Parkinson’s disease detection based on deep learning

Current Directions in Biomedical Engineering ◽

10.1515/cdbme-2019-0003 ◽

2019 ◽

Vol 5 (1) ◽

pp. 9-12

Author(s):

Jyothsna Kondragunta ◽

Christian Wiede ◽

Gangolf Hirtz

Keyword(s):

Parkinson’S Disease ◽

Parkinson's Disease ◽

Deep Learning ◽

Gait Analysis ◽

Pose Estimation ◽

Early Identification ◽

Human Gait ◽

3D Pose Estimation ◽

Wearable Technologies ◽

Advantages And Disadvantages

AbstractBetter handling of neurological or neurodegenerative disorders such as Parkinson’s Disease (PD) is only possible with an early identification of relevant symptoms. Although the entire disease can’t be treated but the effects of the disease can be delayed with proper care and treatment. Due to this fact, early identification of symptoms for the PD plays a key role. Recent studies state that gait abnormalities are clearly evident while performing dual cognitive tasks by people suffering with PD. Researches also proved that the early identification of the abnormal gaits leads to the identification of PD in advance. Novel technologies provide many options for the identification and analysis of human gait. These technologies can be broadly classified as wearable and non-wearable technologies. As PD is more prominent in elderly people, wearable sensors may hinder the natural persons movement and is considered out of scope of this paper. Non-wearable technologies especially Image Processing (IP) approaches captures data of the person’s gait through optic sensors Existing IP approaches which perform gait analysis is restricted with the parameters such as angle of view, background and occlusions due to objects or due to own body movements. Till date there exists no researcher in terms of analyzing gait through 3D pose estimation. As deep leaning has proven efficient in 2D pose estimation, we propose an 3D pose estimation along with proper dataset. This paper outlines the advantages and disadvantages of the state-of-the-art methods in application of gait analysis for early PD identification. Furthermore, the importance of extracting the gait parameters from 3D pose estimation using deep learning is outlined.

Download Full-text

Human skill integrated motion planning of assembly manipulation for 6R industrial robot

Industrial Robot the international journal of robotics research and application ◽

10.1108/ir-09-2018-0189 ◽

2019 ◽

Vol 46 (1) ◽

pp. 171-180

Author(s):

Yi Liu ◽

Ming Cong ◽

Hang Dong ◽

Dong Liu

Keyword(s):

Deep Learning ◽

Motion Planning ◽

Industrial Robot ◽

Robotic Assembly ◽

3D Vision ◽

Content Type ◽

Human Skill ◽

Depth Images ◽

Positioning Method ◽

Assembly Technology

Purpose The purpose of this paper is to propose a new method based on three-dimensional (3D) vision technologies and human skill integrated deep learning to solve assembly positioning task such as peg-in-hole. Design/methodology/approach Hybrid camera configuration was used to provide the global and local views. Eye-in-hand mode guided the peg to be in contact with the hole plate using 3D vision in global view. When the peg was in contact with the workpiece surface, eye-to-hand mode provided the local view to accomplish peg-hole positioning based on trained CNN. Findings The results of assembly positioning experiments proved that the proposed method successfully distinguished the target hole from the other same size holes according to the CNN. The robot planned the motion according to the depth images and human skill guide line. The final positioning precision was good enough for the robot to carry out force controlled assembly. Practical implications The developed framework can have an important impact on robotic assembly positioning process, which combine with the existing force-guidance assembly technology as to build a whole set of autonomous assembly technology. Originality/value This paper proposed a new approach to the robotic assembly positioning based on 3D visual technologies and human skill integrated deep learning. Dual cameras swapping mode was used to provide visual feedback for the entire assembly motion planning process. The proposed workpiece positioning method provided an effective disturbance rejection, autonomous motion planning and increased overall performance with depth images feedback. The proposed peg-hole positioning method with human skill integrated provided the capability of target perceptual aliasing avoiding and successive motion decision for the robotic assembly manipulation.

Download Full-text

A Coarse-to-Fine Method for Estimating the Axis Pose Based on 3D Point Clouds in Robotic Cylindrical Shaft-in-Hole Assembly

Sensors ◽

10.3390/s21124064 ◽

2021 ◽

Vol 21 (12) ◽

pp. 4064

Author(s):

Can Li ◽

Ping Chen ◽

Xin Xu ◽

Xinyu Wang ◽

Aijun Yin

Keyword(s):

Pose Estimation ◽

Point Clouds ◽

Geometric Constraints ◽

Admittance Control ◽

3D Vision ◽

Axis Orientation ◽

3D Point Clouds ◽

Object Pose Estimation ◽

Traditional Approaches ◽

Coarse To Fine

In this work, we propose a novel coarse-to-fine method for object pose estimation coupled with admittance control to promote robotic shaft-in-hole assembly. Considering that traditional approaches to locate the hole by force sensing are time-consuming, we employ 3D vision to estimate the axis pose of the hole. Thus, robots can locate the target hole in both position and orientation and enable the shaft to move into the hole along the axis orientation. In our method, first, the raw point cloud of a hole is processed to acquire the keypoints. Then, a coarse axis is extracted according to the geometric constraints between the surface normals and axis. Lastly, axis refinement is performed on the coarse axis to achieve higher precision. Practical experiments verified the effectiveness of the axis pose estimation. The assembly strategy composed of axis pose estimation and admittance control was effectively applied to the robotic shaft-in-hole assembly.

Download Full-text