Colored Point Cloud Registration by Depth Filtering

In the last stage of colored point cloud registration, depth measurement errors hinder the achievement of accurate and visually plausible alignments. Recently, an algorithm has been proposed to extend the Iterative Closest Point (ICP) algorithm to refine the measured depth values instead of the pose between point clouds. However, the algorithm suffers from numerical instability, so a postprocessing step is needed to restrict erroneous output depth values. In this paper, we present a new algorithm with improved numerical stability. Unlike the previous algorithm heavily relying on point-to-plane distances, our algorithm constructs a cost function based on an adaptive combination of two different projected distances to prevent numerical instability. We address the problem of registering a source point cloud to the union of the source and reference point clouds. This extension allows all source points to be processed in a unified filtering framework, irrespective of the existence of their corresponding points in the reference point cloud. The extension also improves the numerical stability of using the point-to-plane distances. The experiments show that the proposed algorithm improves the registration accuracy and provides high-quality alignments of colored point clouds.

Download Full-text

3-D Point Cloud Registration Using Convolutional Neural Networks

Applied Sciences ◽

10.3390/app9163273 ◽

2019 ◽

Vol 9 (16) ◽

pp. 3273 ◽

Cited By ~ 4

Author(s):

Wen-Chung Chang ◽

Van-Toan Pham

Keyword(s):

Pose Estimation ◽

Point Cloud ◽

Full Range ◽

Point Clouds ◽

Fixed Time ◽

Training Data ◽

Data Sets ◽

Point Cloud Registration ◽

Cloud Data ◽

Model Point

This paper develops a registration architecture for the purpose of estimating relative pose including the rotation and the translation of an object in terms of a model in 3-D space based on 3-D point clouds captured by a 3-D camera. Particularly, this paper addresses the time-consuming problem of 3-D point cloud registration which is essential for the closed-loop industrial automated assembly systems that demand fixed time for accurate pose estimation. Firstly, two different descriptors are developed in order to extract coarse and detailed features of these point cloud data sets for the purpose of creating training data sets according to diversified orientations. Secondly, in order to guarantee fast pose estimation in fixed time, a seemingly novel registration architecture by employing two consecutive convolutional neural network (CNN) models is proposed. After training, the proposed CNN architecture can estimate the rotation between the model point cloud and a data point cloud, followed by the translation estimation based on computing average values. By covering a smaller range of uncertainty of the orientation compared with a full range of uncertainty covered by the first CNN model, the second CNN model can precisely estimate the orientation of the 3-D point cloud. Finally, the performance of the algorithm proposed in this paper has been validated by experiments in comparison with baseline methods. Based on these results, the proposed algorithm significantly reduces the estimation time while maintaining high precision.

Download Full-text

Evaluating Thermal Attribute Mapping Strategies for Oblique Airborne Photogrammetric System AOS-Tx8

Remote Sensing ◽

10.3390/rs12010112 ◽

2019 ◽

Vol 12 (1) ◽

pp. 112 ◽

Cited By ~ 1

Author(s):

Dong Lin ◽

Lutz Bannehr ◽

Christoph Ulrich ◽

Hans-Gerd Maas

Keyword(s):

Point Cloud ◽

Large Scale ◽

Texture Mapping ◽

Point Clouds ◽

Registration Accuracy ◽

Point Cloud Registration ◽

Thermal Imagery ◽

Processing Scheme ◽

Data Registration ◽

Thermal Cameras

Thermal imagery is widely used in various fields of remote sensing. In this study, a novel processing scheme is developed to process the data acquired by the oblique airborne photogrammetric system AOS-Tx8 consisting of four thermal cameras and four RGB cameras with the goal of large-scale area thermal attribute mapping. In order to merge 3D RGB data and 3D thermal data, registration is conducted in four steps: First, thermal and RGB point clouds are generated independently by applying structure from motion (SfM) photogrammetry to both the thermal and RGB imagery. Next, a coarse point cloud registration is performed by the support of georeferencing data (global positioning system, GPS). Subsequently, a fine point cloud registration is conducted by octree-based iterative closest point (ICP). Finally, three different texture mapping strategies are compared. Experimental results showed that the global image pose refinement outperforms the other two strategies at registration accuracy between thermal imagery and RGB point cloud. Potential building thermal leakages in large areas can be fast detected in the generated texture mapping results. Furthermore, a combination of the proposed workflow and the oblique airborne system allows for a detailed thermal analysis of building roofs and facades.

Download Full-text

Iterative K-Closest Point Algorithms for Colored Point Cloud Registration

Sensors ◽

10.3390/s20185331 ◽

2020 ◽

Vol 20 (18) ◽

pp. 5331

Author(s):

Ouk Choi ◽

Min-Gyu Park ◽

Youngbae Hwang

Keyword(s):

Real World ◽

Point Cloud ◽

Point Clouds ◽

The Other ◽

Iterative Closest Point ◽

Depth Images ◽

The Cost ◽

Soft Matching ◽

Colored Point ◽

Refinement Algorithm

We present two algorithms for aligning two colored point clouds. The two algorithms are designed to minimize a probabilistic cost based on the color-supported soft matching of points in a point cloud to their K-closest points in the other point cloud. The first algorithm, like prior iterative closest point algorithms, refines the pose parameters to minimize the cost. Assuming that the point clouds are obtained from RGB-depth images, our second algorithm regards the measured depth values as variables and minimizes the cost to obtain refined depth values. Experiments with our synthetic dataset show that our pose refinement algorithm gives better results compared to the existing algorithms. Our depth refinement algorithm is shown to achieve more accurate alignments from the outputs of the pose refinement step. Our algorithms are applied to a real-world dataset, providing accurate and visually improved results.

Download Full-text

Low Overlapping Point Cloud Registration Using Line Features Detection

Remote Sensing ◽

10.3390/rs12010061 ◽

2019 ◽

Vol 12 (1) ◽

pp. 61

Author(s):

Miloš Prokop ◽

Salman Ahmed Shaikh ◽

Kyoung-Sook Kim

Keyword(s):

Point Cloud ◽

False Positive Rate ◽

Computational Cost ◽

Point Clouds ◽

Multiple Point ◽

Point Cloud Registration ◽

Positive Rate ◽

Effective Exchange ◽

Line Features ◽

The Right

Modern robotic exploratory strategies assume multi-agent cooperation that raises a need for an effective exchange of acquired scans of the environment with the absence of a reliable global positioning system. In such situations, agents compare the scans of the outside world to determine if they overlap in some region, and if they do so, they determine the right matching between them. The process of matching multiple point-cloud scans is called point-cloud registration. Using the existing point-cloud registration approaches, a good match between any two-point-clouds is achieved if and only if there exists a large overlap between them, however, this limits the advantage of using multiple robots, for instance, for time-effective 3D mapping. Hence, a point-cloud registration approach is highly desirable if it can work with low overlapping scans. This work proposes a novel solution for the point-cloud registration problem with a very low overlapping area between the two scans. In doing so, no initial relative positions of the point-clouds are assumed. Most of the state-of-the-art point-cloud registration approaches iteratively match keypoints in the scans, which is computationally expensive. In contrast to the traditional approaches, a more efficient line-features-based point-cloud registration approach is proposed in this work. This approach, besides reducing the computational cost, avoids the problem of high false-positive rate of existing keypoint detection algorithms, which becomes especially significant in low overlapping point-cloud registration. The effectiveness of the proposed approach is demonstrated with the help of experiments.

Download Full-text

TIF-Reg: Point Cloud Registration with Transform-Invariant Features in SE(3)

Sensors ◽

10.3390/s21175778 ◽

2021 ◽

Vol 21 (17) ◽

pp. 5778

Author(s):

Baifan Chen ◽

Hong Chen ◽

Baojun Song ◽

Grace Gong

Keyword(s):

Feature Extraction ◽

Point Cloud ◽

Point Clouds ◽

Rigid Transformation ◽

Point Cloud Registration ◽

Registration Method ◽

Invariant Features ◽

Invariant Feature ◽

Deep Feature ◽

Wide Range

Three-dimensional point cloud registration (PCReg) has a wide range of applications in computer vision, 3D reconstruction and medical fields. Although numerous advances have been achieved in the field of point cloud registration in recent years, large-scale rigid transformation is a problem that most algorithms still cannot effectively handle. To solve this problem, we propose a point cloud registration method based on learning and transform-invariant features (TIF-Reg). Our algorithm includes four modules, which are the transform-invariant feature extraction module, deep feature embedding module, corresponding point generation module and decoupled singular value decomposition (SVD) module. In the transform-invariant feature extraction module, we design TIF in SE(3) (which means the 3D rigid transformation space) which contains a triangular feature and local density feature for points. It fully exploits the transformation invariance of point clouds, making the algorithm highly robust to rigid transformation. The deep feature embedding module embeds TIF into a high-dimension space using a deep neural network, further improving the expression ability of features. The corresponding point cloud is generated using an attention mechanism in the corresponding point generation module, and the final transformation for registration is calculated in the decoupled SVD module. In an experiment, we first train and evaluate the TIF-Reg method on the ModelNet40 dataset. The results show that our method keeps the root mean squared error (RMSE) of rotation within 0.5∘ and the RMSE of translation error close to 0 m, even when the rotation is up to [−180∘, 180∘] or the translation is up to [−20 m, 20 m]. We also test the generalization of our method on the TUM3D dataset using the model trained on Modelnet40. The results show that our method’s errors are close to the experimental results on Modelnet40, which verifies the good generalization ability of our method. All experiments prove that the proposed method is superior to state-of-the-art PCReg algorithms in terms of accuracy and complexity.

Download Full-text

Accuracy assessment of building point clouds automatically generated from iphone images

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprsarchives-xl-5-547-2014 ◽

2014 ◽

Vol XL-5 ◽

pp. 547-552 ◽

Cited By ~ 5

Author(s):

B. Sirmacek ◽

R. Lindenbergh

Keyword(s):

Measurement Errors ◽

Point Cloud ◽

Model Updating ◽

Accuracy Assessment ◽

Low Cost ◽

Point Clouds ◽

3D Models ◽

Map Updating ◽

Cloud Points ◽

Point To Point

Low-cost sensor generated 3D models can be useful for quick 3D urban model updating, yet the quality of the models is questionable. In this article, we evaluate the reliability of an automatic point cloud generation method using multi-view iPhone images or an iPhone video file as an input. We register such automatically generated point cloud on a TLS point cloud of the same object to discuss accuracy, advantages and limitations of the iPhone generated point clouds. For the chosen example showcase, we have classified 1.23% of the iPhone point cloud points as outliers, and calculated the mean of the point to point distances to the TLS point cloud as 0.11 m. Since a TLS point cloud might also include measurement errors and noise, we computed local noise values for the point clouds from both sources. Mean (μ) and standard deviation (&sigma;) of roughness histograms are calculated as (μ1 = 0.44 m., &sigma;1 = 0.071 m.) and (μ2 = 0.025 m., &sigma;2 = 0.037 m.) for the iPhone and TLS point clouds respectively. Our experimental results indicate possible usage of the proposed automatic 3D model generation framework for 3D urban map updating, fusion and detail enhancing, quick and real-time change detection purposes. However, further insights should be obtained first on the circumstances that are needed to guarantee a successful point cloud generation from smartphone images.

Download Full-text

TOWARDS RADIOMETRICAL ALIGNMENT OF 3D POINT CLOUDS

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-2-w3-419-2017 ◽

2017 ◽

Vol XLII-2/W3 ◽

pp. 419-424 ◽

Cited By ~ 1

Author(s):

H. A. Lauterbach ◽

D. Borrmann ◽

A. Nüchter

Keyword(s):

Point Cloud ◽

Dynamic Range ◽

Point Clouds ◽

Color Information ◽

Lighting Conditions ◽

3D Point Clouds ◽

Laser Scanners ◽

3D Scans ◽

Colored Point

3D laser scanners are typically not able to collect color information. Therefore coloring is often done by projecting photos of an additional camera to the 3D scans. The capturing process is time consuming and therefore prone to changes in the environment. The appearance of the colored point cloud is mainly effected by changes of lighting conditions and corresponding camera settings. In case of panorama images these exposure variations are typically corrected by radiometrical aligning the input images to each other. In this paper we adopt existing methods for panorama optimization in order to correct the coloring of point clouds. Therefore corresponding pixels from overlapping images are selected by using geometrically closest points of the registered 3D scans and their neighboring pixels in the images. The dynamic range of images in raw format allows for correction of large exposure differences. Two experiments demonstrate the abilities of the approach.

Download Full-text

An Automatic 3D Point Cloud Registration Method Based on Biological Vision

Applied Sciences ◽

10.3390/app11104538 ◽

2021 ◽

Vol 11 (10) ◽

pp. 4538

Author(s):

Jinbo Liu ◽

Pengyu Guo ◽

Xiaoliang Sun

Keyword(s):

Point Cloud ◽

Feature Detection ◽

Surface Deformation ◽

Point Clouds ◽

Transformation Matrix ◽

Point Cloud Registration ◽

Biological Vision ◽

Registration Method ◽

Shape Fitting ◽

Initial Values

When measuring surface deformation, because the overlap of point clouds before and after deformation is small and the accuracy of the initial value of point cloud registration cannot be guaranteed, traditional point cloud registration methods cannot be applied. In order to solve this problem, a complete solution is proposed, first, by fixing at least three cones to the target. Then, through cone vertices, initial values of the transformation matrix can be calculated. On the basis of this, the point cloud registration can be performed accurately through the iterative closest point (ICP) algorithm using the neighboring point clouds of cone vertices. To improve the automation of this solution, an accurate and automatic point cloud registration method based on biological vision is proposed. First, the three-dimensional (3D) coordinates of cone vertices are obtained through multi-view observation, feature detection, data fusion, and shape fitting. In shape fitting, a closed-form solution of cone vertices is derived on the basis of the quadratic form. Second, a random strategy is designed to calculate the initial values of the transformation matrix between two point clouds. Then, combined with ICP, point cloud registration is realized automatically and precisely. The simulation results showed that, when the intensity of Gaussian noise ranged from 0 to 1 mr (where mr denotes the average mesh resolution of the models), the rotation and translation errors of point cloud registration were less than 0.1° and 1 mr, respectively. Lastly, a camera-projector system to dynamically measure the surface deformation during ablation tests in an arc-heated wind tunnel was developed, and the experimental results showed that the measuring precision for surface deformation exceeded 0.05 mm when surface deformation was smaller than 4 mm.

Download Full-text

FINDING A GOOD FEATURE DETECTOR-DESCRIPTOR COMBINATION FOR THE 2D KEYPOINT-BASED REGISTRATION OF TLS POINT CLOUDS

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprsannals-ii-3-w5-121-2015 ◽

2015 ◽

Vol II-3/W5 ◽

pp. 121-128 ◽

Cited By ~ 16

Author(s):

S. Urban ◽

M. Weinmann

Keyword(s):

Point Cloud ◽

Laser Scanning ◽

Point Clouds ◽

Scale Invariant ◽

Point Cloud Registration ◽

Speeded Up Robust Features ◽

Extensive Evaluation ◽

Feature Correspondences ◽

Intensity Images ◽

Keypoint Detector

The automatic and accurate registration of terrestrial laser scanning (TLS) data is a topic of great interest in the domains of city modeling, construction surveying or cultural heritage. While numerous of the most recent approaches focus on keypoint-based point cloud registration relying on forward-projected 2D keypoints detected in panoramic intensity images, little attention has been paid to the selection of appropriate keypoint detector-descriptor combinations. Instead, keypoints are commonly detected and described by applying well-known methods such as the Scale Invariant Feature Transform (SIFT) or Speeded-Up Robust Features (SURF). In this paper, we present a framework for evaluating the influence of different keypoint detector-descriptor combinations on the results of point cloud registration. For this purpose, we involve five different approaches for extracting local features from the panoramic intensity images and exploit the range information of putative feature correspondences in order to define bearing vectors which, in turn, may be exploited to transfer the task of point cloud registration from the object space to the observation space. With an extensive evaluation of our framework on a standard benchmark TLS dataset, we clearly demonstrate that replacing SIFT and SURF detectors and descriptors by more recent approaches significantly alleviates point cloud registration in terms of accuracy, efficiency and robustness.

Download Full-text

MPCR-Net: Multiple Partial Point Clouds Registration Network Using a Global Template

10.20944/preprints202109.0180.v1 ◽

2021 ◽

Author(s):

Shijie Su ◽

Chao Wang ◽

Ke Chen ◽

Jian Zhang ◽

Yang Hui

Keyword(s):

Point Cloud ◽

Cloud Model ◽

Ideal Point ◽

Point Clouds ◽

3D Measurement ◽

Registration Accuracy ◽

Point Cloud Registration ◽

Point Groups ◽

Iterative Closest Point Algorithm ◽

Rigid Body Transformation

With the advancement of photoelectric technology and computer image processing technology, the visual measurement method based on point clouds is gradually applied to the 3D measurement of large workpieces. Point cloud registration is a key step in 3D measurement, and its registration accuracy directly affects the accuracy of 3D measurements. In this study, we designed a novel MPCR-Net for multiple partial point cloud registration networks. First, an ideal point cloud was extracted from the CAD model of the workpiece and was used as the global template. Next, a deep neural network was used to search for the corresponding point groups between each partial point cloud and the global template point cloud. Then, the rigid body transformation matrix was learned according to these correspondence point groups to realize the registration of each partial point cloud. Finally, the iterative closest point algorithm was used to optimize the registration results to obtain a final point cloud model of the workpiece. We conducted point cloud registration experiments on untrained models and actual workpieces, and by comparing them with existing point cloud registration methods, we verified that the MPCR-Net could improve the accuracy and robustness of the 3D point cloud registration.

Download Full-text