Self-Supervised Point Set Local Descriptors for Point Cloud Registration

Descriptors play an important role in point cloud registration. The current state-of-the-art resorts to the high regression capability of deep learning. However, recent deep learning-based descriptors require different levels of annotation and selection of patches, which make the model hard to migrate to new scenarios. In this work, we learn local registration descriptors for point clouds in a self-supervised manner. In each iteration of the training, the input of the network is merely one unlabeled point cloud. Thus, the whole training requires no manual annotation and manual selection of patches. In addition, we propose to involve keypoint sampling into the pipeline, which further improves the performance of our model. Our experiments demonstrate the capability of our self-supervised local descriptor to achieve even better performance than the supervised model, while being easier to train and requiring no data labeling.

Download Full-text

IMPROVING PERFORMANCE OF FEATURE EXTRACTION IN SFM ALGORITHMS FOR 3D SPARSE POINT CLOUD

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-2-w17-101-2019 ◽

2019 ◽

Vol XLII-2/W17 ◽

pp. 101-106

Author(s):

F. Condorelli ◽

R. Higuchi ◽

S. Nasu ◽

F. Rinaudo ◽

H. Sugawara

Keyword(s):

Point Cloud ◽

Point Clouds ◽

Low Density ◽

Feature Points ◽

Historical Archives ◽

Before And After ◽

Manual Selection ◽

Art And Architecture ◽

Selection Of

Abstract. The use of Structure-from-Motion algorithms is a common practice to obtain a rapid photogrammetric reconstruction. However, the performance of these algorithms is limited by the fact that in some conditions the resulting point clouds present low density. This is the case when processing materials from historical archives, such as photographs and videos, which generates only sparse point clouds due to the lack of necessary information in the photogrammetric reconstruction. This paper explores ways to improve the performance of open source SfM algorithms in order to guarantee the presence of strategic feature points in the resulting point cloud, even if sparse. To reach this objective, a photogrammetric workflow is proposed to process historical images. The first part of the workflow presents a method that allows the manual selection of feature points during the photogrammetric process. The second part evaluates the metric quality of the reconstruction on the basis of a comparison with a point cloud that has a different density from the sparse point cloud. The workflow was applied to two different case studies. Transformations of wall paintings of the Karanlık church in Cappadocia were analysed thanks to the comparison of 3D model resulting from archive photographs and a recent survey. Then a comparison was performed between the state of the Komise building in Japan, before and after restoration. The findings show that the method applied allows the metric scale and evaluation of the model also in bad condition and when only low-density point clouds are available. Moreover, this tool should be of great use for both art and architecture historians and geomatics experts, to study the evolution of Cultural Heritage.

Download Full-text

Multi-Dimensional Underwater Point Cloud Detection Based on Deep Learning

Sensors ◽

10.3390/s21030884 ◽

2021 ◽

Vol 21 (3) ◽

pp. 884

Author(s):

Chia-Ming Tsai ◽

Yi-Horng Lai ◽

Yung-Da Sun ◽

Yu-Jen Chung ◽

Jau-Woei Perng

Keyword(s):

Deep Learning ◽

Point Cloud ◽

Three Dimensional ◽

Point Clouds ◽

Training Data ◽

Network Architectures ◽

Point Cloud Data ◽

Data Types ◽

Raw Data ◽

Cloud Data

Numerous sensors can obtain images or point cloud data on land, however, the rapid attenuation of electromagnetic signals and the lack of light in water have been observed to restrict sensing functions. This study expands the utilization of two- and three-dimensional detection technologies in underwater applications to detect abandoned tires. A three-dimensional acoustic sensor, the BV5000, is used in this study to collect underwater point cloud data. Some pre-processing steps are proposed to remove noise and the seabed from raw data. Point clouds are then processed to obtain two data types: a 2D image and a 3D point cloud. Deep learning methods with different dimensions are used to train the models. In the two-dimensional method, the point cloud is transferred into a bird’s eye view image. The Faster R-CNN and YOLOv3 network architectures are used to detect tires. Meanwhile, in the three-dimensional method, the point cloud associated with a tire is cut out from the raw data and is used as training data. The PointNet and PointConv network architectures are then used for tire classification. The results show that both approaches provide good accuracy.

Download Full-text

3-D Point Cloud Registration Using Convolutional Neural Networks

Applied Sciences ◽

10.3390/app9163273 ◽

2019 ◽

Vol 9 (16) ◽

pp. 3273 ◽

Cited By ~ 4

Author(s):

Wen-Chung Chang ◽

Van-Toan Pham

Keyword(s):

Pose Estimation ◽

Point Cloud ◽

Full Range ◽

Point Clouds ◽

Fixed Time ◽

Training Data ◽

Data Sets ◽

Point Cloud Registration ◽

Cloud Data ◽

Model Point

This paper develops a registration architecture for the purpose of estimating relative pose including the rotation and the translation of an object in terms of a model in 3-D space based on 3-D point clouds captured by a 3-D camera. Particularly, this paper addresses the time-consuming problem of 3-D point cloud registration which is essential for the closed-loop industrial automated assembly systems that demand fixed time for accurate pose estimation. Firstly, two different descriptors are developed in order to extract coarse and detailed features of these point cloud data sets for the purpose of creating training data sets according to diversified orientations. Secondly, in order to guarantee fast pose estimation in fixed time, a seemingly novel registration architecture by employing two consecutive convolutional neural network (CNN) models is proposed. After training, the proposed CNN architecture can estimate the rotation between the model point cloud and a data point cloud, followed by the translation estimation based on computing average values. By covering a smaller range of uncertainty of the orientation compared with a full range of uncertainty covered by the first CNN model, the second CNN model can precisely estimate the orientation of the 3-D point cloud. Finally, the performance of the algorithm proposed in this paper has been validated by experiments in comparison with baseline methods. Based on these results, the proposed algorithm significantly reduces the estimation time while maintaining high precision.

Download Full-text

Classification of Point Clouds for Indoor Components Using Few Labeled Samples

Remote Sensing ◽

10.3390/rs12142181 ◽

2020 ◽

Vol 12 (14) ◽

pp. 2181

Author(s):

Hangbin Wu ◽

Huimin Yang ◽

Shengyu Huang ◽

Doudou Zeng ◽

Chun Liu ◽

...

Keyword(s):

Deep Learning ◽

Point Cloud ◽

Point Clouds ◽

Neighborhood Search ◽

Learning Methods ◽

Semantic Classification ◽

Cloud Classification ◽

Mixed Features ◽

Indoor Scenarios ◽

Point Cloud Classification

The existing deep learning methods for point cloud classification are trained using abundant labeled samples and used to test only a few samples. However, classification tasks are diverse, and not all tasks have enough labeled samples for training. In this paper, a novel point cloud classification method for indoor components using few labeled samples is proposed to solve the problem of the requirement for abundant labeled samples for training with deep learning classification methods. This method is composed of four parts: mixing samples, feature extraction, dimensionality reduction, and semantic classification. First, the few labeled point clouds are mixed with unlabeled point clouds. Next, the mixed high-dimensional features are extracted using a deep learning framework. Subsequently, a nonlinear manifold learning method is used to embed the mixed features into a low-dimensional space. Finally, the few labeled point clouds in each cluster are identified, and semantic labels are provided for unlabeled point clouds in the same cluster by a neighborhood search strategy. The validity and versatility of the proposed method were validated by different experiments and compared with three state-of-the-art deep learning methods. Our method uses fewer than 30 labeled point clouds to achieve an accuracy that is 1.89–19.67% greater than existing methods. More importantly, the experimental results suggest that this method is not only suitable for single-attribute indoor scenarios but also for comprehensive complex indoor scenarios.

Download Full-text

Evaluating Thermal Attribute Mapping Strategies for Oblique Airborne Photogrammetric System AOS-Tx8

Remote Sensing ◽

10.3390/rs12010112 ◽

2019 ◽

Vol 12 (1) ◽

pp. 112 ◽

Cited By ~ 1

Author(s):

Dong Lin ◽

Lutz Bannehr ◽

Christoph Ulrich ◽

Hans-Gerd Maas

Keyword(s):

Point Cloud ◽

Large Scale ◽

Texture Mapping ◽

Point Clouds ◽

Registration Accuracy ◽

Point Cloud Registration ◽

Thermal Imagery ◽

Processing Scheme ◽

Data Registration ◽

Thermal Cameras

Thermal imagery is widely used in various fields of remote sensing. In this study, a novel processing scheme is developed to process the data acquired by the oblique airborne photogrammetric system AOS-Tx8 consisting of four thermal cameras and four RGB cameras with the goal of large-scale area thermal attribute mapping. In order to merge 3D RGB data and 3D thermal data, registration is conducted in four steps: First, thermal and RGB point clouds are generated independently by applying structure from motion (SfM) photogrammetry to both the thermal and RGB imagery. Next, a coarse point cloud registration is performed by the support of georeferencing data (global positioning system, GPS). Subsequently, a fine point cloud registration is conducted by octree-based iterative closest point (ICP). Finally, three different texture mapping strategies are compared. Experimental results showed that the global image pose refinement outperforms the other two strategies at registration accuracy between thermal imagery and RGB point cloud. Potential building thermal leakages in large areas can be fast detected in the generated texture mapping results. Furthermore, a combination of the proposed workflow and the oblique airborne system allows for a detailed thermal analysis of building roofs and facades.

Download Full-text

Topology Reconstruction of BIM Wall Objects from Point Cloud Data

Remote Sensing ◽

10.3390/rs12111800 ◽

2020 ◽

Vol 12 (11) ◽

pp. 1800 ◽

Cited By ~ 3

Author(s):

Maarten Bassier ◽

Maarten Vergauwen

Keyword(s):

Point Cloud ◽

Point Clouds ◽

Evaluation Framework ◽

Information Modeling ◽

Point Cloud Data ◽

Cloud Data ◽

Current State ◽

Building Information ◽

Popular Subject ◽

Best Fit

The processing of remote sensing measurements to Building Information Modeling (BIM) is a popular subject in current literature. An important step in the process is the enrichment of the geometry with the topology of the wall observations to create a logical model. However, this remains an unsolved task as methods struggle to deal with the noise, incompleteness and the complexity of point cloud data of building scenes. Current methods impose severe abstractions such as Manhattan-world assumptions and single-story procedures to overcome these obstacles, but as a result, a general data processing approach is still missing. In this paper, we propose a method that solves these shortcomings and creates a logical BIM model in an unsupervised manner. More specifically, we propose a connection evaluation framework that takes as input a set of preprocessed point clouds of a building’s wall observations and compute the best fit topology between them. We transcend the current state of the art by processing point clouds of both straight, curved and polyline-based walls. Also, we consider multiple connection types in a novel reasoning framework that decides which operations are best fit to reconstruct the topology of the walls. The geometry and topology produced by our method is directly usable by BIM processes as it is structured conform the IFC data structure. The experimental results conducted on the Stanford 2D-3D-Semantics dataset (2D-3D-S) show that the proposed method is a promising framework to reconstruct complex multi-story wall elements in an unsupervised manner.

Download Full-text

Deep Learning Point Cloud Registration based on Distance Features

IEEE Latin America Transactions ◽

10.1109/tla.2019.9011551 ◽

2019 ◽

Vol 17 (12) ◽

pp. 2053-2060 ◽

Cited By ~ 1

Author(s):

Jorge Perez-Gonzalez ◽

Fernando Luna-Madrigal ◽

Omar Pina-Ramirez

Keyword(s):

Deep Learning ◽

Point Cloud ◽

Point Cloud Registration

Download Full-text

DEM Extraction from ALS Point Clouds in Forest Areas via Graph Convolution Network

Remote Sensing ◽

10.3390/rs12010178 ◽

2020 ◽

Vol 12 (1) ◽

pp. 178 ◽

Cited By ~ 1

Author(s):

Jinming Zhang ◽

Xiangyun Hu ◽

Hengming Dai ◽

ShenRun Qu

Keyword(s):

Deep Learning ◽

Point Cloud ◽

Laser Scanning ◽

Large Scale ◽

Spatial Relationship ◽

Point Clouds ◽

Current Data ◽

Data Sampling ◽

Dynamic Graph ◽

Convolution Model

It is difficult to extract a digital elevation model (DEM) from an airborne laser scanning (ALS) point cloud in a forest area because of the irregular and uneven distribution of ground and vegetation points. Machine learning, especially deep learning methods, has shown powerful feature extraction in accomplishing point cloud classification. However, most of the existing deep learning frameworks, such as PointNet, dynamic graph convolutional neural network (DGCNN), and SparseConvNet, cannot consider the particularity of ALS point clouds. For large-scene laser point clouds, the current data preprocessing methods are mostly based on random sampling, which is not suitable for DEM extraction tasks. In this study, we propose a novel data sampling algorithm for the data preparation of patch-based training and classification named T-Sampling. T-Sampling uses the set of the lowest points in a certain area as basic points with other points added to supplement it, which can guarantee the integrity of the terrain in the sampling area. In the learning part, we propose a new convolution model based on terrain named Tin-EdgeConv that fully considers the spatial relationship between ground and non-ground points when constructing a directed graph. We design a new network based on Tin-EdgeConv to extract local features and use PointNet architecture to extract global context information. Finally, we combine this information effectively with a designed attention fusion module. These aspects are important in achieving high classification accuracy. We evaluate the proposed method by using large-scale data from forest areas. Results show that our method is more accurate than existing algorithms.

Download Full-text

Voxel-based 3D Point Cloud Semantic Segmentation: Unsupervised Geometric and Relationship Featuring vs Deep Learning Methods

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi8050213 ◽

2019 ◽

Vol 8 (5) ◽

pp. 213 ◽

Cited By ~ 19

Author(s):

Florent Poux ◽

Roland Billen

Keyword(s):

Deep Learning ◽

Point Cloud ◽

Semantic Representation ◽

Structural Connectivity ◽

Three Dimensional ◽

Strong Support ◽

Semantic Segmentation ◽

Point Clouds ◽

Learning Methods ◽

Cloud Data

Automation in point cloud data processing is central in knowledge discovery within decision-making systems. The definition of relevant features is often key for segmentation and classification, with automated workflows presenting the main challenges. In this paper, we propose a voxel-based feature engineering that better characterize point clusters and provide strong support to supervised or unsupervised classification. We provide different feature generalization levels to permit interoperable frameworks. First, we recommend a shape-based feature set (SF1) that only leverages the raw X, Y, Z attributes of any point cloud. Afterwards, we derive relationship and topology between voxel entities to obtain a three-dimensional (3D) structural connectivity feature set (SF2). Finally, we provide a knowledge-based decision tree to permit infrastructure-related classification. We study SF1/SF2 synergy on a new semantic segmentation framework for the constitution of a higher semantic representation of point clouds in relevant clusters. Finally, we benchmark the approach against novel and best-performing deep-learning methods while using the full S3DIS dataset. We highlight good performances, easy-integration, and high F1-score (> 85%) for planar-dominant classes that are comparable to state-of-the-art deep learning.

Download Full-text

Low Overlapping Point Cloud Registration Using Line Features Detection

Remote Sensing ◽

10.3390/rs12010061 ◽

2019 ◽

Vol 12 (1) ◽

pp. 61

Author(s):

Miloš Prokop ◽

Salman Ahmed Shaikh ◽

Kyoung-Sook Kim

Keyword(s):

Point Cloud ◽

False Positive Rate ◽

Computational Cost ◽

Point Clouds ◽

Multiple Point ◽

Point Cloud Registration ◽

Positive Rate ◽

Effective Exchange ◽

Line Features ◽

The Right

Modern robotic exploratory strategies assume multi-agent cooperation that raises a need for an effective exchange of acquired scans of the environment with the absence of a reliable global positioning system. In such situations, agents compare the scans of the outside world to determine if they overlap in some region, and if they do so, they determine the right matching between them. The process of matching multiple point-cloud scans is called point-cloud registration. Using the existing point-cloud registration approaches, a good match between any two-point-clouds is achieved if and only if there exists a large overlap between them, however, this limits the advantage of using multiple robots, for instance, for time-effective 3D mapping. Hence, a point-cloud registration approach is highly desirable if it can work with low overlapping scans. This work proposes a novel solution for the point-cloud registration problem with a very low overlapping area between the two scans. In doing so, no initial relative positions of the point-clouds are assumed. Most of the state-of-the-art point-cloud registration approaches iteratively match keypoints in the scans, which is computationally expensive. In contrast to the traditional approaches, a more efficient line-features-based point-cloud registration approach is proposed in this work. This approach, besides reducing the computational cost, avoids the problem of high false-positive rate of existing keypoint detection algorithms, which becomes especially significant in low overlapping point-cloud registration. The effectiveness of the proposed approach is demonstrated with the help of experiments.

Download Full-text