POINTNET FOR THE AUTOMATIC CLASSIFICATION OF AERIAL POINT CLOUDS

Abstract. During the last couple of years, there has been an increased interest to develop new deep learning networks specifically for processing 3D point cloud data. In that context, this work intends to expand the applicability of one of these networks, PointNet, from the semantic segmentation of indoor scenes, to outdoor point clouds acquired with Airborne Laser Scanning (ALS) systems. Our goal is to of assist the classification of future iterations of a national wide dataset such as the Actueel Hoogtebestand Nederland (AHN), using a classification model trained with a previous iteration. First, a simple application such as ground classification is proposed in order to prove the capabilities of the proposed deep learning architecture to perform an efficient point-wise classification with aerial point clouds. Then, two different models based on PointNet are defined to classify the most relevant elements in the case study data: Ground, vegetation and buildings. While the model for ground classification performs with a F-score metric above 96%, motivating the second part of the work, the overall accuracy of the remaining models is around 87%, showing consistency across different versions of AHN but with improvable false positive and false negative rates. Therefore, this work concludes that the proposed classification of future AHN iterations is feasible but needs more experimentation.

Download Full-text

3DLEB-Net: Label-Efficient Deep Learning-Based Semantic Segmentation of Building Point Clouds at LoD3 Level

Applied Sciences ◽

10.3390/app11198996 ◽

2021 ◽

Vol 11 (19) ◽

pp. 8996

Author(s):

Yuwei Cao ◽

Marco Scaioni

Keyword(s):

Deep Learning ◽

Semantic Segmentation ◽

Point Clouds ◽

Training Data ◽

Second Step ◽

Point Cloud Data ◽

Dynamic Graph ◽

Cloud Data ◽

Supervised Methods ◽

Global And Local

In current research, fully supervised Deep Learning (DL) techniques are employed to train a segmentation network to be applied to point clouds of buildings. However, training such networks requires large amounts of fine-labeled buildings’ point-cloud data, presenting a major challenge in practice because they are difficult to obtain. Consequently, the application of fully supervised DL for semantic segmentation of buildings’ point clouds at LoD3 level is severely limited. In order to reduce the number of required annotated labels, we proposed a novel label-efficient DL network that obtains per-point semantic labels of LoD3 buildings’ point clouds with limited supervision, named 3DLEB-Net. In general, it consists of two steps. The first step (Autoencoder, AE) is composed of a Dynamic Graph Convolutional Neural Network (DGCNN) encoder and a folding-based decoder. It is designed to extract discriminative global and local features from input point clouds by faithfully reconstructing them without any label. The second step is the semantic segmentation network. By supplying a small amount of task-specific supervision, a segmentation network is proposed for semantically segmenting the encoded features acquired from the pre-trained AE. Experimentally, we evaluated our approach based on the Architectural Cultural Heritage (ArCH) dataset. Compared to the fully supervised DL methods, we found that our model achieved state-of-the-art results on the unseen scenes, with only 10% of labeled training data from fully supervised methods as input. Moreover, we conducted a series of ablation studies to show the effectiveness of the design choices of our model.

Download Full-text

Comparing Machine and Deep Learning Methods for Large 3D Heritage Semantic Segmentation

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi9090535 ◽

2020 ◽

Vol 9 (9) ◽

pp. 535

Author(s):

Francesca Matrone ◽

Eleonora Grilli ◽

Massimo Martini ◽

Marina Paolanti ◽

Roberto Pierdicca ◽

...

Keyword(s):

Deep Learning ◽

Cultural Heritage ◽

Laser Scanning ◽

Semantic Segmentation ◽

Point Clouds ◽

Classification Algorithms ◽

Learning Methods ◽

3D Point Clouds ◽

The Subject

In recent years semantic segmentation of 3D point clouds has been an argument that involves different fields of application. Cultural heritage scenarios have become the subject of this study mainly thanks to the development of photogrammetry and laser scanning techniques. Classification algorithms based on machine and deep learning methods allow to process huge amounts of data as 3D point clouds. In this context, the aim of this paper is to make a comparison between machine and deep learning methods for large 3D cultural heritage classification. Then, considering the best performances of both techniques, it proposes an architecture named DGCNN-Mod+3Dfeat that combines the positive aspects and advantages of these two methodologies for semantic segmentation of cultural heritage point clouds. To demonstrate the validity of our idea, several experiments from the ArCH benchmark are reported and commented.

Download Full-text

Voxel-based 3D Point Cloud Semantic Segmentation: Unsupervised Geometric and Relationship Featuring vs Deep Learning Methods

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi8050213 ◽

2019 ◽

Vol 8 (5) ◽

pp. 213 ◽

Cited By ~ 19

Author(s):

Florent Poux ◽

Roland Billen

Keyword(s):

Deep Learning ◽

Point Cloud ◽

Semantic Representation ◽

Structural Connectivity ◽

Three Dimensional ◽

Strong Support ◽

Semantic Segmentation ◽

Point Clouds ◽

Learning Methods ◽

Cloud Data

Automation in point cloud data processing is central in knowledge discovery within decision-making systems. The definition of relevant features is often key for segmentation and classification, with automated workflows presenting the main challenges. In this paper, we propose a voxel-based feature engineering that better characterize point clusters and provide strong support to supervised or unsupervised classification. We provide different feature generalization levels to permit interoperable frameworks. First, we recommend a shape-based feature set (SF1) that only leverages the raw X, Y, Z attributes of any point cloud. Afterwards, we derive relationship and topology between voxel entities to obtain a three-dimensional (3D) structural connectivity feature set (SF2). Finally, we provide a knowledge-based decision tree to permit infrastructure-related classification. We study SF1/SF2 synergy on a new semantic segmentation framework for the constitution of a higher semantic representation of point clouds in relevant clusters. Finally, we benchmark the approach against novel and best-performing deep-learning methods while using the full S3DIS dataset. We highlight good performances, easy-integration, and high F1-score (> 85%) for planar-dominant classes that are comparable to state-of-the-art deep learning.

Download Full-text

FUSION OF FEATURE BASED AND DEEP LEARNING METHODS FOR CLASSIFICATION OF MMS POINT CLOUDS

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-2-w16-235-2019 ◽

2019 ◽

Vol XLII-2/W16 ◽

pp. 235-242 ◽

Cited By ~ 1

Author(s):

D. Tosic ◽

S. Tuttas ◽

L. Hoegner ◽

U. Stilla

Keyword(s):

Deep Learning ◽

Point Cloud ◽

Semantic Segmentation ◽

Point Clouds ◽

Local Geometry ◽

Learning Approach ◽

Semantic Classification ◽

Feature Based ◽

Urban Scene

Abstract. This work proposes an approach for semantic classification of an outdoor-scene point cloud acquired with a high precision Mobile Mapping System (MMS), with major goal to contribute to the automatic creation of High Definition (HD) Maps. The automatic point labeling is achieved by utilizing the combination of a feature-based approach for semantic classification of point clouds and a deep learning approach for semantic segmentation of images. Both, point cloud data, as well as the data from a multi-camera system are used for gaining spatial information in an urban scene. Two types of classification applied for this task are: 1) Feature-based approach, in which the point cloud is organized into a supervoxel structure for capturing geometric characteristics of points. Several geometric features are then extracted for appropriate representation of the local geometry, followed by removing the effect of local tendency for each supervoxel to enhance the distinction between similar structures. And lastly, the Random Forests (RF) algorithm is applied in the classification phase, for assigning labels to supervoxels and therefore to points within them. 2) The deep learning approach is employed for semantic segmentation of MMS images of the same scene. To achieve this, an implementation of Pyramid Scene Parsing Network is used. Resulting segmented images with each pixel containing a class label are then projected onto the point cloud, enabling label assignment for each point. At the end, experiment results are presented from a complex urban scene and the performance of this method is evaluated on a manually labeled dataset, for the deep learning and feature-based classification individually, as well as for the result of the labels fusion. The achieved overall accuracy with fusioned output is 0.87 on the final test set, which significantly outperforms the results of individual methods on the same point cloud. The labeled data is published on the TUM-PF Semantic-Labeling-Benchmark.

Download Full-text

Road Environment Semantic Segmentation with Deep Learning from MLS Point Cloud Data

Sensors ◽

10.3390/s19163466 ◽

2019 ◽

Vol 19 (16) ◽

pp. 3466 ◽

Cited By ~ 15

Author(s):

Balado ◽

Martínez-Sánchez ◽

Arias ◽

Novo

Keyword(s):

Point Cloud ◽

Laser Scanning ◽

Semantic Segmentation ◽

Point Clouds ◽

Success Rates ◽

Cloud Data ◽

Road Surfaces ◽

Autonomous Cars ◽

Training Cost ◽

Near Future

In the near future, the communication between autonomous cars will produce a network of sensors that will allow us to know the state of the roads in real time. Lidar technology, upon which most autonomous cars are based, allows the acquisition of 3D geometric information of the environment. The objective of this work is to use point clouds acquired by Mobile Laser Scanning (MLS) to segment the main elements of road environment (road surface, ditches, guardrails, fences, embankments, and borders) through the use of PointNet. Previously, the point cloud was automatically divided into sections in order for semantic segmentation to be scalable to different case studies, regardless of their shape or length. An overall accuracy of 92.5% has been obtained, but with large variations between classes. Elements with a greater number of points have been segmented more effectively than the other elements. In comparison with other point-by-point extraction and ANN-based classification techniques, the same success rates have been obtained for road surfaces and fences, and better results have been obtained for guardrails. Semantic segmentation with PointNet is suitable when segmenting the scene as a whole, however, if certain classes have more interest, there are other alternatives that do not need a high training cost.

Download Full-text

JOINT CLASSIFICATION OF ALS AND DIM POINT CLOUDS

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-2-w13-1113-2019 ◽

2019 ◽

Vol XLII-2/W13 ◽

pp. 1113-1120

Author(s):

F. Politz ◽

M. Sester

Keyword(s):

Transfer Learning ◽

Point Cloud ◽

Laser Scanning ◽

Measurement Techniques ◽

Point Clouds ◽

Aerial Images ◽

Individual Measurement ◽

Cloud Data ◽

Practical Applications

Abstract. National mapping agencies (NMAs) have to acquire nation-wide Digital Terrain Models on a regular basis as part of their obligations to provide up-to-date data. Point clouds from Airborne Laser Scanning (ALS) are an important data source for this task; recently, NMAs also started deriving Dense Image Matching (DIM) point clouds from aerial images. As a result, NMAs have both point cloud data sources available, which they can exploit for their purposes. In this study, we investigate the potential of transfer learning from ALS to DIM data, so the time consuming step of data labelling can be reduced. Due to their specific individual measurement techniques, both point clouds have various distinct properties such as RGB or intensity values, which are often exploited for classification of either ALS or DIM point clouds. However, those features also hinder transfer learning between these two point cloud types, since they do not exist in the other point cloud type. As the mere 3D point is available in both point cloud types, we focus on transfer learning from an ALS to a DIM point cloud using exclusively the point coordinates. We are tackling the issue of different point densities by rasterizing the point cloud into a 2D grid and take important height features as input for classification. We train an encoder-decoder convolutional neural network with labelled ALS data as a baseline and then fine-tune this baseline with an increasing amount of labelled DIM data. We also train the same network exclusively on all available DIM data as reference to compare our results. We show that only 10% of labelled DIM data increase the classification results notably, which is especially relevant for practical applications.

Download Full-text

Extraction of linear structures from digital terrain models using deep learning

AGILE: GIScience Series ◽

10.5194/agile-giss-2-11-2021 ◽

2021 ◽

Vol 2 ◽

pp. 1-14

Author(s):

Ramish Satari ◽

Bashir Kazimi ◽

Monika Sester

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Laser Scanning ◽

Semantic Segmentation ◽

Point Clouds ◽

Linear Structures ◽

Digital Terrain Models ◽

Deep Convolutional Neural Networks ◽

Digital Terrain ◽

Terrain Models

Abstract. This paper explores the role deep convolutional neural networks play in automated extraction of linear structures using semantic segmentation techniques in Digital Terrain Models (DTMs). DTM is a regularly gridded raster created from laser scanning point clouds and represents elevations of the bare earth surface with respect to a reference. Recent advances in Deep Learning (DL) have made it possible to explore the use of semantic segmentation for detection of terrain structures in DTMs. This research examines two novel and practical deep convolutional neural network architectures i.e. an encoder-decoder network named as SegNet and the recent state-of-the-art high-resolution network (HRNet). This paper initially focuses on the pixel-wise binary classification in order to validate the applicability of the proposed approaches. The networks are trained to distinguish between points belonging to linear structures and those belonging to background. In the second step, multi-class segmentation is carried out on the same DTM dataset. The model is trained to not only detect a linear feature, but also to categorize it as one of the classes: hollow ways, roads, forest paths, historical paths, and streams. Results of the experiment in addition to the quantitative and qualitative analysis show the applicability of deep neural networks for detection of terrain structures in DTMs. From the deep learning models utilized, HRNet gives better results.

Download Full-text

Automatic Point Cloud Semantic Segmentation of Complex Railway Environments

Remote Sensing ◽

10.3390/rs13122332 ◽

2021 ◽

Vol 13 (12) ◽

pp. 2332

Author(s):

Daniel Lamas ◽

Mario Soilán ◽

Javier Grandío ◽

Belén Riveiro

Keyword(s):

Point Cloud ◽

Three Dimensional ◽

Semantic Segmentation ◽

Point Clouds ◽

Study Data ◽

Cloud Data ◽

Traffic Lights ◽

Cloud Processing ◽

3D Point Clouds ◽

Point Cloud Processing

The growing development of data digitalisation methods has increased their demand and applications in the transportation infrastructure field. Currently, mobile mapping systems (MMSs) are one of the most popular technologies for the acquisition of infrastructure data, with three-dimensional (3D) point clouds as their main product. In this work, a heuristic-based workflow for semantic segmentation of complex railway environments is presented, in which their most relevant elements are classified, namely, rails, masts, wiring, droppers, traffic lights, and signals. This method takes advantage of existing methodologies in the field for point cloud processing and segmentation, taking into account the geometry and spatial context of each classified element in the railway environment. This method is applied to a 90-kilometre-long railway lane and validated against a manual reference on random sections of the case study data. The results are presented and discussed at the object level, differentiating the type of the element. The indicators F1 scores obtained for each element are superior to 85%, being higher than 99% in rails, the most significant element of the infrastructure. These metrics showcase the quality of the algorithm, which proves that this method is efficient for the classification of long and variable railway sections, and for the assisted labelling of point cloud data for future applications based on training supervised learning models.

Download Full-text

Deep-Learning-Based Classification of Point Clouds for Bridge Inspection

Remote Sensing ◽

10.3390/rs12223757 ◽

2020 ◽

Vol 12 (22) ◽

pp. 3757

Author(s):

Hyunsoo Kim ◽

Changwan Kim

Keyword(s):

Deep Learning ◽

Dimensional Space ◽

Three Dimensional ◽

Point Clouds ◽

Dynamic Graph ◽

Bridge Inspection ◽

Cloud Data ◽

Unit Value ◽

Main Component

Conventional bridge maintenance requires significant time and effort because it involves manual inspection and two-dimensional drawings are used to record any damage. For this reason, a process that identifies the location of the damage in three-dimensional space and classifies the bridge components involved is required. In this study, three deep-learning models—PointNet, PointCNN, and Dynamic Graph Convolutional Neural Network (DGCNN)—were compared to classify the components of bridges. Point cloud data were acquired from three types of bridge (Rahmen, girder, and gravity bridges) to determine the optimal model for use across all three types. Three-fold cross-validation was employed, with overall accuracy and intersection over unions used as the performance measures. The mean interval over unit value of DGCNN is 86.85%, which is higher than 84.29% of Pointnet, 74.68% of PointCNN. The accurate classification of a bridge component based on its relationship with the surrounding components may assist in identifying whether the damage to a bridge affects a structurally important main component.

Download Full-text

Classification of Handheld Laser Scanning Tree Point Cloud Based on Different KNN Algorithms and Random Forest Algorithm

Forests ◽

10.3390/f12030292 ◽

2021 ◽

Vol 12 (3) ◽

pp. 292

Author(s):

Wenshu Lin ◽

Weiwei Fan ◽

Haoran Liu ◽

Yongsheng Xu ◽

Jinzhuo Wu

Keyword(s):

Random Forest ◽

Classification Accuracy ◽

Point Cloud ◽

Laser Scanning ◽

Training Sample ◽

Classification Model ◽

Cloud Data ◽

Sample Classification ◽

Adaptive Radius

Handheld mobile laser scanning (HMLS) can quickly acquire point cloud data, and has the potential to conduct forest inventory at the plot scale. Considering the problems associated with HMLS data such as large discreteness and difficulty in classification, different classification models were compared in order to realize efficient separation of stem, branch and leaf points from HMLS data. First, the HMLS point cloud was normalized and ground points were removed, then the neighboring points were identified according to three KNN algorithms and eight geometric features were constructed. On this basis, the random forest classifier was used to calculate feature importance and perform dataset training. Finally, the classification accuracy of different KNN algorithms-based models was evaluated. Results showed that the training sample classification accuracy based on the adaptive radius KNN algorithm was the highest (0.9659) among the three KNN algorithms, but its feature calculation time was also longer; The validation accuracy of two test sets was 0.9596 and 0.9201, respectively, which is acceptable, and the misclassification mainly occurred in the branch junction of the canopy. Therefore, the optimal classification model can effectively achieve the classification of stem, branch and leaf points from HMLS point cloud under the premise of comprehensive training.

Download Full-text