SEMANTIC LABELING OF ALS POINT CLOUDS FOR TREE SPECIES MAPPING USING THE DEEP NEURAL NETWORK POINTNET++

Abstract. Most methods for the mapping of tree species are based on the segmentation of single trees that are subsequently classified using a set of hand-crafted features and an appropriate classifier. The classification accuracy for coniferous and deciduous trees just using airborne laser scanning (ALS) data is only around 90% in case the geometric information of the point cloud is used. As deep neural networks (DNNs) have the ability to adaptively learn features from the underlying data, they have outperformed classic machine learning (ML) approaches on well-known benchmark datasets provided by the robotics, computer vision and remote sensing community. Though, tree species classification using deep learning (DL) procedures has been of minor research interest so far. Some studies have been conducted based on an extensive prior generation of images or voxels from the 3D raw data. Since innovative DNNs directly operate on irregular and unordered 3D point clouds on a large scale, the objective of this study is to exemplarily use PointNet++ for the semantic labeling of ALS point clouds to map deciduous and coniferous trees. The dataset for our experiments consists of ALS data from the Bavarian Forest National Park (366 trees/ha), only including spruces (coniferous) and beeches (deciduous). First, the training data were generated automatically using a classic feature-based Random Forest (RF) approach classifying coniferous trees (precision&thinsp;=&thinsp;93%, recall&thinsp;=&thinsp;80%) and deciduous trees (precision&thinsp;=&thinsp;82%, recall&thinsp;=&thinsp;92%). Second, PointNet++ was trained and subsequently evaluated using 80 randomly chosen test batches à 400&thinsp;m2. The achieved per-point classification results after 163 training epochs for coniferous trees (precision&thinsp;=&thinsp;90%, recall&thinsp;=&thinsp;79%) and deciduous trees (precision&thinsp;=&thinsp;81%, recall&thinsp;=&thinsp;91%) are fairly high considering that only the geometry was included. Nevertheless, the classification results using PointNet++ are slightly lower than those of the baseline method using a RF classifier. Errors in the training data and occurring edge effects limited a better performance. Our first results demonstrate that the architecture of the 3D DNN PointNet++ can successfully be adapted to the semantic labeling of large ALS point clouds to map deciduous and coniferous trees. Future work will focus on the integration of additional features like i.e. the laser intensity, the surface normals and multispectral features into the DNN. Thus, a further improvement of the accuracy of the proposed approach is to be expected. Furthermore, the classification of numerous individual tree species based on pre-segmented single trees should be investigated.

Download Full-text

Predicting Tree Species From 3D Laser Scanning Point Clouds Using Deep Learning

Frontiers in Plant Science ◽

10.3389/fpls.2021.635440 ◽

2021 ◽

Vol 12 ◽

Author(s):

Dominik Seidel ◽

Peter Annighöfer ◽

Anton Thielman ◽

Quentin Edward Seifert ◽

Jan-Henrik Thauer ◽

...

Keyword(s):

Tree Species ◽

Classification Accuracy ◽

Laser Scanning ◽

Confusion Matrix ◽

Point Clouds ◽

Small Sample ◽

Training Data ◽

Computationally Efficient ◽

Species Classification ◽

3D Point Clouds

Automated species classification from 3D point clouds is still a challenge. It is, however, an important task for laser scanning-based forest inventory, ecosystem models, and to support forest management. Here, we tested the performance of an image classification approach based on convolutional neural networks (CNNs) with the aim to classify 3D point clouds of seven tree species based on 2D representation in a computationally efficient way. We were particularly interested in how the approach would perform with artificially increased training data size based on image augmentation techniques. Our approach yielded a high classification accuracy (86%) and the confusion matrix revealed that despite rather small sample sizes of the training data for some tree species, classification accuracy was high. We could partly relate this to the successful application of the image augmentation technique, improving our result by 6% in total and 13, 14, and 24% for ash, oak and pine, respectively. The introduced approach is hence not only applicable to small-sized datasets, it is also computationally effective since it relies on 2D instead of 3D data to be processed in the CNN. Our approach was faster and more accurate when compared to the point cloud-based “PointNet” approach.

Download Full-text

Predicting tree species based on the geometry and intensity of aerial laser scanning point cloud of treetops

Geodetski vestnik ◽

10.15292/geodetski-vestnik.2021.02.234-259 ◽

2021 ◽

Vol 65 (02) ◽

pp. 234-259

Author(s):

Nina Kranjec ◽

Mihaela Triglav Čekada ◽

Milan Kobal

Keyword(s):

Tree Species ◽

Laser Scanning ◽

Model Performance ◽

Point Clouds ◽

Average Intensity ◽

Acer Pseudoplatanus ◽

Individual Tree ◽

Coniferous Trees ◽

Explanatory Variables ◽

Individual Trees

Based on the laser point clouds of 240 individual trees that were also identified in the field, we developed decision trees to distinguish deciduous and coniferous trees and individual tree species: Picea abies, Larix decidua, Pinus sylvestris, Fagus sylvatica, Acer pseudoplatanus, Fraxinus excelsior. The volume of the upper part of the tree crown (height of 3 m) and the average intensity of the laser reflections were used as explanatory variables. There were four aerial laser datasets: May 2012, September 2012, March 2013 and July 2015. We found that the combination of the volume and the average intensity of the first three laser datasets was the most reliable for predicting the selected tree species (60% model performance). A slightly poorer model performance was obtained if only the average intensity of the first three datasets was used (54% model performance). The worst model performance was given by the intensities (31 % model performance) or the volumes (21 % model performance) of dataset 4, which represents the national laser scanning of Slovenia (LSS). The best performing was the deciduous and coniferous separation, which achieved 75% and 95% success based on the test data (combination of volume and average intensity of the first three laser datasets). Using only the LSS intensities, deciduous and coniferous trees could be separated with 81% success.

Download Full-text

Automated Visual Recognizability Evaluation of Traffic Sign Based on 3D LiDAR Point Clouds

Remote Sensing ◽

10.3390/rs11121453 ◽

2019 ◽

Vol 11 (12) ◽

pp. 1453 ◽

Cited By ~ 5

Author(s):

Shanxin Zhang ◽

Cheng Wang ◽

Lili Lin ◽

Chenglu Wen ◽

Chenhui Yang ◽

...

Keyword(s):

Traffic Safety ◽

Laser Scanning ◽

Large Scale ◽

Evaluation Method ◽

Evaluation Model ◽

Point Clouds ◽

Traffic Sign ◽

Traffic Signs ◽

3D Point Clouds ◽

Traffic Environment

Maintaining the high visual recognizability of traffic signs for traffic safety is a key matter for road network management. Mobile Laser Scanning (MLS) systems provide efficient way of 3D measurement over large-scale traffic environment. This paper presents a quantitative visual recognizability evaluation method for traffic signs in large-scale traffic environment based on traffic recognition theory and MLS 3D point clouds. We first propose the Visibility Evaluation Model (VEM) to quantitatively describe the visibility of traffic sign from any given viewpoint, then we proposed the concept of visual recognizability field and Traffic Sign Visual Recognizability Evaluation Model (TSVREM) to measure the visual recognizability of a traffic sign. Finally, we present an automatic TSVREM calculation algorithm for MLS 3D point clouds. Experimental results on real MLS 3D point clouds show that the proposed method is feasible and efficient.

Download Full-text

HYBRID ACQUISITION OF HIGH QUALITY TRAINING DATA FOR SEMANTIC SEGMENTATION OF 3D POINT CLOUDS USING CROWD-BASED ACTIVE LEARNING

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-annals-v-2-2020-501-2020 ◽

2020 ◽

Vol V-2-2020 ◽

pp. 501-508

Author(s):

M. Kölle ◽

V. Walter ◽

S. Schmohl ◽

U. Soergel

Keyword(s):

Active Learning ◽

Semantic Segmentation ◽

Point Clouds ◽

Human Interaction ◽

Training Data ◽

Semantic Interpretation ◽

Training Dataset ◽

3D Point Clouds ◽

Semantic Labeling ◽

3D Cnn

Abstract. Automated semantic interpretation of 3D point clouds is crucial for many tasks in the domain of geospatial data analysis. For this purpose, labeled training data is required, which has often to be provided manually by experts. One approach to minimize effort in terms of costs of human interaction is Active Learning (AL). The aim is to process only the subset of an unlabeled dataset that is particularly helpful with respect to class separation. Here a machine identifies informative instances which are then labeled by humans, thereby increasing the performance of the machine. In order to completely avoid involvement of an expert, this time-consuming annotation can be resolved via crowdsourcing. Therefore, we propose an approach combining AL with paid crowdsourcing. Although incorporating human interaction, our method can run fully automatically, so that only an unlabeled dataset and a fixed financial budget for the payment of the crowdworkers need to be provided. We conduct multiple iteration steps of the AL process on the ISPRS Vaihingen 3D Semantic Labeling benchmark dataset (V3D) and especially evaluate the performance of the crowd when labeling 3D points. We prove our concept by using labels derived from our crowd-based AL method for classifying the test dataset. The analysis outlines that by labeling only 0:4% of the training dataset by the crowd and spending less than 145 $, both our trained Random Forest and sparse 3D CNN classifier differ in Overall Accuracy by less than 3 percentage points compared to the same classifiers trained on the complete V3D training set.

Download Full-text

Using Leaf-Off and Leaf-On Multispectral Airborne Laser Scanning Data to Characterize Seedling Stands

Remote Sensing ◽

10.3390/rs12203328 ◽

2020 ◽

Vol 12 (20) ◽

pp. 3328

Author(s):

Mohammad Imangholiloo ◽

Ninni Saarinen ◽

Markus Holopainen ◽

Xiaowei Yu ◽

Juha Hyyppä ◽

...

Keyword(s):

Tree Species ◽

Laser Scanning ◽

Sustainable Forest Management ◽

Single Channel ◽

Point Clouds ◽

Tree Density ◽

Airborne Laser Scanning ◽

Tree Species Composition ◽

Coniferous Trees ◽

Airborne Laser

Information from seedling stands in time and space is essential for sustainable forest management. To fulfil these informational needs with limited resources, remote sensing is seen as an intriguing alternative for forest inventorying. The structure and tree species composition in seedling stands have created challenges for capturing this information using sensors providing sparse point densities that do not have the ability to penetrate canopy gaps or provide spectral information. Therefore, multispectral airborne laser scanning (mALS) systems providing dense point clouds coupled with multispectral intensity data theoretically offer advantages for the characterization of seedling stands. The aim of this study was to investigate the capability of Optech Titan mALS data to characterize seedling stands in leaf-off and leaf-on conditions, as well as to retrieve the most important forest inventory attributes, such as distinguishing deciduous from coniferous trees, and estimating tree density and height. First, single-tree detection approaches were used to derive crown boundaries and tree heights from which forest structural attributes were aggregated for sample plots. To predict tree species, a random forests classifier was trained using features from two single-channel intensities (SCIs) with wavelengths of 1550 (SCI-Ch1) and 1064 nm (SCI-Ch2), and multichannel intensity (MCI) data composed of three mALS channels. The most important and uncorrelated features were analyzed and selected from 208 features. The highest overall accuracies in classification of Norway spruce, birch, and nontree class in leaf-off and leaf-on conditions obtained using SCI-Ch1 and SCI-Ch2 were 87.36% and 69.47%, respectively. The use of MCI data improved classification by up to 96.55% and 92.54% in leaf-off and leaf-on conditions, respectively. Overall, leaf-off data were favorable for distinguishing deciduous from coniferous trees and tree density estimation with a relative root mean square error (RMSE) of 37.9%, whereas leaf-on data provided more accurate height estimations, with a relative RMSE of 10.76%. Determining the canopy threshold for separating ground returns from vegetation returns was found to be critical, as mapped trees might have a height below one meter. The results showed that mALS data provided benefits for characterizing seedling stands compared to single-channel ALS systems.

Download Full-text

Virtual Disassembling of Historical Edifices: Experiments and Assessments of an Automatic Approach for Classifying Multi-Scalar Point Clouds into Architectural Elements

Sensors ◽

10.3390/s20082161 ◽

2020 ◽

Vol 20 (8) ◽

pp. 2161 ◽

Cited By ~ 4

Author(s):

Arnadi Murtiyoso ◽

Pierre Grussenmeyer

Keyword(s):

Point Cloud ◽

Laser Scanning ◽

Semantic Annotation ◽

Semantic Segmentation ◽

Point Clouds ◽

Training Data ◽

Algorithmic Approach ◽

Architectural Elements ◽

Semantic Labeling ◽

Geometric Point

3D heritage documentation has seen a surge in the past decade due to developments in reality-based 3D recording techniques. Several methods such as photogrammetry and laser scanning are becoming ubiquitous amongst architects, archaeologists, surveyors, and conservators. The main result of these methods is a 3D representation of the object in the form of point clouds. However, a solely geometric point cloud is often insufficient for further analysis, monitoring, and model predicting of the heritage object. The semantic annotation of point clouds remains an interesting research topic since traditionally it requires manual labeling and therefore a lot of time and resources. This paper proposes an automated pipeline to segment and classify multi-scalar point clouds in the case of heritage object. This is done in order to perform multi-level segmentation from the scale of a historical neighborhood up until that of architectural elements, specifically pillars and beams. The proposed workflow involves an algorithmic approach in the form of a toolbox which includes various functions covering the semantic segmentation of large point clouds into smaller, more manageable and semantically labeled clusters. The first part of the workflow will explain the segmentation and semantic labeling of heritage complexes into individual buildings, while a second part will discuss the use of the same toolbox to segment the resulting buildings further into architectural elements. The toolbox was tested on several historical buildings and showed promising results. The ultimate intention of the project is to help the manual point cloud labeling, especially when confronted with the large training data requirements of machine learning-based algorithms.

Download Full-text

TUM-MLS-2016: An Annotated Mobile LiDAR Dataset of the TUM City Campus for Semantic Point Cloud Interpretation in Urban Areas

Remote Sensing ◽

10.3390/rs12111875 ◽

2020 ◽

Vol 12 (11) ◽

pp. 1875 ◽

Cited By ~ 1

Author(s):

Jingwei Zhu ◽

Joachim Gehrung ◽

Rong Huang ◽

Björn Borgmann ◽

Zhenghao Sun ◽

...

Keyword(s):

Test Data ◽

Urban Areas ◽

Point Cloud ◽

Large Scale ◽

Point Clouds ◽

Semantic Interpretation ◽

3D Point Clouds ◽

Semantic Labeling ◽

Benchmark Datasets ◽

Semantic Point

In the past decade, a vast amount of strategies, methods, and algorithms have been developed to explore the semantic interpretation of 3D point clouds for extracting desirable information. To assess the performance of the developed algorithms or methods, public standard benchmark datasets should invariably be introduced and used, which serve as an indicator and ruler in the evaluation and comparison. In this work, we introduce and present large-scale Mobile LiDAR point clouds acquired at the city campus of the Technical University of Munich, which have been manually annotated and can be used for the evaluation of related algorithms and methods for semantic point cloud interpretation. We created three datasets from a measurement campaign conducted in April 2016, including a benchmark dataset for semantic labeling, test data for instance segmentation, and test data for annotated single 360 ° laser scans. These datasets cover an urban area of approximately 1 km long roadways and include more than 40 million annotated points with eight classes of objects labeled. Moreover, experiments were carried out with results from several baseline methods compared and analyzed, revealing the quality of this dataset and its effectiveness when using it for performance evaluation.

Download Full-text

VALIDATING A WORKFLOW FOR TREE INVENTORY UPDATING WITH 3D POINT CLOUDS OBTAINED BY MOBILE LASER SCANNING

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-2-1163-2018 ◽

2018 ◽

Vol XLII-2 ◽

pp. 1163-1168

Author(s):

J. Wang ◽

R. Lindenbergh

Keyword(s):

Tree Species ◽

Laser Scanning ◽

Spatial Information ◽

Open Data ◽

Point Clouds ◽

Urban Trees ◽

Biomass Estimation ◽

Species Classification ◽

3D Point Clouds ◽

Tree Inventory

Urban trees are an important component of our environment and ecosystem. Trees are able to combat climate change, clean the air and cool the streets and city. Tree inventory and monitoring are of great interest for biomass estimation and change monitoring. Conventionally, parameters of trees are manually measured and documented in situ, which is not efficient regarding labour and costs. Light Detection And Ranging (LiDAR) has become a well-established surveying technique for the acquisition of geo-spatial information. Combined with automatic point cloud processing techniques, this in principle enables the efficient extraction of geometric tree parameters. In recent years, studies have investigated to what extend it is possible to perform tree inventories using laser scanning point clouds. Give the availability of a city of Delft Open data tree repository, we are now able to present, validate and extend a workflow to automatically obtain tree data from tree location until tree species. The results of a test over 47 trees show that the proposed methods in the workflow are able to individual urban trees. The tree species classification results based on the extracted tree parameters show that only one tree was wrongly classified using k-means clustering.

Download Full-text

Can 3D Point Clouds Replace GCPs?

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprsannals-ii-5-347-2014 ◽

2014 ◽

Vol II-5 ◽

pp. 347-354 ◽

Cited By ~ 2

Author(s):

G. Stavropoulou ◽

G. Tzovla ◽

A. Georgopoulos

Keyword(s):

Point Cloud ◽

Laser Scanning ◽

Large Scale ◽

Point Clouds ◽

Field Work ◽

Surface Model ◽

Control Points ◽

3D Point Clouds ◽

Surface Models

Over the past decade, large-scale photogrammetric products have been extensively used for the geometric documentation of cultural heritage monuments, as they combine metric information with the qualities of an image document. Additionally, the rising technology of terrestrial laser scanning has enabled the easier and faster production of accurate digital surface models (DSM), which have in turn contributed to the documentation of heavily textured monuments. However, due to the required accuracy of control points, the photogrammetric methods are always applied in combination with surveying measurements and hence are dependent on them. Along this line of thought, this paper explores the possibility of limiting the surveying measurements and the field work necessary for the production of large-scale photogrammetric products and proposes an alternative method on the basis of which the necessary control points instead of being measured with surveying procedures are chosen from a dense and accurate point cloud. Using this point cloud also as a surface model, the only field work necessary is the scanning of the object and image acquisition, which need not be subject to strict planning. To evaluate the proposed method an algorithm and the complementary interface were produced that allow the parallel manipulation of 3D point clouds and images and through which single image procedures take place. The paper concludes by presenting the results of a case study in the ancient temple of Hephaestus in Athens and by providing a set of guidelines for implementing effectively the method.

Download Full-text

LASDU: A Large-Scale Aerial LiDAR Dataset for Semantic Labeling in Dense Urban Areas

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi9070450 ◽

2020 ◽

Vol 9 (7) ◽

pp. 450

Author(s):

Zhen Ye ◽

Yusheng Xu ◽

Rong Huang ◽

Xiaohua Tong ◽

Xin Li ◽

...

Keyword(s):

Urban Area ◽

Urban Areas ◽

Point Cloud ◽

Large Scale ◽

Point Clouds ◽

Light Detection And Ranging ◽

Early Age ◽

3D Mapping ◽

3D Point Clouds ◽

Semantic Labeling

The semantic labeling of the urban area is an essential but challenging task for a wide variety of applications such as mapping, navigation, and monitoring. The rapid advance in Light Detection and Ranging (LiDAR) systems provides this task with a possible solution using 3D point clouds, which are accessible, affordable, accurate, and applicable. Among all types of platforms, the airborne platform with LiDAR can serve as an efficient and effective tool for large-scale 3D mapping in the urban area. Against this background, a large number of algorithms and methods have been developed to fully explore the potential of 3D point clouds. However, the creation of publicly accessible large-scale annotated datasets, which are critical for assessing the performance of the developed algorithms and methods, is still at an early age. In this work, we present a large-scale aerial LiDAR point cloud dataset acquired in a highly-dense and complex urban area for the evaluation of semantic labeling methods. This dataset covers an urban area with highly-dense buildings of approximately 1 km2 and includes more than three million points with five classes of objects labeled. Moreover, experiments are carried out with the results from several baseline methods, demonstrating the feasibility and capability of the dataset serving as a benchmark for assessing semantic labeling methods.

Download Full-text