Fast segmentation of 3D point clouds: A paradigm on LiDAR data for autonomous vehicle applications

Maintenance of roadside ditches is important to avoid localized flooding and premature failure of pavements. Scheduling effective preventative maintenance requires a reasonably detailed mapping of the ditch profile to identify areas in need of excavation to remove long-term sediment accumulation. This study utilizes high-resolution, high-quality point clouds collected by mobile LiDAR mapping systems (MLMS) for mapping roadside ditches and performing hydrological analyses. The performance of alternative MLMS units, including an unmanned aerial vehicle, an unmanned ground vehicle, a portable backpack system along with its vehicle-mounted version, a medium-grade wheel-based system, and a high-grade wheel-based system, is evaluated. Point clouds from all the MLMS units are in agreement within the ±3 cm range for solid surfaces and ±7 cm range for vegetated areas along the vertical direction. The portable backpack system that could be carried by a surveyor or mounted on a vehicle is found to be the most cost-effective method for mapping roadside ditches, followed by the medium-grade wheel-based system. Furthermore, a framework for ditch line characterization is proposed and tested using datasets acquired by the medium-grade wheel-based and vehicle-mounted portable systems over a state highway. An existing ground-filtering approach—cloth simulation—is modified to handle variations in point density of mobile LiDAR data. Hydrological analyses, including flow direction and flow accumulation, are applied to extract the drainage network from the digital terrain model (DTM). Cross-sectional/longitudinal profiles of the ditch are automatically extracted from the LiDAR data and visualized in 3D point clouds and 2D images. The slope derived from the LiDAR data turned out to be very close to the highway cross slope design standards of 2% on driving lanes, 4% on shoulders, and a 6-by-1 slope for ditch lines.

Download Full-text

State Transition for Statistical SLAM Using Planar Features in 3D Point Clouds

Sensors ◽

10.3390/s19071614 ◽

2019 ◽

Vol 19 (7) ◽

pp. 1614 ◽

Cited By ~ 4

Author(s):

Amirali Gostar ◽

Chunyun Fu ◽

Weiqin Chuah ◽

Mohammed Hossain ◽

Ruwan Tennakoon ◽

...

Keyword(s):

Autonomous Vehicle ◽

Large Body ◽

Point Clouds ◽

Substantial Part ◽

Transition Model ◽

3D Point Clouds ◽

Autonomous Cars ◽

Stochastic Transition ◽

Vehicle Sensors ◽

Bayesian Filters

There is a large body of literature on solving the SLAM problem for various autonomous vehicle applications. A substantial part of the solutions is formulated based on using statistical (mainly Bayesian) filters such as Kalman filter and its extended version. In such solutions, the measurements are commonly some point features or detections collected by the sensor(s) on board the autonomous vehicle. With the increasing utilization of scanners with common autonomous cars, and availability of 3D point clouds in real-time and at fast rates, it is now possible to use more sophisticated features extracted from the point clouds for filtering. This paper presents the idea of using planar features with multi-object Bayesian filters for SLAM. With Bayesian filters, the first step is prediction, where the object states are propagated to the next time based on a stochastic transition model. We first present how such a transition model can be developed, and then propose a solution for state prediction. In the simulation studies, using a dataset of measurements acquired from real vehicle sensors, we apply the proposed model to predict the next planar features and vehicle states. The results show reasonable accuracy and efficiency for statistical filtering-based SLAM applications.

Download Full-text

FWNet: Semantic Segmentation for Full-Waveform LiDAR Data Using Deep Learning

Sensors ◽

10.3390/s20123568 ◽

2020 ◽

Vol 20 (12) ◽

pp. 3568 ◽

Cited By ~ 2

Author(s):

Takayuki Shinohara ◽

Haoyi Xiu ◽

Masashi Matsuoka

Keyword(s):

Deep Learning ◽

Semantic Segmentation ◽

Point Clouds ◽

Lidar Data ◽

Global Features ◽

Waveform Data ◽

Full Waveform ◽

3D Point Clouds ◽

Waveform Lidar ◽

Full Waveform Lidar

In the computer vision field, many 3D deep learning models that directly manage 3D point clouds (proposed after PointNet) have been published. Moreover, deep learning-based-techniques have demonstrated state-of-the-art performance for supervised learning tasks on 3D point cloud data, such as classification and segmentation tasks for open datasets in competitions. Furthermore, many researchers have attempted to apply these deep learning-based techniques to 3D point clouds observed by aerial laser scanners (ALSs). However, most of these studies were developed for 3D point clouds without radiometric information. In this paper, we investigate the possibility of using a deep learning method to solve the semantic segmentation task of airborne full-waveform light detection and ranging (lidar) data that consists of geometric information and radiometric waveform data. Thus, we propose a data-driven semantic segmentation model called the full-waveform network (FWNet), which handles the waveform of full-waveform lidar data without any conversion process, such as projection onto a 2D grid or calculating handcrafted features. Our FWNet is based on a PointNet-based architecture, which can extract the local and global features of each input waveform data, along with its corresponding geographical coordinates. Subsequently, the classifier consists of 1D convolutional operational layers, which predict the class vector corresponding to the input waveform from the extracted local and global features. Our trained FWNet achieved higher scores in its recall, precision, and F1 score for unseen test data—higher scores than those of previously proposed methods in full-waveform lidar data analysis domain. Specifically, our FWNet achieved a mean recall of 0.73, a mean precision of 0.81, and a mean F1 score of 0.76. We further performed an ablation study, that is assessing the effectiveness of our proposed method, of the above-mentioned metric. Moreover, we investigated the effectiveness of our PointNet based local and global feature extraction method using the visualization of the feature vector. In this way, we have shown that our network for local and global feature extraction allows training with semantic segmentation without requiring expert knowledge on full-waveform lidar data or translation into 2D images or voxels.

Download Full-text

MULTI-SCALE BASED EXTRACION OF VEGETATION FROM TERRESTRIAL LiDAR DATA FOR ASSESSING LOCAL LANDSCAPE

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprsannals-ii-3-w4-263-2015 ◽

2015 ◽

Vol II-3/W4 ◽

pp. 263-270 ◽

Cited By ~ 4

Author(s):

T. Wakita ◽

J. Susaki

Keyword(s):

Urban Areas ◽

Three Dimensional ◽

Point Clouds ◽

Lidar Data ◽

Data Sets ◽

Landscape Index ◽

Terrestrial Lidar ◽

Multi Scale ◽

3D Point Clouds ◽

Set Of Points

In this study, we propose a method to accurately extract vegetation from terrestrial three-dimensional (3D) point clouds for estimating landscape index in urban areas. Extraction of vegetation in urban areas is challenging because the light returned by vegetation does not show as clear patterns as man-made objects and because urban areas may have various objects to discriminate vegetation from. The proposed method takes a multi-scale voxel approach to effectively extract different types of vegetation in complex urban areas. With two different voxel sizes, a process is repeated that calculates the eigenvalues of the planar surface using a set of points, classifies voxels using the approximate curvature of the voxel of interest derived from the eigenvalues, and examines the connectivity of the valid voxels. We applied the proposed method to two data sets measured in a residential area in Kyoto, Japan. The validation results were acceptable, with F-measures of approximately 95% and 92%. It was also demonstrated that several types of vegetation were successfully extracted by the proposed method whereas the occluded vegetation were omitted. We conclude that the proposed method is suitable for extracting vegetation in urban areas from terrestrial light detection and ranging (LiDAR) data. In future, the proposed method will be applied to mobile LiDAR data and the performance of the method against lower density of point clouds will be examined.

Download Full-text

Automatic Background Filtering Method for Roadside LiDAR Data

Transportation Research Record Journal of the Transportation Research Board ◽

10.1177/0361198118775841 ◽

2018 ◽

Vol 2672 (45) ◽

pp. 106-114 ◽

Cited By ~ 27

Author(s):

Jianqing Wu ◽

Hao Xu ◽

Yuan Sun ◽

Jianying Zheng ◽

Rui Yue

Keyword(s):

Data Processing ◽

Real Time ◽

Point Clouds ◽

Identification Accuracy ◽

Lidar Data ◽

Connected Vehicle ◽

Time Data ◽

Night Time ◽

3D Point Clouds ◽

Real Time Data Processing

The high-resolution micro traffic data (HRMTD) of all roadway users is important for serving the connected-vehicle system in mixed traffic situations. The roadside LiDAR sensor gives a solution to providing HRMTD from real-time 3D point clouds of its scanned objects. Background filtering is the preprocessing step to obtain the HRMTD of different roadway users from roadside LiDAR data. It can significantly reduce the data processing time and improve the vehicle/pedestrian identification accuracy. An algorithm is proposed in this paper, based on the spatial distribution of laser points, which filters both static and moving background efficiently. Various thresholds of point density are applied in this algorithm to exclude background at different distances from the roadside sensor. The case study shows that the algorithm can filter background LiDAR points in different situations (different road geometries, different traffic demands, day/night time, different speed limits). Vehicle and pedestrian shape can be retained well after background filtering. The low computational load guarantees this method can be applied for real-time data processing such as vehicle monitoring and pedestrian tracking.

Download Full-text

USING SIMULATION DATA FROM GAMING ENVIRONMENTS FOR TRAINING A DEEP LEARNING ALGORITHM ON 3D POINT CLOUDS

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-annals-viii-4-w2-2021-67-2021 ◽

2021 ◽

Vol VIII-4/W2-2021 ◽

pp. 67-74

Author(s):

S. Spiegel ◽

J. Chen

Keyword(s):

Neural Networks ◽

Real World ◽

Resource Constraints ◽

Learning Algorithm ◽

Point Clouds ◽

Training Data ◽

Lidar Data ◽

Learning Approaches ◽

Data Set ◽

3D Point Clouds

Abstract. Deep neural networks (DNNs) and convolutional neural networks (CNNs) have demonstrated greater robustness and accuracy in classifying two-dimensional images and three-dimensional point clouds compared to more traditional machine learning approaches. However, their main drawback is the need for large quantities of semantically labeled training data sets, which are often out of reach for those with resource constraints. In this study, we evaluated the use of simulated 3D point clouds for training a CNN learning algorithm to segment and classify 3D point clouds of real-world urban environments. The simulation involved collecting light detection and ranging (LiDAR) data using a simulated 16 channel laser scanner within the the CARLA (Car Learning to Act) autonomous vehicle gaming environment. We used this labeled data to train the Kernel Point Convolution (KPConv) and KPConv Segmentation Network for Point Clouds (KP-FCNN), which we tested on real-world LiDAR data from the NPM3D benchmark data set. Our results showed that high accuracy can be achieved using data collected in a simulator.

Download Full-text

COMPARISON OF 3D POINT CLOUDS FROM AERIAL STEREO IMAGES AND LIDAR FOR FOREST CHANGE DETECTION

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-3-w3-1-2017 ◽

2017 ◽

Vol XLII-3/W3 ◽

pp. 1-5 ◽

Cited By ~ 1

Author(s):

D. Ali-Sisto ◽

P. Packalen

Keyword(s):

Classification Accuracy ◽

Digital Terrain Model ◽

Ground Level ◽

Point Clouds ◽

Aerial Image ◽

Lidar Data ◽

Forest Change ◽

Time Points ◽

3D Point Clouds ◽

The Difference

This study compares performance of aerial image based point clouds (IPCs) and light detection and ranging (LiDAR) based point clouds in detection of thinnings and clear cuts in forests. IPCs are an appealing method to update forest resource data, because of their accuracy in forest height estimation and cost-efficiency of aerial image acquisition. We predicted forest changes over a period of three years by creating difference layers that displayed the difference in height or volume between the initial and subsequent time points. Both IPCs and LiDAR data were used in this process. The IPCs were constructed with the Semi-Global Matching (SGM) algorithm. Difference layers were constructed by calculating differences in fitted height or volume models or in canopy height models (CHMs) from both time points. The LiDAR-derived digital terrain model (DTM) was used to scale heights to above ground level. The study area was classified in logistic regression into the categories ClearCut, Thinning or NoChange with the values from the difference layers. We compared the predicted changes with the true changes verified in the field, and obtained at best a classification accuracy for clear cuts 93.1&thinsp;% with IPCs and 91.7&thinsp;% with LiDAR data. However, a classification accuracy for thinnings was only 8.0&thinsp;% with IPCs. With LiDAR data 41.4&thinsp;% of thinnings were detected. In conclusion, the LiDAR data proved to be more accurate method to predict the minor changes in forests than IPCs, but both methods are useful in detection of major changes.

Download Full-text

Road Ditch Line Mapping with Mobile LiDAR

10.5703/1288284317354 ◽

2021 ◽

Author(s):

Ayman Habib ◽

◽

Darcy M. Bullock ◽

Yi-Chun Lin ◽

Raja Manish

Keyword(s):

Flow Direction ◽

Digital Terrain Model ◽

Point Clouds ◽

Lidar Data ◽

Design Standards ◽

Cross Sectional ◽

Terrain Model ◽

State Highway ◽

3D Point Clouds ◽

Longitudinal Profiles

Maintenance of roadside ditches is important to avoid localized flooding and premature failure of pavements. Scheduling effective preventative maintenance requires mapping of the ditch profile to identify areas requiring excavation of long-term sediment accumulation. High-resolution, high-quality point clouds collected by mobile LiDAR mapping systems (MLMS) provide an opportunity for effective monitoring of roadside ditches and performing hydrological analyses. This study evaluated the applicability of mobile LiDAR for mapping roadside ditches for slope and drainage analyses. The performance of alternative MLMS units was performed. These MLMS included an unmanned ground vehicle, an unmanned aerial vehicle, a portable backpack system along with its vehicle-mounted version, a medium-grade wheel-based system, and a high-grade wheel-based system. Point cloud from all the MLMS units were in agreement in the vertical direction within the ±3 cm range for solid surfaces, such as paved roads, and ±7 cm range for surfaces with vegetation. The portable backpack system that could be carried by a surveyor or mounted on a vehicle and was the most flexible MLMS. The report concludes that due to flexibility and cost effectiveness of the portable backpack system, it is the preferred platform for mapping roadside ditches, followed by the medium-grade wheel-based system. Furthermore, a framework for ditch line characterization is proposed and tested using datasets acquired by the medium-grade wheel-based and vehicle-mounted portable systems over a state highway. An existing ground filtering approach is modified to handle variations in point density of mobile LiDAR data. Hydrological analyses, including flow direction and flow accumulation, are applied to extract the drainage network from the digital terrain model (DTM). Cross-sectional/longitudinal profiles of the ditch are automatically extracted from LiDAR data and visualized in 3D point clouds and 2D images. The slope derived from the LiDAR data was found to be very close to highway cross slope design standards of 2% on driving lanes, 4% on shoulders, as well as 6-by-1 slope for ditch lines. Potential flooded regions are identified by detecting areas with no LiDAR return and a recall score of 54% and 92% was achieved by the medium-grade wheel-based and vehicle-mounted portable systems, respectively. Furthermore, a framework for ditch line characterization is proposed and tested using datasets acquired by the medium-grade wheel-based and vehicle-mounted portable systems over a state highway. An existing ground filtering approach is modified to handle variations in point density of mobile LiDAR data. Hydrological analyses, including flow direction and flow accumulation, are applied to extract the drainage network from the digital terrain model (DTM). Cross-sectional/longitudinal profiles of the ditch are automatically extracted from LiDAR data, and visualized in 3D point clouds and 2D images. The slope derived from the LiDAR data was found to be very close to highway cross slope design standards of 2% on driving lanes, 4% on shoulder, as well as 6-by-1 slope for ditch lines. Potential flooded regions are identified by detecting areas with no LiDAR return and a recall score of 54% and 92% was achieved by the medium-grade wheel-based and vehicle-mounted portable systems, respectively.

Download Full-text

Mapping Riparian Vegetation Functions Using 3D Bispectral LiDAR Data

Water ◽

10.3390/w11030483 ◽

2019 ◽

Vol 11 (3) ◽

pp. 483 ◽

Cited By ~ 2

Author(s):

Marianne Laslier ◽

Laurence Hubert-Moy ◽

Simon Dufour

Keyword(s):

Management Practices ◽

Riparian Zone ◽

Ecological Status ◽

Point Clouds ◽

Riparian Zones ◽

Lidar Data ◽

Random Forest Classification ◽

3D Point Clouds ◽

Forest Classification ◽

Channel Variation

Riparian zones experience many anthropic pressures and are the subject of European legislation to encourage their monitoring and management, to attenuate these pressures. Assessing the effectiveness of management practices requires producing indicators of ecological functions. Laser Detection and Ranging (LiDAR) data can provide valuable information to assess the ecological status of riparian zones. The objective of this study was to evaluate the potential of LiDAR point clouds to produce indicators of riparian zone status. We used 3D bispectral LiDAR data to produce several indicators of a riparian zone of a dammed river in Normandy (France). The indicators were produced either directly from the 3D point clouds (e.g., biomass overhanging the channel, variation in canopy height) or indirectly, by applying the Random Forest classification algorithm to the point clouds. Results highlight the potential of 3D LiDAR point clouds to produce indicators with sufficient accuracy (ca. 80% for the number of trunks and 68% for species composition). Our results also reveal advantages of using metrics related to the internal structure of trees, such as penetration indexes. However, intensity metrics calculated using bispectral properties of LiDAR did not improve the quality of classifications. Longitudinal analysis of the indicators revealed a difference in attributes between the reservoir and areas downstream from it.

Download Full-text

REMOTELY SENSED DATA FUSION IN MODERN AGE ARCHAEOLOGY AND MILITARY HISTORICAL RECONSTRUCTION

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprsarchives-xli-b5-281-2016 ◽

2016 ◽

Vol XLI-B5 ◽

pp. 281-286 ◽

Cited By ~ 1

Author(s):

A. Juhász ◽

H. Neuberger

Keyword(s):

Remote Sensing ◽

Point Clouds ◽

Lidar Data ◽

Historical Reconstruction ◽

Remotely Sensed Data ◽

Typical Sample ◽

3D Point Clouds ◽

Potential Applications ◽

Edge Method ◽

Remote Sensing Methods

LiDAR technology has become one of the major remote sensing methods in the last few years. There are several areas, where the scanned 3D point clouds can be used very efficiently. In our study we review the potential applications of LiDAR data in military historical reconstruction. Obviously, the base of this kind of investigation must be the archive data, but it is an interesting challenge to integrate a cutting edge method into such tasks. The LiDAR technology can be very useful, especially in vegetation covered areas, where the conventional remote sensing technologies are mostly inefficient. We review two typical sample projects where we integrated LiDAR data in military historical GIS reconstruction. Finally, we summarize, how laser scanned data can support the different parts of reconstruction work and define the technological steps of LiDAR data processing.

Download Full-text