Learning multiscale spatial context for three-dimensional point cloud semantic segmentation

2020 ◽  
Vol 29 (06) ◽  
Author(s):  
Yang Wang ◽  
Shunping Xiao
Keyword(s):  
Point Cloud ◽  
Three Dimensional ◽  
Semantic Segmentation ◽  
Spatial Context

scholarly journals Voxel-based 3D Point Cloud Semantic Segmentation: Unsupervised Geometric and Relationship Featuring vs Deep Learning Methods

2019 ◽  
Vol 8 (5) ◽  
pp. 213 ◽  
Author(s):  
Florent Poux ◽  
Roland Billen
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Semantic Representation ◽  
Structural Connectivity ◽  
Three Dimensional ◽  
Strong Support ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Learning Methods ◽  
Cloud Data

Automation in point cloud data processing is central in knowledge discovery within decision-making systems. The definition of relevant features is often key for segmentation and classification, with automated workflows presenting the main challenges. In this paper, we propose a voxel-based feature engineering that better characterize point clusters and provide strong support to supervised or unsupervised classification. We provide different feature generalization levels to permit interoperable frameworks. First, we recommend a shape-based feature set (SF1) that only leverages the raw X, Y, Z attributes of any point cloud. Afterwards, we derive relationship and topology between voxel entities to obtain a three-dimensional (3D) structural connectivity feature set (SF2). Finally, we provide a knowledge-based decision tree to permit infrastructure-related classification. We study SF1/SF2 synergy on a new semantic segmentation framework for the constitution of a higher semantic representation of point clouds in relevant clusters. Finally, we benchmark the approach against novel and best-performing deep-learning methods while using the full S3DIS dataset. We highlight good performances, easy-integration, and high F1-score (> 85%) for planar-dominant classes that are comparable to state-of-the-art deep learning.

scholarly journals WEAKLY SUPERVISED PSEUDO-LABEL ASSISTED LEARNING FOR ALS POINT CLOUD SEMANTIC SEGMENTATION

2021 ◽  
Vol V-2-2021 ◽  
pp. 43-50
Author(s):  
P. Wang ◽  
W. Yao
Keyword(s):  
Point Cloud ◽  
Three Dimensional ◽  
Ground Truth ◽  
Semantic Segmentation ◽  
Training Data ◽  
Data Annotation ◽  
Cloud Data ◽  
Point Cloud Segmentation ◽  
Prediction Probability ◽  
Weakly Supervised

Abstract. Competitive point cloud semantic segmentation results usually rely on a large amount of labeled data. However, data annotation is a time-consuming and labor-intensive task, particularly for three-dimensional point cloud data. Thus, obtaining accurate results with limited ground truth as training data is considerably important. As a simple and effective method, pseudo labels can use information from unlabeled data for training neural networks. In this study, we propose a pseudo-label-assisted point cloud segmentation method with very few sparsely sampled labels that are normally randomly selected for each class. An adaptive thresholding strategy was proposed to generate a pseudo-label based on the prediction probability. Pseudo-label learning is an iterative process, and pseudo labels were updated solely on ground-truth weak labels as the model converged to improve the training efficiency. Experiments using the ISPRS 3D sematic labeling benchmark dataset indicated that our proposed method achieved an equally competitive result compared to that using a full supervision scheme with only up to 2‰ of labeled points from the original training set, with an overall accuracy of 83.7% and an average F1 score of 70.2%.

scholarly journals DEEP LEARNING FOR SEMANTIC SEGMENTATION OF 3D POINT CLOUD

2019 ◽  
Vol XLII-2/W15 ◽  
pp. 735-742 ◽  
Author(s):  
E. S. Malinverni ◽  
R. Pierdicca ◽  
M. Paolanti ◽  
M. Martini ◽  
C. Morbidoni ◽  
...  
Keyword(s):  
Deep Learning ◽  
Cultural Heritage ◽  
Point Cloud ◽  
Cultural Landscapes ◽  
Three Dimensional ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Training Data ◽  
Historical Building ◽  
3D Point Clouds

<p><strong>Abstract.</strong> Cultural Heritage is a testimony of past human activity, and, as such, its objects exhibit great variety in their nature, size and complexity; from small artefacts and museum items to cultural landscapes, from historical building and ancient monuments to city centers and archaeological sites. Cultural Heritage around the globe suffers from wars, natural disasters and human negligence. The importance of digital documentation is well recognized and there is an increasing pressure to document our heritage both nationally and internationally. For this reason, the three-dimensional scanning and modeling of sites and artifacts of cultural heritage have remarkably increased in recent years. The semantic segmentation of point clouds is an essential step of the entire pipeline; in fact, it allows to decompose complex architectures in single elements, which are then enriched with meaningful information within Building Information Modelling software. Notwithstanding, this step is very time consuming and completely entrusted on the manual work of domain experts, far from being automatized. This work describes a method to label and cluster automatically a point cloud based on a supervised Deep Learning approach, using a state-of-the-art Neural Network called PointNet++. Despite other methods are known, we have choose PointNet++ as it reached significant results for classifying and segmenting 3D point clouds. PointNet++ has been tested and improved, by training the network with annotated point clouds coming from a real survey and to evaluate how performance changes according to the input training data. It can result of great interest for the research community dealing with the point cloud semantic segmentation, since it makes public a labelled dataset of CH elements for further tests.</p>

scholarly journals AttAN: Attention Adversarial Networks for 3D Point Cloud Semantic Segmentation

2020 ◽  
Author(s):  
Gege Zhang ◽  
Qinghua Ma ◽  
Licheng Jiao ◽  
Fang Liu ◽  
Qigong Sun
Keyword(s):  
Point Cloud ◽  
Dimensional Space ◽  
Three Dimensional ◽  
Semantic Segmentation ◽  
Autonomous Driving ◽  
3D Point Cloud ◽  
Adversarial Learning ◽  
Adversarial Networks ◽  
Segmentation Accuracy ◽  
Segmentation Task

3D point cloud semantic segmentation has attracted wide attention with its extensive applications in autonomous driving, AR/VR, and robot sensing fields. However, in existing methods, each point in the segmentation results is predicted independently from each other. This property causes the non-contiguity of label sets in three-dimensional space and produces many noisy label points, which hinders the improvement of segmentation accuracy. To address this problem, we first extend adversarial learning to this task and propose a novel framework Attention Adversarial Networks (AttAN). With high-order correlations in label sets learned from the adversarial learning, segmentation network can predict labels closer to the real ones and correct noisy results. Moreover, we design an additive attention block for the segmentation network, which is used to automatically focus on regions critical to the segmentation task by learning the correlation between multi-scale features. Adversarial learning, which explores the underlying relationship between labels in high-dimensional space, opens up a new way in 3D point cloud semantic segmentation. Experimental results on ScanNet and S3DIS datasets show that this framework effectively improves the segmentation quality and outperforms other state-of-the-art methods.

scholarly journals Automatic Point Cloud Semantic Segmentation of Complex Railway Environments

2021 ◽  
Vol 13 (12) ◽  
pp. 2332
Author(s):  
Daniel Lamas ◽  
Mario Soilán ◽  
Javier Grandío ◽  
Belén Riveiro
Keyword(s):  
Point Cloud ◽  
Three Dimensional ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Study Data ◽  
Cloud Data ◽  
Traffic Lights ◽  
Cloud Processing ◽  
3D Point Clouds ◽  
Point Cloud Processing

The growing development of data digitalisation methods has increased their demand and applications in the transportation infrastructure field. Currently, mobile mapping systems (MMSs) are one of the most popular technologies for the acquisition of infrastructure data, with three-dimensional (3D) point clouds as their main product. In this work, a heuristic-based workflow for semantic segmentation of complex railway environments is presented, in which their most relevant elements are classified, namely, rails, masts, wiring, droppers, traffic lights, and signals. This method takes advantage of existing methodologies in the field for point cloud processing and segmentation, taking into account the geometry and spatial context of each classified element in the railway environment. This method is applied to a 90-kilometre-long railway lane and validated against a manual reference on random sections of the case study data. The results are presented and discussed at the object level, differentiating the type of the element. The indicators F1 scores obtained for each element are superior to 85%, being higher than 99% in rails, the most significant element of the infrastructure. These metrics showcase the quality of the algorithm, which proves that this method is efficient for the classification of long and variable railway sections, and for the assisted labelling of point cloud data for future applications based on training supervised learning models.

scholarly journals Multispectral LiDAR Point Cloud Classification Using SE-PointNet++

2021 ◽  
Vol 13 (13) ◽  
pp. 2516
Author(s):  
Zhuangwei Jing ◽  
Haiyan Guan ◽  
Peiran Zhao ◽  
Dilong Li ◽  
Yongtao Yu ◽  
...  
Keyword(s):  
Point Cloud ◽  
Three Dimensional ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Cloud Classification ◽  
Feature Saliency ◽  
Intensity Information ◽  
Multi Wavelength ◽  
Data Volume ◽  
Point Cloud Classification

A multispectral light detection and ranging (LiDAR) system, which simultaneously collects spatial geometric data and multi-wavelength intensity information, opens the door to three-dimensional (3-D) point cloud classification and object recognition. Because of the irregular distribution property of point clouds and the massive data volume, point cloud classification directly from multispectral LiDAR data is still challengeable and questionable. In this paper, a point-wise multispectral LiDAR point cloud classification architecture termed as SE-PointNet++ is proposed via integrating a Squeeze-and-Excitation (SE) block with an improved PointNet++ semantic segmentation network. PointNet++ extracts local features from unevenly sampled points and represents local geometrical relationships among the points through multi-scale grouping. The SE block is embedded into PointNet++ to strengthen important channels to increase feature saliency for better point cloud classification. Our SE-PointNet++ architecture has been evaluated on the Titan multispectral LiDAR test datasets and achieved an overall accuracy, a mean Intersection over Union (mIoU), an F1-score, and a Kappa coefficient of 91.16%, 60.15%, 73.14%, and 0.86, respectively. Comparative studies with five established deep learning models confirmed that our proposed SE-PointNet++ achieves promising performance in multispectral LiDAR point cloud classification tasks.

scholarly journals DeepLabV3-Refiner-Based Semantic Segmentation Model for Dense 3D Point Clouds

2021 ◽  
Vol 13 (8) ◽  
pp. 1565
Author(s):  
Jeonghoon Kwak ◽  
Yunsick Sung
Keyword(s):  
Virtual Environment ◽  
Point Cloud ◽  
Three Dimensional ◽  
Fold Increase ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
3D Point Cloud ◽  
3D Point Clouds ◽  
Segmented Image ◽  
Human Object

Three-dimensional virtual environments can be configured as test environments of autonomous things, and remote sensing by 3D point clouds collected by light detection and range (LiDAR) can be used to detect virtual human objects by segmenting collected 3D point clouds in a virtual environment. The use of a traditional encoder-decoder model, such as DeepLabV3, improves the quality of the low-density 3D point clouds of human objects, where the quality is determined by the measurement gap of the LiDAR lasers. However, whenever a human object with a surrounding environment in a 3D point cloud is used by the traditional encoder-decoder model, it is difficult to increase the density fitting of the human object. This paper proposes a DeepLabV3-Refiner model, which is a model that refines the fit of human objects using human objects whose density has been increased through DeepLabV3. An RGB image that has a segmented human object is defined as a dense segmented image. DeepLabV3 is used to make predictions of dense segmented images and 3D point clouds for human objects in 3D point clouds. In the Refiner model, the results of DeepLabV3 are refined to fit human objects, and a dense segmented image fit to human objects is predicted. The dense 3D point cloud is calculated using the dense segmented image provided by the DeepLabV3-Refiner model. The 3D point clouds that were analyzed by the DeepLabV3-Refiner model had a 4-fold increase in density, which was verified experimentally. The proposed method had a 0.6% increase in density accuracy compared to that of DeepLabV3, and a 2.8-fold increase in the density corresponding to the human object. The proposed method was able to provide a 3D point cloud that increased the density to fit the human object. The proposed method can be used to provide an accurate 3D virtual environment by using the improved 3D point clouds.

scholarly journals Reconstruction of High-Precision Semantic Map

Sensors ◽  
2020 ◽  
Vol 20 (21) ◽  
pp. 6264
Author(s):  
Xinyuan Tu ◽  
Jian Zhang ◽  
Runhao Luo ◽  
Kai Wang ◽  
Qingji Zeng ◽  
...  
Keyword(s):  
Real Time ◽  
Point Cloud ◽  
Semantic Information ◽  
Feature Fusion ◽  
Three Dimensional ◽  
Semantic Segmentation ◽  
Line Of Sight ◽  
Lidar Data ◽  
Implicit Surface ◽  
Robotic Mapping

We present a real-time Truncated Signed Distance Field (TSDF)-based three-dimensional (3D) semantic reconstruction for LiDAR point cloud, which achieves incremental surface reconstruction and highly accurate semantic segmentation. The high-precise 3D semantic reconstruction in real time on LiDAR data is important but challenging. Lighting Detection and Ranging (LiDAR) data with high accuracy is massive for 3D reconstruction. We so propose a line-of-sight algorithm to update implicit surface incrementally. Meanwhile, in order to use more semantic information effectively, an online attention-based spatial and temporal feature fusion method is proposed, which is well integrated into the reconstruction system. We implement parallel computation in the reconstruction and semantic fusion process, which achieves real-time performance. We demonstrate our approach on the CARLA dataset, Apollo dataset, and our dataset. When compared with the state-of-art mapping methods, our method has a great advantage in terms of both quality and speed, which meets the needs of robotic mapping and navigation.

Handheld Laser Scanner Research

2019 ◽  
Vol 952 (10) ◽  
pp. 47-54
Author(s):  
A.V. Komissarov ◽  
A.V. Remizov ◽  
M.M. Shlyakhova ◽  
K.K. Yambaev
Keyword(s):  
Point Cloud ◽  
Three Dimensional ◽  
Laser Scanner ◽  
Test Site ◽  
Optical Systems ◽  
Advantages And Disadvantages ◽  
Site Survey ◽  
Laser Scanners ◽  
Three Dimensional Models ◽  
The Stability

The authors consider hand-held laser scanners, as a new photogrammetric tool for obtaining three-dimensional models of objects. The principle of their work and the newest optical systems based on various sensors measuring the depth of space are described in detail. The method of simultaneous navigation and mapping (SLAM) used for combining single scans into point cloud is outlined. The formulated tasks and methods for performing studies of the DotProduct (USA) hand-held laser scanner DPI?8X based on a test site survey are presented. The accuracy requirements for determining the coordinates of polygon points are given. The essence of the performed experimental research of the DPI?8X scanner is described, including scanning of a test object at various scanner distances, shooting a test polygon from various scanner positions and building point cloud, repeatedly shooting the same area of the polygon to check the stability of the scanner. The data on the assessment of accuracy and analysis of research results are given. Fields of applying hand-held laser scanners, their advantages and disadvantages are identified.

Sign in / Sign up

Export Citation Format

Share Document