Region-Based Edge Convolutions With Geometric Attributes for the Semantic Segmentation of Large-Scale 3-D Point Clouds

Semantic segmentation of large-scale outdoor 3D LiDAR point clouds becomes essential to understand the scene environment in various applications, such as geometry mapping, autonomous driving, and more. With an advantage of being a 3D metric space, 3D LiDAR point clouds, on the other hand, pose a challenge for a deep learning approach, due to their unstructured, unorder, irregular, and large-scale characteristics. Therefore, this paper presents an encoder–decoder shared multi-layer perceptron (MLP) with multiple losses, to address an issue of this semantic segmentation. The challenge rises a trade-off between efficiency and effectiveness in performance. To balance this trade-off, we proposed common mechanisms, which is simple and yet effective, by defining a random point sampling layer, an attention-based pooling layer, and a summation of multiple losses integrated with the encoder–decoder shared MLPs method for the large-scale outdoor point clouds semantic segmentation. We conducted our experiments on the following two large-scale benchmark datasets: Toronto-3D and DALES dataset. Our experimental results achieved an overall accuracy (OA) and a mean intersection over union (mIoU) of both the Toronto-3D dataset, with 83.60% and 71.03%, and the DALES dataset, with 76.43% and 59.52%, respectively. Additionally, our proposed method performed a few numbers of parameters of the model, and faster than PointNet++ by about three times during inferencing.

Download Full-text

JSNet: Joint Instance and Semantic Segmentation of 3D Point Clouds

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i07.6994 ◽

2020 ◽

Vol 34 (07) ◽

pp. 12951-12958 ◽

Cited By ~ 3

Author(s):

Lin Zhao ◽

Wenbing Tao

Keyword(s):

Point Cloud ◽

Large Scale ◽

Feature Fusion ◽

Mean Shift ◽

Semantic Segmentation ◽

Point Clouds ◽

Semantic Features ◽

Backbone Network ◽

3D Point Clouds ◽

Instance Segmentation

In this paper, we propose a novel joint instance and semantic segmentation approach, which is called JSNet, in order to address the instance and semantic segmentation of 3D point clouds simultaneously. Firstly, we build an effective backbone network to extract robust features from the raw point clouds. Secondly, to obtain more discriminative features, a point cloud feature fusion module is proposed to fuse the different layer features of the backbone network. Furthermore, a joint instance semantic segmentation module is developed to transform semantic features into instance embedding space, and then the transformed features are further fused with instance features to facilitate instance segmentation. Meanwhile, this module also aggregates instance features into semantic feature space to promote semantic segmentation. Finally, the instance predictions are generated by applying a simple mean-shift clustering on instance embeddings. As a result, we evaluate the proposed JSNet on a large-scale 3D indoor point cloud dataset S3DIS and a part dataset ShapeNet, and compare it with existing approaches. Experimental results demonstrate our approach outperforms the state-of-the-art method in 3D instance segmentation with a significant improvement in 3D semantic prediction and our method is also beneficial for part segmentation. The source code for this work is available at https://github.com/dlinzhao/JSNet.

Download Full-text

EDC-Net: Edge Detection Capsule Network for 3D Point Clouds

Applied Sciences ◽

10.3390/app11041833 ◽

2021 ◽

Vol 11 (4) ◽

pp. 1833 ◽

Cited By ~ 1

Author(s):

Dena Bazazian ◽

M. Eulàlia Parés

Keyword(s):

Edge Detection ◽

Network Architecture ◽

Large Scale ◽

Semantic Segmentation ◽

Point Clouds ◽

3D Point Clouds ◽

Abstract Shape ◽

Edge Points ◽

Weakly Supervised ◽

Edge Features

Edge features in point clouds are prominent due to the capability of describing an abstract shape of a set of points. Point clouds obtained by 3D scanner devices are often immense in terms of size. Edges are essential features in large scale point clouds since they are capable of describing the shapes in down-sampled point clouds while maintaining the principal information. In this paper, we tackle challenges of edge detection tasks in 3D point clouds. To this end, we propose a novel technique to detect edges of point clouds based on a capsule network architecture. In this approach, we define the edge detection task of point clouds as a semantic segmentation problem. We built a classifier through the capsules to predict edge and non-edge points in 3D point clouds. We applied a weakly-supervised learning approach in order to improve the performance of our proposed method and built in the capability of testing the technique in wider range of shapes. We provide several quantitative and qualitative experimental results to demonstrate the robustness of our proposed EDC-Net for edge detection in 3D point clouds. We performed a statistical analysis over the ABC and ShapeNet datasets. Our numerical results demonstrate the robust and efficient performance of EDC-Net.

Download Full-text

Continuous Mapping Convolution for Large-Scale Point Clouds Semantic Segmentation

IEEE Geoscience and Remote Sensing Letters ◽

10.1109/lgrs.2021.3107006 ◽

2021 ◽

pp. 1-5

Author(s):

Kunping Yan ◽

Qingyong Hu ◽

Hanyun Wang ◽

Xiaohong Huang ◽

Li Li ◽

...

Keyword(s):

Large Scale ◽

Continuous Mapping ◽

Semantic Segmentation ◽

Point Clouds ◽

Scale Point

Download Full-text

Ground-distance segmentation of 3D LiDAR point cloud toward autonomous driving

APSIPA Transactions on Signal and Information Processing ◽

10.1017/atsip.2020.21 ◽

2020 ◽

Vol 9 ◽

Author(s):

Jian Wu ◽

Qingxiong Yang

Keyword(s):

Point Cloud ◽

Large Scale ◽

Ground Plane ◽

Semantic Segmentation ◽

Point Clouds ◽

Autonomous Driving ◽

Urban Environments ◽

Cloud Data ◽

Dense Point ◽

3D Lidar

In this paper, we study the semantic segmentation of 3D LiDAR point cloud data in urban environments for autonomous driving, and a method utilizing the surface information of the ground plane was proposed. In practice, the resolution of a LiDAR sensor installed in a self-driving vehicle is relatively low and thus the acquired point cloud is indeed quite sparse. While recent work on dense point cloud segmentation has achieved promising results, the performance is relatively low when directly applied to sparse point clouds. This paper is focusing on semantic segmentation of the sparse point clouds obtained from 32-channel LiDAR sensor with deep neural networks. The main contribution is the integration of the ground information which is used to group ground points far away from each other. Qualitative and quantitative experiments on two large-scale point cloud datasets show that the proposed method outperforms the current state-of-the-art.

Download Full-text

SUBMANIFOLD SPARSE CONVOLUTIONAL NETWORKS FOR SEMANTIC SEGMENTATION OF LARGE-SCALE ALS POINT CLOUDS

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-annals-iv-2-w5-77-2019 ◽

2019 ◽

Vol IV-2/W5 ◽

pp. 77-84 ◽

Cited By ~ 5

Author(s):

S. Schmohl ◽

U. Sörgel

Keyword(s):

Laser Scanning ◽

Large Scale ◽

Computing Time ◽

Semantic Segmentation ◽

Point Clouds ◽

Convolutional Networks ◽

Airborne Laser ◽

Semantic Labeling ◽

3D Data ◽

Automated Processing

Abstract. Semantic segmentation of point clouds is one of the main steps in automated processing of data from Airborne Laser Scanning (ALS). Established methods usually require expensive calculation of handcrafted, point-wise features. In contrast, Convolutional Neural Networks (CNNs) have been established as powerful classifiers, which at the same time also learn a set of features by themselves. However, their application to ALS data is not trivial. Pure 3D CNNs require a lot of memory and computing time, therefore most related approaches project ALS point clouds into two-dimensional images. Sparse Submanifold Convolutional Networks (SSCNs) address this issue by exploiting the sparsity often inherent in 3D data. In this work, we propose the application of SSCNs for efficient semantic segmentation of voxelized ALS point clouds in an end-to-end encoder-decoder architecture. We evaluate this method on the ISPRS Vaihingen 3D Semantic Labeling benchmark and achieve state-of-the-art 85.0% overall accuracy. Furthermore, we demonstrate its capabilities regarding large-scale ALS data by classifying a 2.5&thinsp;km2 subset containing 41&thinsp;M points from the Actueel Hoogtebestand Nederland (AHN3) with 95% overall accuracy in just 48&thinsp;s inference time or with 96% in 108&thinsp;s.

Download Full-text

Semantic segmentation of large-scale point clouds based on dilated nearest neighbors graph

Complex & Intelligent Systems ◽

10.1007/s40747-021-00618-0 ◽

2022 ◽

Author(s):

Lei Wang ◽

Jiaji Wu ◽

Xunyu Liu ◽

Xiaoliang Ma ◽

Jun Cheng

Keyword(s):

Large Scale ◽

Nearest Neighbor ◽

Three Dimensional ◽

Semantic Segmentation ◽

Point Clouds ◽

Sampling Point ◽

Automatic Driving ◽

Scale Point ◽

Memory Efficient

AbstractThree-dimensional (3D) semantic segmentation of point clouds is important in many scenarios, such as automatic driving, robotic navigation, while edge computing is indispensable in the devices. Deep learning methods based on point sampling prove to be computation and memory efficient to tackle large-scale point clouds (e.g. millions of points). However, some local features may be abandoned while sampling. In this paper, We present one end-to-end 3D semantic segmentation framework based on dilated nearest neighbor encoding. Instead of down-sampling point cloud directly, we propose a dilated nearest neighbor encoding module to broaden the network’s receptive field to learn more 3D geometric information. Without increase of network parameters, our method is computation and memory efficient for large-scale point clouds. We have evaluated the dilated nearest neighbor encoding in two different networks. The first is the random sampling with local feature aggregation. The second is the Point Transformer. We have evaluated the quality of the semantic segmentation on the benchmark 3D dataset S3DIS, and demonstrate that the proposed dilated nearest neighbor encoding exhibited stable advantages over baseline and competing methods.

Download Full-text