EDC-Net: Edge Detection Capsule Network for 3D Point Clouds

Edge features in point clouds are prominent due to the capability of describing an abstract shape of a set of points. Point clouds obtained by 3D scanner devices are often immense in terms of size. Edges are essential features in large scale point clouds since they are capable of describing the shapes in down-sampled point clouds while maintaining the principal information. In this paper, we tackle challenges of edge detection tasks in 3D point clouds. To this end, we propose a novel technique to detect edges of point clouds based on a capsule network architecture. In this approach, we define the edge detection task of point clouds as a semantic segmentation problem. We built a classifier through the capsules to predict edge and non-edge points in 3D point clouds. We applied a weakly-supervised learning approach in order to improve the performance of our proposed method and built in the capability of testing the technique in wider range of shapes. We provide several quantitative and qualitative experimental results to demonstrate the robustness of our proposed EDC-Net for edge detection in 3D point clouds. We performed a statistical analysis over the ABC and ShapeNet datasets. Our numerical results demonstrate the robust and efficient performance of EDC-Net.

Download Full-text

JSNet: Joint Instance and Semantic Segmentation of 3D Point Clouds

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i07.6994 ◽

2020 ◽

Vol 34 (07) ◽

pp. 12951-12958 ◽

Cited By ~ 3

Author(s):

Lin Zhao ◽

Wenbing Tao

Keyword(s):

Point Cloud ◽

Large Scale ◽

Feature Fusion ◽

Mean Shift ◽

Semantic Segmentation ◽

Point Clouds ◽

Semantic Features ◽

Backbone Network ◽

3D Point Clouds ◽

Instance Segmentation

In this paper, we propose a novel joint instance and semantic segmentation approach, which is called JSNet, in order to address the instance and semantic segmentation of 3D point clouds simultaneously. Firstly, we build an effective backbone network to extract robust features from the raw point clouds. Secondly, to obtain more discriminative features, a point cloud feature fusion module is proposed to fuse the different layer features of the backbone network. Furthermore, a joint instance semantic segmentation module is developed to transform semantic features into instance embedding space, and then the transformed features are further fused with instance features to facilitate instance segmentation. Meanwhile, this module also aggregates instance features into semantic feature space to promote semantic segmentation. Finally, the instance predictions are generated by applying a simple mean-shift clustering on instance embeddings. As a result, we evaluate the proposed JSNet on a large-scale 3D indoor point cloud dataset S3DIS and a part dataset ShapeNet, and compare it with existing approaches. Experimental results demonstrate our approach outperforms the state-of-the-art method in 3D instance segmentation with a significant improvement in 3D semantic prediction and our method is also beneficial for part segmentation. The source code for this work is available at https://github.com/dlinzhao/JSNet.

Download Full-text

Decoupled Iterative Deep Sensor Fusion for 3D Semantic Segmentation

International Journal of Semantic Computing ◽

10.1142/s1793351x21400067 ◽

2021 ◽

Vol 15 (03) ◽

pp. 293-312

Author(s):

Fabian Duerr ◽

Hendrik Weigel ◽

Jürgen Beyerer

Keyword(s):

Autonomous Vehicles ◽

Network Architecture ◽

Semantic Segmentation ◽

Point Clouds ◽

Sensor Data ◽

Range Image ◽

Fusion Strategy ◽

3D Point Clouds ◽

3D Environment ◽

Wide Range

One of the key tasks for autonomous vehicles or robots is a robust perception of their 3D environment, which is why autonomous vehicles or robots are equipped with a wide range of different sensors. Building upon a robust sensor setup, understanding and interpreting their 3D environment is the next important step. Semantic segmentation of 3D sensor data, e.g. point clouds, provides valuable information for this task and is often seen as key enabler for 3D scene understanding. This work presents an iterative deep fusion architecture for semantic segmentation of 3D point clouds, which builds upon a range image representation of the point clouds and additionally exploits camera features to increase accuracy and robustness. In contrast to other approaches, which fuse lidar and camera features once, the proposed fusion strategy iteratively combines and refines lidar and camera features at different scales inside the network architecture. Additionally, the proposed approach can deal with camera failure as well as jointly predict lidar and camera segmentation. We demonstrate the benefits of the presented iterative deep fusion approach on two challenging datasets, outperforming all range image-based lidar and fusion approaches. An in-depth evaluation underlines the effectiveness of the proposed fusion strategy and the potential of camera features for 3D semantic segmentation.

Download Full-text

GECNN for Weakly Supervised Semantic Segmentation of 3D Point Clouds

IEICE Transactions on Information and Systems ◽

10.1587/transinf.2021edp7134 ◽

2021 ◽

Vol E104.D (12) ◽

pp. 2237-2243

Author(s):

Zifen HE ◽

Shouye ZHU ◽

Ying HUANG ◽

Yinhui ZHANG

Keyword(s):

Semantic Segmentation ◽

Point Clouds ◽

3D Point Clouds ◽

Weakly Supervised

Download Full-text

Semantic Segmentation of Large-Scale 3D Point Clouds Using Sparse Convolutional Neural Networks

10.1145/3460569.3460584 ◽

2021 ◽

Author(s):

Li Gao ◽

Qian Gao

Keyword(s):

Neural Networks ◽

Convolutional Neural Networks ◽

Large Scale ◽

Semantic Segmentation ◽

Point Clouds ◽

3D Point Clouds

Download Full-text

Exploiting Structured CNNs For Semantic Segmentation Of Unstructured Point Clouds From LiDAR Sensor

Remote Sensing ◽

10.3390/rs13183621 ◽

2021 ◽

Vol 13 (18) ◽

pp. 3621

Author(s):

Muhammad Ibrahim ◽

Naveed Akhtar ◽

Khalil Ullah ◽

Ajmal Mian

Keyword(s):

Point Cloud ◽

Large Scale ◽

Semantic Segmentation ◽

Point Clouds ◽

Histogram Equalization ◽

Lidar Data ◽

3D Point Clouds ◽

Novel Method ◽

Public Datasets ◽

Processing Techniques

Accurate semantic segmentation of 3D point clouds is a long-standing problem in remote sensing and computer vision. Due to the unstructured nature of point clouds, designing deep neural architectures for point cloud semantic segmentation is often not straightforward. In this work, we circumvent this problem by devising a technique to exploit structured neural architectures for unstructured data. In particular, we employ the popular convolutional neural network (CNN) architectures to perform semantic segmentation of LiDAR data. We propose a projection-based scheme that performs an angle-wise slicing of large 3D point clouds and transforms those slices into 2D grids. Accounting for intensity and reflectivity of the LiDAR input, the 2D grid allows us to construct a pseudo image for the point cloud slice. We enhance this image with low-level image processing techniques of normalization, histogram equalization, and decorrelation stretch to suit our ultimate object of semantic segmentation. A large number of images thus generated are used to induce an encoder-decoder CNN model that learns to compute a segmented 2D projection of the scene, which we finally back project to the 3D point cloud. In addition to a novel method, this article also makes a second major contribution of introducing the enhanced version of our large-scale public PC-Urban outdoor dataset which is captured in a civic setup with an Ouster LiDAR sensor. The updated dataset (PC-Urban_V2) provides nearly 8 billion points including over 100 million points labeled for 25 classes of interest. We provide a thorough evaluation of our technique on PC-Urban_V2 and three other public datasets.

Download Full-text

JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D Point Clouds

Computer Vision – ECCV 2020 - Lecture Notes in Computer Science ◽

10.1007/978-3-030-58565-5_14 ◽

2020 ◽

pp. 222-239 ◽

Cited By ~ 1

Author(s):

Zeyu Hu ◽

Mingmin Zhen ◽

Xuyang Bai ◽

Hongbo Fu ◽

Chiew-lan Tai

Keyword(s):

Edge Detection ◽

Semantic Segmentation ◽

Point Clouds ◽

3D Point Clouds

Download Full-text

ThickSeg: Efficient semantic segmentation of large-scale 3D point clouds using multi-layer projection

Image and Vision Computing ◽

10.1016/j.imavis.2021.104161 ◽

2021 ◽

Vol 108 ◽

pp. 104161

Author(s):

Qian Gao ◽

Xukun Shen

Keyword(s):

Large Scale ◽

Semantic Segmentation ◽

Point Clouds ◽

3D Point Clouds

Download Full-text

SketchGNN: Semantic Sketch Segmentation with Graph Neural Networks

ACM Transactions on Graphics ◽

10.1145/3450284 ◽

2021 ◽

Vol 40 (3) ◽

pp. 1-13

Author(s):

Lumin Yang ◽

Jiajie Zhuang ◽

Hongbo Fu ◽

Xiangzhi Wei ◽

Kun Zhou ◽

...

Keyword(s):

Neural Network ◽

Neural Networks ◽

Network Architecture ◽

Large Scale ◽

State Of The Art ◽

Semantic Segmentation ◽

Structure Information ◽

Graph Neural Networks ◽

Node Labels ◽

Point Level

We introduce SketchGNN , a convolutional graph neural network for semantic segmentation and labeling of freehand vector sketches. We treat an input stroke-based sketch as a graph with nodes representing the sampled points along input strokes and edges encoding the stroke structure information. To predict the per-node labels, our SketchGNN uses graph convolution and a static-dynamic branching network architecture to extract the features at three levels, i.e., point-level, stroke-level, and sketch-level. SketchGNN significantly improves the accuracy of the state-of-the-art methods for semantic sketch segmentation (by 11.2% in the pixel-based metric and 18.2% in the component-based metric over a large-scale challenging SPG dataset) and has magnitudes fewer parameters than both image-based and sequence-based methods.

Download Full-text

An Inverse Node Graph-Based Method for the Urban Scene Segmentation of 3D Point Clouds

Remote Sensing ◽

10.3390/rs13153021 ◽

2021 ◽

Vol 13 (15) ◽

pp. 3021

Author(s):

Bufan Zhao ◽

Xianghong Hua ◽

Kegen Yu ◽

Xiaoxing He ◽

Weixing Xue ◽

...

Keyword(s):

Semantic Segmentation ◽

Point Clouds ◽

Intelligent Vehicles ◽

Critical Data ◽

Multi Scale ◽

3D Point Clouds ◽

Cluster Optimization ◽

Urban Scene ◽

Processing Steps

Urban object segmentation and classification tasks are critical data processing steps in scene understanding, intelligent vehicles and 3D high-precision maps. Semantic segmentation of 3D point clouds is the foundational step in object recognition. To identify the intersecting objects and improve the accuracy of classification, this paper proposes a segment-based classification method for 3D point clouds. This method firstly divides points into multi-scale supervoxels and groups them by proposed inverse node graph (IN-Graph) construction, which does not need to define prior information about the node, it divides supervoxels by judging the connection state of edges between them. This method reaches minimum global energy by graph cutting, obtains the structural segments as completely as possible, and retains boundaries at the same time. Then, the random forest classifier is utilized for supervised classification. To deal with the mislabeling of scattered fragments, higher-order CRF with small-label cluster optimization is proposed to refine the classification results. Experiments were carried out on mobile laser scan (MLS) point dataset and terrestrial laser scan (TLS) points dataset, and the results show that overall accuracies of 97.57% and 96.39% were obtained in the two datasets. The boundaries of objects were retained well, and the method achieved a good result in the classification of cars and motorcycles. More experimental analyses have verified the advantages of the proposed method and proved the practicability and versatility of the method.

Download Full-text