Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusion

AbstractThe irregular domain and lack of ordering make it challenging to design deep neural networks for point cloud processing. This paper presents a novel framework named Point Cloud Transformer (PCT) for point cloud learning. PCT is based on Transformer, which achieves huge success in natural language processing and displays great potential in image processing. It is inherently permutation invariant for processing a sequence of points, making it well-suited for point cloud learning. To better capture local context within the point cloud, we enhance input embedding with the support of farthest point sampling and nearest neighbor search. Extensive experiments demonstrate that the PCT achieves the state-of-the-art performance on shape classification, part segmentation, semantic segmentation, and normal estimation tasks.

Download Full-text

GA-NET: Global Attention Network for Point Cloud Semantic Segmentation

IEEE Signal Processing Letters ◽

10.1109/lsp.2021.3082851 ◽

2021 ◽

pp. 1-1

Author(s):

Shuang Deng ◽

Qiulei Dong

Keyword(s):

Point Cloud ◽

Semantic Segmentation ◽

Attention Network

Download Full-text

Semantic Segmentation in Depth Data: A Comparative Evaluation of Image and Point Cloud Based Methods

10.1109/icip42928.2021.9506334 ◽

2021 ◽

Author(s):

Jigyasa Singh Katrolia ◽

Lars Kramer ◽

Jason Rambach ◽

Bruno Mirbach ◽

Didier Stricker

Keyword(s):

Comparative Evaluation ◽

Point Cloud ◽

Semantic Segmentation ◽

Depth Data

Download Full-text

MLFNet-Point Cloud Semantic Segmentation Convolution Network Based on Multi-scale Feature Fusion

IEEE Access ◽

10.1109/access.2021.3057612 ◽

2021 ◽

pp. 1-1

Author(s):

Jingfang Yang ◽

Bochang Zou ◽

Huadong Qiu ◽

Zhi Li

Keyword(s):

Point Cloud ◽

Feature Fusion ◽

Semantic Segmentation ◽

Scale Feature ◽

Multi Scale

Download Full-text

KVGCN: A KNN Searching and VLAD Combined Graph Convolutional Network for Point Cloud Segmentation

Remote Sensing ◽

10.3390/rs13051003 ◽

2021 ◽

Vol 13 (5) ◽

pp. 1003

Author(s):

Nan Luo ◽

Hongquan Yu ◽

Zhenfeng Huo ◽

Jinhui Liu ◽

Quan Wang ◽

...

Keyword(s):

Point Cloud ◽

Nearest Neighbor ◽

Semantic Segmentation ◽

K Nearest Neighbor ◽

Topological Graph ◽

Convolutional Network ◽

Cloud Data ◽

Nearest Neighbor Searching ◽

Point Cloud Segmentation ◽

Local Feature Extraction

Semantic segmentation of the sensed point cloud data plays a significant role in scene understanding and reconstruction, robot navigation, etc. This work presents a Graph Convolutional Network integrating K-Nearest Neighbor searching (KNN) and Vector of Locally Aggregated Descriptors (VLAD). KNN searching is utilized to construct the topological graph of each point and its neighbors. Then, we perform convolution on the edges of constructed graph to extract representative local features by multiple Multilayer Perceptions (MLPs). Afterwards, a trainable VLAD layer, NetVLAD, is embedded in the feature encoder to aggregate the local and global contextual features. The designed feature encoder is repeated for multiple times, and the extracted features are concatenated in a jump-connection style to strengthen the distinctiveness of features and thereby improve the segmentation. Experimental results on two datasets show that the proposed work settles the shortcoming of insufficient local feature extraction and promotes the accuracy (mIoU 60.9% and oAcc 87.4% for S3DIS) of semantic segmentation comparing to existing models.

Download Full-text

EXPLORING ALS AND DIM DATA FOR SEMANTIC SEGMENTATION USING CNNS

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-1-347-2018 ◽

2018 ◽

Vol XLII-1 ◽

pp. 347-354 ◽

Cited By ~ 5

Author(s):

F. Politz ◽

M. Sester

Keyword(s):

Point Cloud ◽

Laser Scanning ◽

Semantic Segmentation ◽

Point Clouds ◽

Good Alternative ◽

Aerial Images ◽

Learning Approaches ◽

Advantages And Disadvantages ◽

Sensing Applications ◽

High Level

Abstract. Over the past years, the algorithms for dense image matching (DIM) to obtain point clouds from aerial images improved significantly. Consequently, DIM point clouds are now a good alternative to the established Airborne Laser Scanning (ALS) point clouds for remote sensing applications. In order to derive high-level applications such as digital terrain models or city models, each point within a point cloud must be assigned a class label. Usually, ALS and DIM are labelled with different classifiers due to their varying characteristics. In this work, we explore both point cloud types in a fully convolutional encoder-decoder network, which learns to classify ALS as well as DIM point clouds. As input, we project the point clouds onto a 2D image raster plane and calculate the minimal, average and maximal height values for each raster cell. The network then differentiates between the classes ground, non-ground, building and no data. We test our network in six training setups using only one point cloud type, both point clouds as well as several transfer-learning approaches. We quantitatively and qualitatively compare all results and discuss the advantages and disadvantages of all setups. The best network achieves an overall accuracy of 96% in an ALS and 83% in a DIM test set.

Download Full-text

UNDERSTANDING 3D POINT CLOUD DEEP NEURAL NETWORKS BY VISUALIZATION TECHNIQUES

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xliii-b2-2020-651-2020 ◽

2020 ◽

Vol XLIII-B2-2020 ◽

pp. 651-657

Author(s):

Y. Cao ◽

M. Previtali ◽

M. Scaioni

Keyword(s):

Point Cloud ◽

Semantic Segmentation ◽

Point Clouds ◽

Learning Networks ◽

Quantitative Investigation ◽

Different Types ◽

Visualization Techniques ◽

Point Cloud Classification ◽

Learned Features ◽

Excellent Tool

Abstract. In the wake of the success of Deep Learning Networks (DLN) for image recognition, object detection, shape classification and semantic segmentation, this approach has proven to be both a major breakthrough and an excellent tool in point cloud classification. However, understanding how different types of DLN achieve still lacks. In several studies the output of segmentation/classification process is compared against benchmarks, but the network is treated as a “black-box” and intermediate steps are not deeply analysed. Specifically, here the following questions are discussed: (1) what exactly did DLN learn from a point cloud? (2) On the basis of what information do DLN make decisions? To conduct such a quantitative investigation of these DLN applied to point clouds, this paper investigates the visual interpretability for the decision-making process. Firstly, we introduce a reconstruction network able to reconstruct and visualise the learned features, in order to face with question (1). Then, we propose 3DCAM to indicate the discriminative point cloud regions used by these networks to identify that category, thus dealing with question (2). Through answering the above two questions, the paper would like to offer some initial solutions to better understand the application of DLN to point clouds.

Download Full-text