Hierarchical semantic segmentation of urban scene point clouds via group proposal and graph attention network

Urban object segmentation and classification tasks are critical data processing steps in scene understanding, intelligent vehicles and 3D high-precision maps. Semantic segmentation of 3D point clouds is the foundational step in object recognition. To identify the intersecting objects and improve the accuracy of classification, this paper proposes a segment-based classification method for 3D point clouds. This method firstly divides points into multi-scale supervoxels and groups them by proposed inverse node graph (IN-Graph) construction, which does not need to define prior information about the node, it divides supervoxels by judging the connection state of edges between them. This method reaches minimum global energy by graph cutting, obtains the structural segments as completely as possible, and retains boundaries at the same time. Then, the random forest classifier is utilized for supervised classification. To deal with the mislabeling of scattered fragments, higher-order CRF with small-label cluster optimization is proposed to refine the classification results. Experiments were carried out on mobile laser scan (MLS) point dataset and terrestrial laser scan (TLS) points dataset, and the results show that overall accuracies of 97.57% and 96.39% were obtained in the two datasets. The boundaries of objects were retained well, and the method achieved a good result in the classification of cars and motorcycles. More experimental analyses have verified the advantages of the proposed method and proved the practicability and versatility of the method.

Download Full-text

AN IMPROVED AUTOMATIC POINTWISE SEMANTIC SEGMENTATION OF A 3D URBAN SCENE FROM MOBILE TERRESTRIAL AND AIRBORNE LIDAR POINT CLOUDS: A MACHINE LEARNING APPROACH

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-annals-iv-4-w8-139-2019 ◽

2019 ◽

Vol IV-4/W8 ◽

pp. 139-146

Author(s):

X.-F. Xing ◽

M. A. Mostafavi ◽

G. Edwards ◽

N. Sabo

Keyword(s):

Machine Learning ◽

Urban Areas ◽

Learning Algorithm ◽

Semantic Segmentation ◽

Point Clouds ◽

Airborne Lidar ◽

Urban Scenes ◽

Machine Learning Approach ◽

Urban Scene ◽

Point Level

Abstract. Automatic semantic segmentation of point clouds observed in a 3D complex urban scene is a challenging issue. Semantic segmentation of urban scenes based on machine learning algorithm requires appropriate features to distinguish objects from mobile terrestrial and airborne LiDAR point clouds in point level. In this paper, we propose a pointwise semantic segmentation method based on our proposed features derived from Difference of Normal and the features “directional height above” that compare height difference between a given point and neighbors in eight directions in addition to the features based on normal estimation. Random forest classifier is chosen to classify points in mobile terrestrial and airborne LiDAR point clouds. The results obtained from our experiments show that the proposed features are effective for semantic segmentation of mobile terrestrial and airborne LiDAR point clouds, especially for vegetation, building and ground classes in an airborne LiDAR point clouds in urban areas.

Download Full-text

Point attention network for semantic segmentation of 3D point clouds

Pattern Recognition ◽

10.1016/j.patcog.2020.107446 ◽

2020 ◽

Vol 107 ◽

pp. 107446 ◽

Cited By ~ 1

Author(s):

Mingtao Feng ◽

Liang Zhang ◽

Xuefei Lin ◽

Syed Zulqarnain Gilani ◽

Ajmal Mian

Keyword(s):

Semantic Segmentation ◽

Point Clouds ◽

Attention Network ◽

3D Point Clouds

Download Full-text

FUSION OF FEATURE BASED AND DEEP LEARNING METHODS FOR CLASSIFICATION OF MMS POINT CLOUDS

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-2-w16-235-2019 ◽

2019 ◽

Vol XLII-2/W16 ◽

pp. 235-242 ◽

Cited By ~ 1

Author(s):

D. Tosic ◽

S. Tuttas ◽

L. Hoegner ◽

U. Stilla

Keyword(s):

Deep Learning ◽

Point Cloud ◽

Semantic Segmentation ◽

Point Clouds ◽

Local Geometry ◽

Learning Approach ◽

Semantic Classification ◽

Feature Based ◽

Urban Scene

Abstract. This work proposes an approach for semantic classification of an outdoor-scene point cloud acquired with a high precision Mobile Mapping System (MMS), with major goal to contribute to the automatic creation of High Definition (HD) Maps. The automatic point labeling is achieved by utilizing the combination of a feature-based approach for semantic classification of point clouds and a deep learning approach for semantic segmentation of images. Both, point cloud data, as well as the data from a multi-camera system are used for gaining spatial information in an urban scene. Two types of classification applied for this task are: 1) Feature-based approach, in which the point cloud is organized into a supervoxel structure for capturing geometric characteristics of points. Several geometric features are then extracted for appropriate representation of the local geometry, followed by removing the effect of local tendency for each supervoxel to enhance the distinction between similar structures. And lastly, the Random Forests (RF) algorithm is applied in the classification phase, for assigning labels to supervoxels and therefore to points within them. 2) The deep learning approach is employed for semantic segmentation of MMS images of the same scene. To achieve this, an implementation of Pyramid Scene Parsing Network is used. Resulting segmented images with each pixel containing a class label are then projected onto the point cloud, enabling label assignment for each point. At the end, experiment results are presented from a complex urban scene and the performance of this method is evaluated on a manually labeled dataset, for the deep learning and feature-based classification individually, as well as for the result of the labels fusion. The achieved overall accuracy with fusioned output is 0.87 on the final test set, which significantly outperforms the results of individual methods on the same point cloud. The labeled data is published on the TUM-PF Semantic-Labeling-Benchmark.

Download Full-text

Local Fusion Attention Network for Semantic Segmentation of Building Facade Point Clouds

IEEE Geoscience and Remote Sensing Letters ◽

10.1109/lgrs.2021.3126735 ◽

2021 ◽

pp. 1-1

Author(s):

Yanfei Su ◽

Weiquan Liu ◽

Ming Cheng ◽

Zhimin Yuan ◽

Cheng Wang

Keyword(s):

Semantic Segmentation ◽

Point Clouds ◽

Attention Network ◽

Building Facade

Download Full-text

Towards Urban Scene Semantic Segmentation with Deep Learning from LiDAR Point Clouds: A Case Study in Baden-Württemberg, Germany

Remote Sensing ◽

10.3390/rs13163220 ◽

2021 ◽

Vol 13 (16) ◽

pp. 3220

Author(s):

Yanling Zou ◽

Holger Weinacker ◽

Barbara Koch

Keyword(s):

Deep Learning ◽

Smart Cities ◽

Real Life ◽

Semantic Segmentation ◽

Point Clouds ◽

Accuracy Score ◽

City Management ◽

Data Preparation ◽

3D Point Clouds ◽

Urban Scene

An accurate understanding of urban objects is critical for urban modeling, intelligent infrastructure planning and city management. The semantic segmentation of light detection and ranging (LiDAR) point clouds is a fundamental approach for urban scene analysis. Over the last years, several methods have been developed to segment urban furniture with point clouds. However, the traditional processing of large amounts of spatial data has become increasingly costly, both time-wise and financially. Recently, deep learning (DL) techniques have been increasingly used for 3D segmentation tasks. Yet, most of these deep neural networks (DNNs) were conducted on benchmarks. It is, therefore, arguable whether DL approaches can achieve the state-of-the-art performance of 3D point clouds segmentation in real-life scenarios. In this research, we apply an adapted DNN (ARandLA-Net) to directly process large-scale point clouds. In particular, we develop a new paradigm for training and validation, which presents a typical urban scene in central Europe (Munzingen, Freiburg, Baden-Württemberg, Germany). Our dataset consists of nearly 390 million dense points acquired by Mobile Laser Scanning (MLS), which has a rather larger quantity of sample points in comparison to existing datasets and includes meaningful object categories that are particular to applications for smart cities and urban planning. We further assess the DNN on our dataset and investigate a number of key challenges from varying aspects, such as data preparation strategies, the advantage of color information and the unbalanced class distribution in the real world. The final segmentation model achieved a mean Intersection-over-Union (mIoU) score of 54.4% and an overall accuracy score of 83.9%. Our experiments indicated that different data preparation strategies influenced the model performance. Additional RGB information yielded an approximately 4% higher mIoU score. Our results also demonstrate that the use of weighted cross-entropy with inverse square root frequency loss led to better segmentation performance than when other losses were considered.

Download Full-text

MS-RRFSegNet: Multiscale Regional Relation Feature Segmentation Network for Semantic Segmentation of Urban Scene Point Clouds

IEEE Transactions on Geoscience and Remote Sensing ◽

10.1109/tgrs.2020.2985695 ◽

2020 ◽

Vol 58 (12) ◽

pp. 8301-8315 ◽

Cited By ~ 1

Author(s):

Haifeng Luo ◽

Chongcheng Chen ◽

Lina Fang ◽

Kourosh Khoshelham ◽

Guixi Shen

Keyword(s):

Semantic Segmentation ◽

Point Clouds ◽

Feature Segmentation ◽

Urban Scene

Download Full-text

Hybrid Multiple Attention Network for Semantic Segmentation in Aerial Images

IEEE Transactions on Geoscience and Remote Sensing ◽

10.1109/tgrs.2021.3065112 ◽

2021 ◽

pp. 1-18

Author(s):

Ruigang Niu ◽

Xian Sun ◽

Yu Tian ◽

Wenhui Diao ◽

Kaiqiang Chen ◽

...

Keyword(s):

Semantic Segmentation ◽

Aerial Images ◽

Attention Network

Download Full-text

GA-NET: Global Attention Network for Point Cloud Semantic Segmentation

IEEE Signal Processing Letters ◽

10.1109/lsp.2021.3082851 ◽

2021 ◽

pp. 1-1

Author(s):

Shuang Deng ◽

Qiulei Dong

Keyword(s):

Point Cloud ◽

Semantic Segmentation ◽

Attention Network

Download Full-text

Transformer Meets Convolution: A Bilateral Awareness Network for Semantic Segmentation of Very Fine Resolution Urban Scene Images

Remote Sensing ◽

10.3390/rs13163065 ◽

2021 ◽

Vol 13 (16) ◽

pp. 3065

Author(s):

Libo Wang ◽

Rui Li ◽

Dongzhi Wang ◽

Chenxi Duan ◽

Teng Wang ◽

...

Keyword(s):

Large Scale ◽

Texture Features ◽

Semantic Segmentation ◽

Autonomous Driving ◽

Research Field ◽

Learning Approaches ◽

Fine Grained ◽

Urban Scene ◽

Fine Resolution ◽

With Memory

Semantic segmentation from very fine resolution (VFR) urban scene images plays a significant role in several application scenarios including autonomous driving, land cover classification, urban planning, etc. However, the tremendous details contained in the VFR image, especially the considerable variations in scale and appearance of objects, severely limit the potential of the existing deep learning approaches. Addressing such issues represents a promising research field in the remote sensing community, which paves the way for scene-level landscape pattern analysis and decision making. In this paper, we propose a Bilateral Awareness Network which contains a dependency path and a texture path to fully capture the long-range relationships and fine-grained details in VFR images. Specifically, the dependency path is conducted based on the ResT, a novel Transformer backbone with memory-efficient multi-head self-attention, while the texture path is built on the stacked convolution operation. In addition, using the linear attention mechanism, a feature aggregation module is designed to effectively fuse the dependency features and texture features. Extensive experiments conducted on the three large-scale urban scene image segmentation datasets, i.e., ISPRS Vaihingen dataset, ISPRS Potsdam dataset, and UAVid dataset, demonstrate the effectiveness of our BANet. Specifically, a 64.6% mIoU is achieved on the UAVid dataset.

Download Full-text