3D Convolutional Neural Network for Semantic Scene Segmentation based on Unstructured Point Clouds

Background: While detecting the object directly from the 3D point cloud, the natural 3D patterns and invariance of 3D data are often obscure. Objective: In this work, we aimed at studying the 3D object detection from discrete, disordered and sparse 3D point clouds. Methods: The CNN is composed of the frustum sequence module, 3D instance segmentation module S-NET, 3D point cloud transformation module T-NET, and 3D boundary box estimation module E-NET. The search space of the object is determined by the frustum sequence module. The instance segmentation of the point cloud is performed by the 3D instance segmentation module. The 3D coordinates of the object are confirmed by the transformation module and the 3D bounding box estimation module. Results: Evaluated on KITTI benchmark dataset, our method outperforms the state of the art by remarkable margins while having real-time capability. Conclusion: We achieve real-time 3D object detection by proposing an improved convolutional neural network (CNN) based on image-driven point clouds.

Convolutional neural network in the classifying problem of point clouds in three-dimensional space

10.1109/itnt52450.2021.9649027 ◽

2021 ◽

Author(s):

Artyom Makovetskii ◽

Vitaly Kober ◽

Dmitrii Zhernov ◽

Alexei Voronin

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Dimensional Space ◽

Three Dimensional ◽

Point Clouds ◽

Three Dimensional Space

Segmentation and Multi-Scale Convolutional Neural Network-Based Classification of Airborne Laser Scanner Data

Sensors ◽

10.3390/s18103347 ◽

2018 ◽

Vol 18 (10) ◽

pp. 3347 ◽

Cited By ~ 14

Author(s):

Zhishuang Yang ◽

Bo Tan ◽

Huikun Pei ◽

Wanshou Jiang

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Laser Scanning ◽

Laser Scanner ◽

Region Growing ◽

Point Clouds ◽

Classification Method ◽

Multi Scale ◽

Airborne Laser

The classification of point clouds is a basic task in airborne laser scanning (ALS) point cloud processing. It is quite a challenge when facing complex observed scenes and irregular point distributions. In order to reduce the computational burden of the point-based classification method and improve the classification accuracy, we present a segmentation and multi-scale convolutional neural network-based classification method. Firstly, a three-step region-growing segmentation method was proposed to reduce both under-segmentation and over-segmentation. Then, a feature image generation method was used to transform the 3D neighborhood features of a point into a 2D image. Finally, feature images were treated as the input of a multi-scale convolutional neural network for training and testing tasks. In order to obtain performance comparisons with existing approaches, we evaluated our framework using the International Society for Photogrammetry and Remote Sensing Working Groups II/4 (ISPRS WG II/4) 3D labeling benchmark tests. The experiment result, which achieved 84.9% overall accuracy and 69.2% of average F1 scores, has a satisfactory performance over all participating approaches analyzed.

Convolutional Neural Network for Extracting 3D Point Clouds of Fibrous Web From Multi-Focus Images

IEEE Access ◽

10.1109/access.2020.2993625 ◽

2020 ◽

Vol 8 ◽

pp. 87857-87869

Author(s):

Jue Hou ◽

Wenbin Ouyang ◽

Bugao Xu ◽

Rongwu Wang

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Point Clouds ◽

3D Point Clouds

Extraction of Guardrails from MMS Data Using Convolutional Neural Network

International Journal of Automation Technology ◽

10.20965/ijat.2021.p0258 ◽

2021 ◽

Vol 15 (3) ◽

pp. 258-267

Author(s):

Hiroki Matsumoto ◽

◽

Yuma Mori ◽

Hiroshi Masuda

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Asset Management ◽

Extraction Method ◽

Point Clouds ◽

Mobile Mapping ◽

Method Point ◽

Mobile Mapping Systems ◽

Capture Point ◽

Mapping Systems

Mobile mapping systems can capture point clouds and digital images of roadside objects. Such data are useful for maintenance, asset management, and 3D map creation. In this paper, we discuss methods for extracting guardrails that separate roadways and walkways. Since there are various shape patterns for guardrails in Japan, flexible methods are required for extracting them. We propose a new extraction method based on point processing and a convolutional neural network (CNN). In our method, point clouds and images are segmented into small fragments, and their features are extracted using CNNs for images and point clouds. Then, features from images and point clouds are combined and investigated using whether they are guardrails or not. Based on our experiments, our method could extract guardrails from point clouds with a high success rate.

A 3-D Lightweight Convolutional Neural Network for Detecting Docking Structures in Cluttered Environments

Marine Technology Society Journal ◽

10.4031/mtsj.55.4.9 ◽

2021 ◽

Vol 55 (4) ◽

pp. 88-98

Author(s):

Maria Inês Pereira ◽

Pedro Nuno Leite ◽

Andry Maykol Pinto

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Research Work ◽

Point Clouds ◽

Measurement Unit ◽

Maritime Industry ◽

Worst Case ◽

Cluttered Environments ◽

Autonomous Surface Vehicles ◽

Average Accuracy

Abstract The maritime industry has been following the paradigm shift toward the automation of typically intelligent procedures, with research regarding autonomous surface vehicles (ASVs) having seen an upward trend in recent years. However, this type of vehicle cannot be employed on a full scale until a few challenges are solved. For example, the docking process of an ASV is still a demanding task that currently requires human intervention. This research work proposes a volumetric convolutional neural network (vCNN) for the detection of docking structures from 3-D data, developed according to a balance between precision and speed. Another contribution of this article is a set of synthetically generated data regarding the context of docking structures. The dataset is composed of LiDAR point clouds, stereo images, GPS, and Inertial Measurement Unit (IMU) information. Several robustness tests carried out with different levels of Gaussian noise demonstrated an average accuracy of 93.34% and a deviation of 5.46% for the worst case. Furthermore, the system was fine-tuned and evaluated in a real commercial harbor, achieving an accuracy of over 96%. The developed classifier is able to detect different types of structures and works faster than other state-of-the-art methods that establish their performance in real environments.

Hierarchical Depthwise Graph Convolutional Neural Network for 3D Semantic Segmentation of Point Clouds

2019 International Conference on Robotics and Automation (ICRA) ◽

10.1109/icra.2019.8794052 ◽

2019 ◽

Cited By ~ 6

Author(s):

Zhidong Liang ◽

Ming Yang ◽

Liuyuan Deng ◽

Chunxiang Wang ◽

Bing Wang

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Semantic Segmentation ◽

Point Clouds

Generalized Sparse Convolutional Neural Networks for Semantic Segmentation of Point Clouds Derived from Tri-Stereo Satellite Imagery

Remote Sensing ◽

10.3390/rs12081289 ◽

2020 ◽

Vol 12 (8) ◽

pp. 1289

Author(s):

Stefan Bachhofner ◽

Ana-Maria Loghin ◽

Johannes Otepka ◽

Norbert Pfeifer ◽

Michael Hornacek ◽

...

Keyword(s):

Neural Network ◽

Decision Tree ◽

Convolutional Neural Network ◽

Semantic Segmentation ◽

Point Clouds ◽

Representation Learning ◽

Geometric Features ◽

Color Information ◽

Geometric Information ◽

Learning Technique

We studied the applicability of point clouds derived from tri-stereo satellite imagery for semantic segmentation for generalized sparse convolutional neural networks by the example of an Austrian study area. We examined, in particular, if the distorted geometric information, in addition to color, influences the performance of segmenting clutter, roads, buildings, trees, and vehicles. In this regard, we trained a fully convolutional neural network that uses generalized sparse convolution one time solely on 3D geometric information (i.e., 3D point cloud derived by dense image matching), and twice on 3D geometric as well as color information. In the first experiment, we did not use class weights, whereas in the second we did. We compared the results with a fully convolutional neural network that was trained on a 2D orthophoto, and a decision tree that was once trained on hand-crafted 3D geometric features, and once trained on hand-crafted 3D geometric as well as color features. The decision tree using hand-crafted features has been successfully applied to aerial laser scanning data in the literature. Hence, we compared our main interest of study, a representation learning technique, with another representation learning technique, and a non-representation learning technique. Our study area is located in Waldviertel, a region in Lower Austria. The territory is a hilly region covered mainly by forests, agriculture, and grasslands. Our classes of interest are heavily unbalanced. However, we did not use any data augmentation techniques to counter overfitting. For our study area, we reported that geometric and color information only improves the performance of the Generalized Sparse Convolutional Neural Network (GSCNN) on the dominant class, which leads to a higher overall performance in our case. We also found that training the network with median class weighting partially reverts the effects of adding color. The network also started to learn the classes with lower occurrences. The fully convolutional neural network that was trained on the 2D orthophoto generally outperforms the other two with a kappa score of over 90% and an average per class accuracy of 61%. However, the decision tree trained on colors and hand-crafted geometric features has a 2% higher accuracy for roads.

Point-PlaneNet: Plane kernel based convolutional neural network for point clouds analysis

Digital Signal Processing ◽

10.1016/j.dsp.2019.102633 ◽

2020 ◽

Vol 98 ◽

pp. 102633

Author(s):

S.M. Moein Peyghambarzadeh ◽

Fatemeh Azizmalayeri ◽

Hassan Khotanlou ◽

Amir Salarpour

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Point Clouds