scholarly journals Study of the Effect of Exploiting 3D Semantic Segmentation in LiDAR Odometry

2020 ◽  
Vol 10 (16) ◽  
pp. 5657
Author(s):  
Francisco Miguel Moreno ◽  
Carlos Guindel ◽  
José María Armingol ◽  
Fernando García
Keyword(s):  
Point Cloud ◽  
Semantic Information ◽  
Semantic Segmentation ◽  
Raw Data ◽  
Special Relevance ◽  
Original Point

This paper presents a study of how the performance of LiDAR odometry is affected by the preprocessing of the point cloud through the use of 3D semantic segmentation. The study analyzed the estimated trajectories when the semantic information is exploited to filter the original raw data. Different filtering configurations were tested: raw (original point cloud), dynamic (dynamic obstacles are removed from the point cloud), dynamic vehicles (vehicles are removed), far (distant points are removed), ground (the points belonging to the ground are removed) and structure (only structures and objects are kept in the point cloud). The experiments were performed using the KITTI and SemanticKITTI datasets, which feature different scenarios that allowed identifying the implications and relevance of each element of the environment in LiDAR odometry algorithms. The conclusions obtained from this work are of special relevance for improving the efficiency of LiDAR odometry algorithms in all kinds of scenarios.

scholarly journals Fast 3D Semantic Mapping in Road Scenes

2019 ◽  
Vol 9 (4) ◽  
pp. 631 ◽  
Author(s):  
Xuanpeng Li ◽  
Dong Wang ◽  
Huanxuan Ao ◽  
Rachid Belaroussi ◽  
Dominique Gruyer
Keyword(s):  
3D Reconstruction ◽  
Autonomous Navigation ◽  
Point Cloud ◽  
Semantic Information ◽  
Computational Cost ◽  
Semantic Segmentation ◽  
Semantic Mapping ◽  
3D Point Cloud ◽  
Semantic Inference ◽  
Mapping System

Fast 3D reconstruction with semantic information in road scenes is of great requirements for autonomous navigation. It involves issues of geometry and appearance in the field of computer vision. In this work, we propose a fast 3D semantic mapping system based on the monocular vision by fusion of localization, mapping, and scene parsing. From visual sequences, it can estimate the camera pose, calculate the depth, predict the semantic segmentation, and finally realize the 3D semantic mapping. Our system consists of three modules: a parallel visual Simultaneous Localization And Mapping (SLAM) and semantic segmentation module, an incrementally semantic transfer from 2D image to 3D point cloud, and a global optimization based on Conditional Random Field (CRF). It is a heuristic approach that improves the accuracy of the 3D semantic labeling in light of the spatial consistency on each step of 3D reconstruction. In our framework, there is no need to make semantic inference on each frame of sequence, since the 3D point cloud data with semantic information is corresponding to sparse reference frames. It saves on the computational cost and allows our mapping system to perform online. We evaluate the system on road scenes, e.g., KITTI, and observe a significant speed-up in the inference stage by labeling on the 3D point cloud.

scholarly journals Point Cloud Semantic Segmentation Network Based on Multi-Scale Feature Fusion

Sensors ◽  
2021 ◽  
Vol 21 (5) ◽  
pp. 1625
Author(s):  
Jing Du ◽  
Zuning Jiang ◽  
Shangfeng Huang ◽  
Zongyue Wang ◽  
Jinhe Su ◽  
...  
Keyword(s):  
Network Optimization ◽  
Point Cloud ◽  
Semantic Information ◽  
Feature Fusion ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Scale Feature ◽  
Multi Scale ◽  
Sensing Applications ◽  
Remote Sensing Applications

The semantic segmentation of small objects in point clouds is currently one of the most demanding tasks in photogrammetry and remote sensing applications. Multi-resolution feature extraction and fusion can significantly enhance the ability of object classification and segmentation, so it is widely used in the image field. For this motivation, we propose a point cloud semantic segmentation network based on multi-scale feature fusion (MSSCN) to aggregate the feature of a point cloud with different densities and improve the performance of semantic segmentation. In our method, random downsampling is first applied to obtain point clouds of different densities. A Spatial Aggregation Net (SAN) is then employed as the backbone network to extract local features from these point clouds, followed by concatenation of the extracted feature descriptors at different scales. Finally, a loss function is used to combine the different semantic information from point clouds of different densities for network optimization. Experiments were conducted on the S3DIS and ScanNet datasets, and our MSSCN achieved accuracies of 89.80% and 86.3%, respectively, on these datasets. Our method showed better performance than the recent methods PointNet, PointNet++, PointCNN, PointSIFT, and SAN.

scholarly journals Automatic Point Cloud Registration for Large Outdoor Scenes Using a Priori Semantic Information

2021 ◽  
Vol 13 (17) ◽  
pp. 3474
Author(s):  
Jian Li ◽  
Shuowen Huang ◽  
Hao Cui ◽  
Yurong Ma ◽  
Xiaolong Chen
Keyword(s):  
Point Cloud ◽  
Semantic Information ◽  
A Priori ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Global Alignment ◽  
Feature Descriptor ◽  
Point Cloud Registration ◽  
Key Points ◽  
Outdoor Scenes

As an important and fundamental step in 3D reconstruction, point cloud registration aims to find rigid transformation that register two point sets. The major challenge in point cloud registration techniques is finding correct correspondences in the scenes which may contain many repetitive structures and noise. This paper is primarily concerned with improving registration using a priori semantic information in the search for correspondences. In particular, we present a new point cloud registration pipeline for large outdoor scenes that takes advantage of semantic segmentation. Our method consists of extracting semantic segments from point clouds uses an efficient deep neural network; then, detecting the key points of the point cloud and using a feature descriptor to get the initial correspondence set; finally, applying a Random Sample Consensus (RANSAC) strategy to estimate the transformations that align segments with the same labels. Instead of using all points to estimate a global alignment, our method aligns two point clouds using transformations calculated by each segment with the highest inlier ratio. We evaluate our method on the publicly available Whu-TLS registration dataset. These experiments demonstrate how a priori semantic information the improves registration in terms of precision and speed.

scholarly journals Reconstruction of High-Precision Semantic Map

Sensors ◽  
2020 ◽  
Vol 20 (21) ◽  
pp. 6264
Author(s):  
Xinyuan Tu ◽  
Jian Zhang ◽  
Runhao Luo ◽  
Kai Wang ◽  
Qingji Zeng ◽  
...  
Keyword(s):  
Real Time ◽  
Point Cloud ◽  
Semantic Information ◽  
Feature Fusion ◽  
Three Dimensional ◽  
Semantic Segmentation ◽  
Line Of Sight ◽  
Lidar Data ◽  
Implicit Surface ◽  
Robotic Mapping

We present a real-time Truncated Signed Distance Field (TSDF)-based three-dimensional (3D) semantic reconstruction for LiDAR point cloud, which achieves incremental surface reconstruction and highly accurate semantic segmentation. The high-precise 3D semantic reconstruction in real time on LiDAR data is important but challenging. Lighting Detection and Ranging (LiDAR) data with high accuracy is massive for 3D reconstruction. We so propose a line-of-sight algorithm to update implicit surface incrementally. Meanwhile, in order to use more semantic information effectively, an online attention-based spatial and temporal feature fusion method is proposed, which is well integrated into the reconstruction system. We implement parallel computation in the reconstruction and semantic fusion process, which achieves real-time performance. We demonstrate our approach on the CARLA dataset, Apollo dataset, and our dataset. When compared with the state-of-art mapping methods, our method has a great advantage in terms of both quality and speed, which meets the needs of robotic mapping and navigation.

scholarly journals AUTOMATED SEMANTIC MODELLING OF BUILDING INTERIORS FROM IMAGES AND DERIVED POINT CLOUDS BASED ON DEEP LEARNING METHODS

2020 ◽  
Vol XLIII-B2-2020 ◽  
pp. 421-426
Author(s):  
E. Gülch ◽  
L. Obrock
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Semantic Information ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
High Potential ◽  
Model Reconstruction ◽  
Semantic Modelling ◽  
Building Components ◽  
Segmentation Of Images

Abstract. In this paper, we present an improved approach of enriching photogrammetric point clouds with semantic information extracted from images to enable a later automation of BIM modelling. Based on the DeepLabv3+ architecture, we use Semantic Segmentation of images to extract building components and objects of interiors. During the photogrammetric reconstruction, we project the segmented categories into the point cloud. Any interpolations that occur during this process are corrected automatically and we achieve a mIoU of 51.9 % in the classified point cloud. Based on the semantic information, we align the point cloud, correct the scale and extract further information. Our investigation confirms that utilizing photogrammetry and Deep Learning to generate a semantically enriched point cloud of interiors achieves good results. The combined extraction of geometric and semantic information yields a high potential for automated BIM model reconstruction.

scholarly journals SEMANTIC SEGMENTATION OF INDOOR POINT CLOUDS USING CONVOLUTIONAL NEURAL NETWORK

2017 ◽  
Vol IV-4/W4 ◽  
pp. 101-108 ◽  
Author(s):  
K. Babacan ◽  
L. Chen ◽  
G. Sohn
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Network Architecture ◽  
Point Cloud ◽  
Semantic Information ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Semantic Content ◽  
Data Representation ◽  
Cloud Data

As Building Information Modelling (BIM) thrives, geometry becomes no longer sufficient; an ever increasing variety of semantic information is needed to express an indoor model adequately. On the other hand, for the existing buildings, automatically generating semantically enriched BIM from point cloud data is in its infancy. The previous research to enhance the semantic content rely on frameworks in which some specific rules and/or features that are hand coded by specialists. These methods immanently lack generalization and easily break in different circumstances. On this account, a generalized framework is urgently needed to automatically and accurately generate semantic information. Therefore we propose to employ deep learning techniques for the semantic segmentation of point clouds into meaningful parts. More specifically, we build a volumetric data representation in order to efficiently generate the high number of training samples needed to initiate a convolutional neural network architecture. The feedforward propagation is used in such a way to perform the classification in voxel level for achieving semantic segmentation. The method is tested both for a mobile laser scanner point cloud, and a larger scale synthetically generated data. We also demonstrate a case study, in which our method can be effectively used to leverage the extraction of planar surfaces in challenging cluttered indoor environments.

scholarly journals PCT: Point cloud transformer

2021 ◽  
Vol 7 (2) ◽  
pp. 187-199
Author(s):  
Meng-Hao Guo ◽  
Jun-Xiong Cai ◽  
Zheng-Ning Liu ◽  
Tai-Jiang Mu ◽  
Ralph R. Martin ◽  
...  
Keyword(s):  
Language Processing ◽  
Point Cloud ◽  
Nearest Neighbor ◽  
Semantic Segmentation ◽  
Nearest Neighbor Search ◽  
Local Context ◽  
Irregular Domain ◽  
Cloud Processing ◽  
Neighbor Search ◽  
Farthest Point

AbstractThe irregular domain and lack of ordering make it challenging to design deep neural networks for point cloud processing. This paper presents a novel framework named Point Cloud Transformer (PCT) for point cloud learning. PCT is based on Transformer, which achieves huge success in natural language processing and displays great potential in image processing. It is inherently permutation invariant for processing a sequence of points, making it well-suited for point cloud learning. To better capture local context within the point cloud, we enhance input embedding with the support of farthest point sampling and nearest neighbor search. Extensive experiments demonstrate that the PCT achieves the state-of-the-art performance on shape classification, part segmentation, semantic segmentation, and normal estimation tasks.

scholarly journals Multi-Dimensional Underwater Point Cloud Detection Based on Deep Learning

Sensors ◽  
2021 ◽  
Vol 21 (3) ◽  
pp. 884
Author(s):  
Chia-Ming Tsai ◽  
Yi-Horng Lai ◽  
Yung-Da Sun ◽  
Yu-Jen Chung ◽  
Jau-Woei Perng
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Three Dimensional ◽  
Point Clouds ◽  
Training Data ◽  
Network Architectures ◽  
Point Cloud Data ◽  
Data Types ◽  
Raw Data ◽  
Cloud Data

Numerous sensors can obtain images or point cloud data on land, however, the rapid attenuation of electromagnetic signals and the lack of light in water have been observed to restrict sensing functions. This study expands the utilization of two- and three-dimensional detection technologies in underwater applications to detect abandoned tires. A three-dimensional acoustic sensor, the BV5000, is used in this study to collect underwater point cloud data. Some pre-processing steps are proposed to remove noise and the seabed from raw data. Point clouds are then processed to obtain two data types: a 2D image and a 3D point cloud. Deep learning methods with different dimensions are used to train the models. In the two-dimensional method, the point cloud is transferred into a bird’s eye view image. The Faster R-CNN and YOLOv3 network architectures are used to detect tires. Meanwhile, in the three-dimensional method, the point cloud associated with a tire is cut out from the raw data and is used as training data. The PointNet and PointConv network architectures are then used for tire classification. The results show that both approaches provide good accuracy.

Semantic Segmentation in Depth Data: A Comparative Evaluation of Image and Point Cloud Based Methods

2021 ◽  
Author(s):  
Jigyasa Singh Katrolia ◽  
Lars Kramer ◽  
Jason Rambach ◽  
Bruno Mirbach ◽  
Didier Stricker
Keyword(s):  
Comparative Evaluation ◽  
Point Cloud ◽  
Semantic Segmentation ◽  
Depth Data
Sign in / Sign up

Export Citation Format

Share Document