scholarly journals LatticeNet: fast spatio-temporal point cloud segmentation using permutohedral lattices

2021 ◽  
Author(s):  
Radu Alexandru Rosu ◽  
Peer Schütt ◽  
Jan Quenzel ◽  
Sven Behnke
Keyword(s):  
Point Cloud ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Local Geometry ◽  
Deep Convolutional Neural Networks ◽  
3D Data ◽  
Novel Approach ◽  
Point Cloud Segmentation ◽  
Spatio Temporal ◽  
Fast Convolutions

AbstractDeep convolutional neural networks have shown outstanding performance in the task of semantically segmenting images. Applying the same methods on 3D data still poses challenges due to the heavy memory requirements and the lack of structured data. Here, we propose LatticeNet, a novel approach for 3D semantic segmentation, which takes raw point clouds as input. A PointNet describes the local geometry which we embed into a sparse permutohedral lattice. The lattice allows for fast convolutions while keeping a low memory footprint. Further, we introduce DeformSlice, a novel learned data-dependent interpolation for projecting lattice features back onto the point cloud. We present results of 3D segmentation on multiple datasets where our method achieves state-of-the-art performance. We also extend and evaluate our network for instance and dynamic object segmentation.

scholarly journals FUSION OF FEATURE BASED AND DEEP LEARNING METHODS FOR CLASSIFICATION OF MMS POINT CLOUDS

2019 ◽  
Vol XLII-2/W16 ◽  
pp. 235-242 ◽  
Author(s):  
D. Tosic ◽  
S. Tuttas ◽  
L. Hoegner ◽  
U. Stilla
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Local Geometry ◽  
Learning Approach ◽  
Semantic Classification ◽  
Feature Based ◽  
Urban Scene

<p><strong>Abstract.</strong> This work proposes an approach for semantic classification of an outdoor-scene point cloud acquired with a high precision Mobile Mapping System (MMS), with major goal to contribute to the automatic creation of High Definition (HD) Maps. The automatic point labeling is achieved by utilizing the combination of a feature-based approach for semantic classification of point clouds and a deep learning approach for semantic segmentation of images. Both, point cloud data, as well as the data from a multi-camera system are used for gaining spatial information in an urban scene. Two types of classification applied for this task are: 1) Feature-based approach, in which the point cloud is organized into a supervoxel structure for capturing geometric characteristics of points. Several geometric features are then extracted for appropriate representation of the local geometry, followed by removing the effect of local tendency for each supervoxel to enhance the distinction between similar structures. And lastly, the Random Forests (RF) algorithm is applied in the classification phase, for assigning labels to supervoxels and therefore to points within them. 2) The deep learning approach is employed for semantic segmentation of MMS images of the same scene. To achieve this, an implementation of Pyramid Scene Parsing Network is used. Resulting segmented images with each pixel containing a class label are then projected onto the point cloud, enabling label assignment for each point. At the end, experiment results are presented from a complex urban scene and the performance of this method is evaluated on a manually labeled dataset, for the deep learning and feature-based classification individually, as well as for the result of the labels fusion. The achieved overall accuracy with fusioned output is 0.87 on the final test set, which significantly outperforms the results of individual methods on the same point cloud. The labeled data is published on the TUM-PF Semantic-Labeling-Benchmark.</p>

scholarly journals Panicle-3D: Efficient Phenotyping Tool for Precise Semantic Segmentation of Rice Panicle Point Cloud

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Liang Gong ◽  
Xiaofeng Du ◽  
Kai Zhu ◽  
Ke Lin ◽  
Qiaojun Lou ◽  
...  
Keyword(s):  
Point Cloud ◽  
Network Models ◽  
Semantic Segmentation ◽  
Spatial Location ◽  
Point Clouds ◽  
Utilization Efficiency ◽  
Data Set ◽  
Cloud Data ◽  
Point Cloud Segmentation ◽  
Segmentation Accuracy

The automated measurement of crop phenotypic parameters is of great significance to the quantitative study of crop growth. The segmentation and classification of crop point cloud help to realize the automation of crop phenotypic parameter measurement. At present, crop spike-shaped point cloud segmentation has problems such as fewer samples, uneven distribution of point clouds, occlusion of stem and spike, disorderly arrangement of point clouds, and lack of targeted network models. The traditional clustering method can realize the segmentation of the plant organ point cloud with relatively independent spatial location, but the accuracy is not acceptable. This paper first builds a desktop-level point cloud scanning apparatus based on a structured-light projection module to facilitate the point cloud acquisition process. Then, the rice ear point cloud was collected, and the rice ear point cloud data set was made. In addition, data argumentation is used to improve sample utilization efficiency and training accuracy. Finally, a 3D point cloud convolutional neural network model called Panicle-3D was designed to achieve better segmentation accuracy. Specifically, the design of Panicle-3D is aimed at the multiscale characteristics of plant organs, combined with the structure of PointConv and long and short jumps, which accelerates the convergence speed of the network and reduces the loss of features in the process of point cloud downsampling. After comparison experiments, the segmentation accuracy of Panicle-3D reaches 93.4%, which is higher than PointNet. Panicle-3D is suitable for other similar crop point cloud segmentation tasks.

scholarly journals Neighborhood co-occurrence modeling in 3D point cloud segmentation

2021 ◽  
Vol 8 (2) ◽  
pp. 303-315
Author(s):  
Jingyu Gong ◽  
Zhou Ye ◽  
Lizhuang Ma
Keyword(s):  
Point Cloud ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Local Region ◽  
Kl Divergence ◽  
Point Cloud Segmentation ◽  
Decoder Architecture ◽  
Significant Performance ◽  
The Difference ◽  
Occurrence Matrix

AbstractA significant performance boost has been achieved in point cloud semantic segmentation by utilization of the encoder-decoder architecture and novel convolution operations for point clouds. However, co-occurrence relationships within a local region which can directly influence segmentation results are usually ignored by current works. In this paper, we propose a neighborhood co-occurrence matrix (NCM) to model local co-occurrence relationships in a point cloud. We generate target NCM and prediction NCM from semantic labels and a prediction map respectively. Then, Kullback-Leibler (KL) divergence is used to maximize the similarity between the target and prediction NCMs to learn the co-occurrence relationship. Moreover, for large scenes where the NCMs for a sampled point cloud and the whole scene differ greatly, we introduce a reverse form of KL divergence which can better handle the difference to supervise the prediction NCMs. We integrate our method into an existing backbone and conduct comprehensive experiments on three datasets: Semantic3D for outdoor space segmentation, and S3DIS and ScanNet v2 for indoor scene segmentation. Results indicate that our method can significantly improve upon the backbone and outperform many leading competitors.

scholarly journals Multi-Scale Attentive Aggregation for LiDAR Point Cloud Segmentation

2021 ◽  
Vol 13 (4) ◽  
pp. 691
Author(s):  
Xiaoxiao Geng ◽  
Shunping Ji ◽  
Meng Lu ◽  
Lingli Zhao
Keyword(s):  
Point Cloud ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Feature Representation ◽  
Channel Structure ◽  
Structure Information ◽  
Multi Scale ◽  
Point Cloud Segmentation ◽  
Global Consistency ◽  
Decoder Architecture

Semantic segmentation of LiDAR point clouds has implications in self-driving, robots, and augmented reality, among others. In this paper, we propose a Multi-Scale Attentive Aggregation Network (MSAAN) to achieve the global consistency of point cloud feature representation and super segmentation performance. First, upon a baseline encoder-decoder architecture for point cloud segmentation, namely, RandLA-Net, an attentive skip connection was proposed to replace the commonly used concatenation to balance the encoder and decoder features of the same scales. Second, a channel attentive enhancement module was introduced to the local attention enhancement module to boost the local feature discriminability and aggregate the local channel structure information. Third, we developed a multi-scale feature aggregation method to capture the global structure of a point cloud from both the encoder and the decoder. The experimental results reported that our MSAAN significantly outperformed state-of-the-art methods, i.e., at least 15.3% mIoU improvement for scene-2 of CSPC dataset, 5.2% for scene-5 of CSPC dataset, and 6.6% for Toronto3D dataset.

scholarly journals Semantic Point Cloud Segmentation Using Fast Deep Neural Network and DCRF

Sensors ◽  
2021 ◽  
Vol 21 (8) ◽  
pp. 2731
Author(s):  
Yunbo Rao ◽  
Menghan Zhang ◽  
Zhanglin Cheng ◽  
Junmin Xue ◽  
Jiansu Pu ◽  
...  
Keyword(s):  
Neural Network ◽  
Point Cloud ◽  
Deep Neural Network ◽  
Conditional Random Field ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Original Data ◽  
Point Cloud Segmentation ◽  
Semantic Point ◽  
3D Scene

Accurate segmentation of entity categories is the critical step for 3D scene understanding. This paper presents a fast deep neural network model with Dense Conditional Random Field (DCRF) as a post-processing method, which can perform accurate semantic segmentation for 3D point cloud scene. On this basis, a compact but flexible framework is introduced for performing segmentation to the semantics of point clouds concurrently, contribute to more precise segmentation. Moreover, based on semantics labels, a novel DCRF model is elaborated to refine the result of segmentation. Besides, without any sacrifice to accuracy, we apply optimization to the original data of the point cloud, allowing the network to handle fewer data. In the experiment, our proposed method is conducted comprehensively through four evaluation indicators, proving the superiority of our method.

scholarly journals HYBRID GEOREFERENCING, ENHANCEMENT AND CLASSIFICATION OF ULTRA-HIGH RESOLUTION UAV LIDAR AND IMAGE POINT CLOUDS FOR MONITORING APPLICATIONS

2020 ◽  
Vol V-2-2020 ◽  
pp. 727-734
Author(s):  
N. Haala ◽  
M. Kölle ◽  
M. Cramer ◽  
D. Laupheimer ◽  
G. Mandlburger ◽  
...  
Keyword(s):  
Point Cloud ◽  
Image Data ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Airborne Lidar ◽  
Image Texture ◽  
Image Point ◽  
Joint Orientation ◽  
3D Point Clouds ◽  
3D Data

Abstract. This paper presents a study on the potential of ultra-high accurate UAV-based 3D data capture by combining both imagery and LiDAR data. Our work is motivated by a project aiming at the monitoring of subsidence in an area of mixed use. Thus, it covers built-up regions in a village with a ship lock as the main object of interest as well as regions of agricultural use. In order to monitor potential subsidence in the order of 10 mm/year, we aim at sub-centimeter accuracies of the respective 3D point clouds. We show that hybrid georeferencing helps to increase the accuracy of the adjusted LiDAR point cloud by integrating results from photogrammetric block adjustment to improve the time-dependent trajectory corrections. As our main contribution, we demonstrate that joint orientation of laser scans and images in a hybrid adjustment framework significantly improves the relative and absolute height accuracies. By these means, accuracies corresponding to the GSD of the integrated imagery can be achieved. Image data can also help to enhance the LiDAR point clouds. As an example, integrating results from Multi-View Stereo potentially increases the point density from airborne LiDAR. Furthermore, image texture can support 3D point cloud classification. This semantic segmentation discussed in the final part of the paper is a prerequisite for further enhancement and analysis of the captured point cloud.

scholarly journals INDOOR 3D POINT CLOUDS SEMANTIC SEGMENTATION BASES ON MODIFIED POINTNET NETWORK

2020 ◽  
Vol XLIII-B2-2020 ◽  
pp. 369-373
Author(s):  
J. Zhao ◽  
X. Zhang ◽  
Y. Wang
Keyword(s):  
Point Cloud ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Local Features ◽  
Indoor Navigation ◽  
Convolution Kernel ◽  
Structural Elements ◽  
Global Features ◽  
3D Point Clouds ◽  
Point Cloud Segmentation

Abstract. Indoor 3D point clouds semantics segmentation is one of the key technologies of constructing 3D indoor models,which play an important role on domains like indoor navigation and positioning,intelligent city, intelligent robot etc. The deep-learning-based methods for point cloud segmentation take on higher degree of automation and intelligence. PointNet,the first deep neural network which manipulate point cloud directly, mainly extracts the global features but lacks of learning and extracting local features,which causes the poor ability of segmenting the local details of architecture and affects the precision of structural elements segmentation . Focusing on the problems above,this paper put forward an automatic end-to-end segmentation method base on the modified PointNet. According to the characteristic that the intensity of different indoor structural elements differ a lot, we input the point cloud information of 3D coordinate, color and intensity into the feature space of points. Also,a MaxPooling is added into the original PointNet network to improve the ability of attracting and learning local features. In addition, replace the 1×1 convolution kernel of original PointNet with 3×3 convolution kernel in the process of attracting features to improve the segmentation precision of indoor point cloud. The result shows that this method improves the automation and precision of indoor point cloud segmentation for the precision achieves over 80% to segment the structural elements like wall,door and so on ,and the average segmentation precision of every structural elements achieves 66%.

Supervoxel-Based Region Growing Segmentation for Point Cloud Data

2020 ◽  
pp. 2154007
Author(s):  
Nan Luo ◽  
Yuanyuan Jiang ◽  
Quan Wang
Keyword(s):  
Point Cloud ◽  
High Efficiency ◽  
Region Growing ◽  
Experimental Tests ◽  
Point Clouds ◽  
Spatial Distance ◽  
Local Geometry ◽  
Cloud Data ◽  
Point Cloud Segmentation ◽  
Region Growing Segmentation

Point cloud segmentation is a crucial fundamental step in 3D reconstruction, object recognition and scene understanding. This paper proposes a supervoxel-based point cloud segmentation algorithm in region growing principle to solve the issues of inaccurate boundaries and nonsmooth segments in the existing methods. To begin with, the input point cloud is voxelized and then pre-segmented into sparse supervoxels by flow constrained clustering, considering the spatial distance and local geometry between voxels. Afterwards, plane fitting is applied to the over-segmented supervoxels and seeds for region growing are selected with respect to the fitting residuals. Starting from pruned seed patches, adjacent supervoxels are merged in region growing style to form the final segments, according to the normalized similarity measure that integrates the smoothness and shape constraints of supervoxels. We determine the values of parameters via experimental tests, and the final results show that, by voxelizing and pre-segmenting the point clouds, the proposed algorithm is robust to noises and can obtain smooth segmentation regions with accurate boundaries in high efficiency.

scholarly journals Uncertainty Estimation in Deep Neural Networks for Point Cloud Segmentation in Factory Planning

Modelling ◽  
2021 ◽  
Vol 2 (1) ◽  
pp. 1-17
Author(s):  
Christina Petschnigg ◽  
Jürgen Pilz
Keyword(s):  
Point Cloud ◽  
Point Clouds ◽  
Scientific Data ◽  
Production Plant ◽  
Data Set ◽  
Factory Planning ◽  
3D Data ◽  
Future Production ◽  
Point Cloud Segmentation ◽  
Approximate Bayesian

The digital factory provides undoubtedly great potential for future production systems in terms of efficiency and effectivity. A key aspect on the way to realize the digital copy of a real factory is the understanding of complex indoor environments on the basis of three-dimensional (3D) data. In order to generate an accurate factory model including the major components, i.e., building parts, product assets, and process details, the 3D data that are collected during digitalization can be processed with advanced methods of deep learning. For instance, the semantic segmentation of a point cloud enables the identification of relevant objects within the environment. In this work, we propose a fully Bayesian and an approximate Bayesian neural network for point cloud segmentation. Both of the networks are used within a workflow in order to generate an environment model on the basis of raw point clouds. The Bayesian and approximate Bayesian networks allow us to analyse how different ways of estimating uncertainty in these networks improve segmentation results on raw point clouds. We achieve superior model performance for both, the Bayesian and the approximate Bayesian model compared to the frequentist one. This performance difference becomes even more striking when incorporating the networks’ uncertainty in their predictions. For evaluation, we use the scientific data set S3DIS as well as a data set, which was collected by the authors at a German automotive production plant. The methods proposed in this work lead to more accurate segmentation results and the incorporation of uncertainty information also makes this approach especially applicable to safety critical applications aside from our factory planning use case.

scholarly journals ON THE ASSOCIATION OF LIDAR POINT CLOUDS AND TEXTURED MESHES FOR MULTI-MODAL SEMANTIC SEGMENTATION

2020 ◽  
Vol V-2-2020 ◽  
pp. 509-516
Author(s):  
D. Laupheimer ◽  
M. H. Shams Eddin ◽  
N. Haala
Keyword(s):  
Information Transfer ◽  
Point Cloud ◽  
Feature Vector ◽  
Semantic Annotation ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Mesh Segmentation ◽  
Triangle Mesh ◽  
3D Data ◽  
Novel Association

Abstract. The semantic segmentation of the huge amount of acquired 3D data has become an important task in recent years. We propose a novel association mechanism that enables information transfer between two 3D representations: point clouds and meshes. The association mechanism can be used in a two-fold manner: (i) feature transfer to stabilize semantic segmentation of one representation with features from the other representation and (ii) label transfer to achieve the semantic annotation of both representations. We claim that point clouds are an intermediate product whereas meshes are a final user product that jointly provides geometrical and textural information. For this reason, we opt for semantic mesh segmentation in the first place. We apply an off-the-shelf PointNet++ to a textured urban triangle mesh as generated from LiDAR and oblique imagery. For each face within a mesh, a feature vector is computed and optionally extended by inherent LiDAR features as provided by the sensor (e.g. intensity). The feature vector extension is accomplished with the proposed association mechanism. By these means, we leverage inherent features from both data representations for the semantic mesh segmentation (multi-modality). We achieve an overall accuracy of 86:40% on the face-level on a dedicated test mesh. Neglecting LiDAR-inherent features in the per-face feature vectors decreases mean intersection over union by ∼2%. Leveraging our association mechanism, we transfer predicted mesh labels to the LiDAR point cloud at a stroke. To this end, we semantically segment the point cloud by implicit usage of geometric and textural mesh features. The semantic point cloud segmentation achieves an overall accuracy close to 84% on the point-level for both feature vector compositions.

Sign in / Sign up

Export Citation Format

Share Document