scholarly journals A SEMANTIC 3D POINT CLOUD SEGMENTATION APPROACH BASED ON OPTIMAL VIEW SELECTION FOR 2D IMAGE FEATURE EXTRACTION

2019 ◽  
Vol XLII-2/W17 ◽  
pp. 9-14
Author(s):  
A. Adam ◽  
L. Grammatikopoulos ◽  
G. Karras ◽  
E. Protopapadakis ◽  
K. Karantzalos
Keyword(s):  
Feature Extraction ◽  
Point Cloud ◽  
Structural Information ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Image Feature ◽  
Hybrid Features ◽  
Semantic Class ◽  
Segmentation Approach ◽  
Traditional Approaches

Abstract. 3D semantic segmentation is the joint task of partitioning a point cloud into semantically consistent 3D regions and assigning them to a semantic class/label. While the traditional approaches for 3D semantic segmentation typically rely only on structural information of the objects (i.e. object geometry and shape), the last years many techniques combining both visual and geometric features have emerged, taking advantage of the progress in SfM/MVS algorithms that reconstruct point clouds from multiple overlapping images. Our work describes a hybrid methodology for 3D semantic segmentation, relying both on 2D and 3D space and aiming at exploring whether image selection is critical as regards the accuracy of 3D semantic segmentation of point clouds. Experimental results are demonstrated on a free online dataset depicting city blocks around Paris. The experimental procedure not only validates that hybrid features (geometric and visual) can achieve a more accurate semantic segmentation, but also demonstrates the importance of the most appropriate view for the 2D feature extraction.

scholarly journals SEMANTIC SEGMENTATION OF BUILDING ELEMENTS USING POINT CLOUD HASHING

2018 ◽  
Vol XLII-2 ◽  
pp. 241-250 ◽  
Author(s):  
M. Chizhova ◽  
A. Gurianov ◽  
M. Hess ◽  
T. Luhmann ◽  
A. Brunn ◽  
...  
Keyword(s):  
Point Cloud ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Probabilistic Networks ◽  
Component Design ◽  
Semantic Modelling ◽  
Segmentation Approach ◽  
Novel Method ◽  
Human Interpretation ◽  
Definition Of

For the interpretation of point clouds, the semantic definition of extracted segments from point clouds or images is a common problem. Usually, the semantic of geometrical pre-segmented point cloud elements are determined using probabilistic networks and scene databases. The proposed semantic segmentation method is based on the psychological human interpretation of geometric objects, especially on fundamental rules of primary comprehension. Starting from these rules the buildings could be quite well and simply classified by a human operator (e.g. architect) into different building types and structural elements (dome, nave, transept etc.), including particular building parts which are visually detected. The key part of the procedure is a novel method based on hashing where point cloud projections are transformed into binary pixel representations. A segmentation approach released on the example of classical Orthodox churches is suitable for other buildings and objects characterized through a particular typology in its construction (e.g. industrial objects in standardized enviroments with strict component design allowing clear semantic modelling).

scholarly journals CONTEXTUAL CLASSIFICATION OF POINT CLOUD DATA BY EXPLOITING INDIVIDUAL 3D NEIGBOURHOODS

2015 ◽  
Vol II-3/W4 ◽  
pp. 271-278 ◽  
Author(s):  
M. Weinmann ◽  
A. Schmidt ◽  
C. Mallet ◽  
S. Hinz ◽  
F. Rottensteiner ◽  
...  
Keyword(s):  
Feature Extraction ◽  
Point Cloud ◽  
Contextual Information ◽  
Point Clouds ◽  
Optimal Size ◽  
Point Cloud Data ◽  
Cloud Data ◽  
Semantic Class ◽  
3D Point Clouds

The fully automated analysis of 3D point clouds is of great importance in photogrammetry, remote sensing and computer vision. For reliably extracting objects such as buildings, road inventory or vegetation, many approaches rely on the results of a point cloud classification, where each 3D point is assigned a respective semantic class label. Such an assignment, in turn, typically involves statistical methods for feature extraction and machine learning. Whereas the different components in the processing workflow have extensively, but separately been investigated in recent years, the respective connection by sharing the results of crucial tasks across all components has not yet been addressed. This connection not only encapsulates the interrelated issues of neighborhood selection and feature extraction, but also the issue of how to involve spatial context in the classification step. In this paper, we present a novel and generic approach for 3D scene analysis which relies on (<i>i</i>) individually optimized 3D neighborhoods for (<i>ii</i>) the extraction of distinctive geometric features and (<i>iii</i>) the contextual classification of point cloud data. For a labeled benchmark dataset, we demonstrate the beneficial impact of involving contextual information in the classification process and that using individual 3D neighborhoods of optimal size significantly increases the quality of the results for both pointwise and contextual classification.

scholarly journals H-RANSAC: A HYBRID POINT CLOUD SEGMENTATION COMBINING 2D AND 3D DATA

2018 ◽  
Vol IV-2 ◽  
pp. 1-8 ◽  
Author(s):  
A. Adam ◽  
E. Chatzilari ◽  
S. Nikolopoulos ◽  
I. Kompatsiaris
Keyword(s):  
Visual Information ◽  
Point Cloud ◽  
Structural Information ◽  
Hybrid Approach ◽  
Point Clouds ◽  
3D Segmentation ◽  
Plane Fitting ◽  
Fitting Algorithm ◽  
Segmentation Approach ◽  
2D Data

In this paper, we present a novel 3D segmentation approach operating on point clouds generated from overlapping images. The aim of the proposed hybrid approach is to effectively segment co-planar objects, by leveraging the structural information originating from the 3D point cloud and the visual information from the 2D images, without resorting to learning based procedures. More specifically, the proposed hybrid approach, H-RANSAC, is an extension of the well-known RANSAC plane-fitting algorithm, incorporating an additional consistency criterion based on the results of 2D segmentation. Our expectation that the integration of 2D data into 3D segmentation will achieve more accurate results, is validated experimentally in the domain of 3D city models. Results show that HRANSAC can successfully delineate building components like main facades and windows, and provide more accurate segmentation results compared to the typical RANSAC plane-fitting algorithm.

scholarly journals EXPLORING ALS AND DIM DATA FOR SEMANTIC SEGMENTATION USING CNNS

2018 ◽  
Vol XLII-1 ◽  
pp. 347-354 ◽  
Author(s):  
F. Politz ◽  
M. Sester
Keyword(s):  
Point Cloud ◽  
Laser Scanning ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Good Alternative ◽  
Aerial Images ◽  
Learning Approaches ◽  
Advantages And Disadvantages ◽  
Sensing Applications ◽  
High Level

<p><strong>Abstract.</strong> Over the past years, the algorithms for dense image matching (DIM) to obtain point clouds from aerial images improved significantly. Consequently, DIM point clouds are now a good alternative to the established Airborne Laser Scanning (ALS) point clouds for remote sensing applications. In order to derive high-level applications such as digital terrain models or city models, each point within a point cloud must be assigned a class label. Usually, ALS and DIM are labelled with different classifiers due to their varying characteristics. In this work, we explore both point cloud types in a fully convolutional encoder-decoder network, which learns to classify ALS as well as DIM point clouds. As input, we project the point clouds onto a 2D image raster plane and calculate the minimal, average and maximal height values for each raster cell. The network then differentiates between the classes ground, non-ground, building and no data. We test our network in six training setups using only one point cloud type, both point clouds as well as several transfer-learning approaches. We quantitatively and qualitatively compare all results and discuss the advantages and disadvantages of all setups. The best network achieves an overall accuracy of 96<span class="thinspace"></span>% in an ALS and 83<span class="thinspace"></span>% in a DIM test set.</p>

scholarly journals Planar surface segmentation using a color-enhanced hybrid model for RGB-D camera-based indoor mobile mapping point clouds

2014 ◽  
Vol II-1 ◽  
pp. 61-67
Author(s):  
C. Wen ◽  
S. Lin ◽  
C. Wang ◽  
J. Li
Keyword(s):  
Point Cloud ◽  
Evaluation Process ◽  
Point Clouds ◽  
Planar Surface ◽  
Mobile Mapping ◽  
Hybrid Features ◽  
Segmentation Evaluation ◽  
Cloud Data ◽  
Surface Segmentation ◽  
Mapping Point

Point clouds acquired by RGB-D camera-based indoor mobile mapping system suffer the problems of being noisy, exhibiting an uneven distribution, and incompleteness, which are the problems that introduce difficulties for point cloud planar surface segmentation. This paper presents a novel color-enhanced hybrid planar surface segmentation model for RGB-D camera-based indoor mobile mapping point clouds based on region growing method, and the model specially addresses the planar surface extraction task over point cloud according to the noisy and incomplete indoor mobile mapping point clouds. The proposed model combines the color moments features with the curvature feature to select the seed points better. Additionally, a more robust growing criteria based on the hybrid features is developed to avoid the generation of excessive over-segmentation debris. A segmentation evaluation process with a small set of labeled segmented data is used to determine the optimal hybrid weight. Several comparative experiments were conducted to evaluate the segmentation model, and the experimental results demonstrate the effectiveness and efficiency of the proposed hybrid segmentation method for indoor mobile mapping three-dimensional (3D) point cloud data.

scholarly journals UNDERSTANDING 3D POINT CLOUD DEEP NEURAL NETWORKS BY VISUALIZATION TECHNIQUES

2020 ◽  
Vol XLIII-B2-2020 ◽  
pp. 651-657
Author(s):  
Y. Cao ◽  
M. Previtali ◽  
M. Scaioni
Keyword(s):  
Point Cloud ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Learning Networks ◽  
Quantitative Investigation ◽  
Different Types ◽  
Visualization Techniques ◽  
Point Cloud Classification ◽  
Learned Features ◽  
Excellent Tool

Abstract. In the wake of the success of Deep Learning Networks (DLN) for image recognition, object detection, shape classification and semantic segmentation, this approach has proven to be both a major breakthrough and an excellent tool in point cloud classification. However, understanding how different types of DLN achieve still lacks. In several studies the output of segmentation/classification process is compared against benchmarks, but the network is treated as a “black-box” and intermediate steps are not deeply analysed. Specifically, here the following questions are discussed: (1) what exactly did DLN learn from a point cloud? (2) On the basis of what information do DLN make decisions? To conduct such a quantitative investigation of these DLN applied to point clouds, this paper investigates the visual interpretability for the decision-making process. Firstly, we introduce a reconstruction network able to reconstruct and visualise the learned features, in order to face with question (1). Then, we propose 3DCAM to indicate the discriminative point cloud regions used by these networks to identify that category, thus dealing with question (2). Through answering the above two questions, the paper would like to offer some initial solutions to better understand the application of DLN to point clouds.

scholarly journals Virtual Disassembling of Historical Edifices: Experiments and Assessments of an Automatic Approach for Classifying Multi-Scalar Point Clouds into Architectural Elements

Sensors ◽  
2020 ◽  
Vol 20 (8) ◽  
pp. 2161 ◽  
Author(s):  
Arnadi Murtiyoso ◽  
Pierre Grussenmeyer
Keyword(s):  
Point Cloud ◽  
Laser Scanning ◽  
Semantic Annotation ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Training Data ◽  
Algorithmic Approach ◽  
Architectural Elements ◽  
Semantic Labeling ◽  
Geometric Point

3D heritage documentation has seen a surge in the past decade due to developments in reality-based 3D recording techniques. Several methods such as photogrammetry and laser scanning are becoming ubiquitous amongst architects, archaeologists, surveyors, and conservators. The main result of these methods is a 3D representation of the object in the form of point clouds. However, a solely geometric point cloud is often insufficient for further analysis, monitoring, and model predicting of the heritage object. The semantic annotation of point clouds remains an interesting research topic since traditionally it requires manual labeling and therefore a lot of time and resources. This paper proposes an automated pipeline to segment and classify multi-scalar point clouds in the case of heritage object. This is done in order to perform multi-level segmentation from the scale of a historical neighborhood up until that of architectural elements, specifically pillars and beams. The proposed workflow involves an algorithmic approach in the form of a toolbox which includes various functions covering the semantic segmentation of large point clouds into smaller, more manageable and semantically labeled clusters. The first part of the workflow will explain the segmentation and semantic labeling of heritage complexes into individual buildings, while a second part will discuss the use of the same toolbox to segment the resulting buildings further into architectural elements. The toolbox was tested on several historical buildings and showed promising results. The ultimate intention of the project is to help the manual point cloud labeling, especially when confronted with the large training data requirements of machine learning-based algorithms.

scholarly journals Voxel-based 3D Point Cloud Semantic Segmentation: Unsupervised Geometric and Relationship Featuring vs Deep Learning Methods

2019 ◽  
Vol 8 (5) ◽  
pp. 213 ◽  
Author(s):  
Florent Poux ◽  
Roland Billen
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Semantic Representation ◽  
Structural Connectivity ◽  
Three Dimensional ◽  
Strong Support ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Learning Methods ◽  
Cloud Data

Automation in point cloud data processing is central in knowledge discovery within decision-making systems. The definition of relevant features is often key for segmentation and classification, with automated workflows presenting the main challenges. In this paper, we propose a voxel-based feature engineering that better characterize point clusters and provide strong support to supervised or unsupervised classification. We provide different feature generalization levels to permit interoperable frameworks. First, we recommend a shape-based feature set (SF1) that only leverages the raw X, Y, Z attributes of any point cloud. Afterwards, we derive relationship and topology between voxel entities to obtain a three-dimensional (3D) structural connectivity feature set (SF2). Finally, we provide a knowledge-based decision tree to permit infrastructure-related classification. We study SF1/SF2 synergy on a new semantic segmentation framework for the constitution of a higher semantic representation of point clouds in relevant clusters. Finally, we benchmark the approach against novel and best-performing deep-learning methods while using the full S3DIS dataset. We highlight good performances, easy-integration, and high F1-score (> 85%) for planar-dominant classes that are comparable to state-of-the-art deep learning.

scholarly journals TIF-Reg: Point Cloud Registration with Transform-Invariant Features in SE(3)

Sensors ◽  
2021 ◽  
Vol 21 (17) ◽  
pp. 5778
Author(s):  
Baifan Chen ◽  
Hong Chen ◽  
Baojun Song ◽  
Grace Gong
Keyword(s):  
Feature Extraction ◽  
Point Cloud ◽  
Point Clouds ◽  
Rigid Transformation ◽  
Point Cloud Registration ◽  
Registration Method ◽  
Invariant Features ◽  
Invariant Feature ◽  
Deep Feature ◽  
Wide Range

Three-dimensional point cloud registration (PCReg) has a wide range of applications in computer vision, 3D reconstruction and medical fields. Although numerous advances have been achieved in the field of point cloud registration in recent years, large-scale rigid transformation is a problem that most algorithms still cannot effectively handle. To solve this problem, we propose a point cloud registration method based on learning and transform-invariant features (TIF-Reg). Our algorithm includes four modules, which are the transform-invariant feature extraction module, deep feature embedding module, corresponding point generation module and decoupled singular value decomposition (SVD) module. In the transform-invariant feature extraction module, we design TIF in SE(3) (which means the 3D rigid transformation space) which contains a triangular feature and local density feature for points. It fully exploits the transformation invariance of point clouds, making the algorithm highly robust to rigid transformation. The deep feature embedding module embeds TIF into a high-dimension space using a deep neural network, further improving the expression ability of features. The corresponding point cloud is generated using an attention mechanism in the corresponding point generation module, and the final transformation for registration is calculated in the decoupled SVD module. In an experiment, we first train and evaluate the TIF-Reg method on the ModelNet40 dataset. The results show that our method keeps the root mean squared error (RMSE) of rotation within 0.5∘ and the RMSE of translation error close to 0 m, even when the rotation is up to [−180∘, 180∘] or the translation is up to [−20 m, 20 m]. We also test the generalization of our method on the TUM3D dataset using the model trained on Modelnet40. The results show that our method’s errors are close to the experimental results on Modelnet40, which verifies the good generalization ability of our method. All experiments prove that the proposed method is superior to state-of-the-art PCReg algorithms in terms of accuracy and complexity.

scholarly journals SEMANTIC SEGMENTATION OF MOBILE LASER SCANNING POINT CLOUDS WITH LONG SHORT-TERM MEMORY NETWORKS: PRELIMINARY RESULTS

2021 ◽  
Vol XLIII-B2-2021 ◽  
pp. 123-130
Author(s):  
J. Balado ◽  
P. van Oosterom ◽  
L. Díaz-Vilariño ◽  
P. Arias
Keyword(s):  
Point Cloud ◽  
Laser Scanning ◽  
Short Term Memory ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Time Signal ◽  
Success Rates ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

Abstract. Although point clouds are characterized as a type of unstructured data, timestamp attribute can structure point clouds into scanlines and shape them into a time signal. The present work studies the transformation of the street point cloud into a time signal based on the Z component for the semantic segmentation using Long Short-Term Memory (LSTM) networks. The experiment was conducted on the point cloud of a real case study. Several training sessions were performed changing the Level of Detail of the classification (coarse level with 3 classes and fine level with 11 classes), two levels of network depth and the use of weighting for the improvement of classes with low number of points. The results showed high accuracy, reaching at best 97.3% in the classification with 3 classes (ground, buildings, and objects) and 95.7% with 11 classes. The distribution of the success rates was not the same for all classes. The classes with the highest number of points obtained better results than the others. The application of weighting improved the classes with few points at the expense of the classes with more points. Increasing the number of hidden layers was shown as a preferable alternative to weighting. Given the high success rates and a behaviour of the LSTM consistent with other Neural Networks in point cloud processing, it is concluded that the LSTM is a feasible alternative for the semantic segmentation of point clouds transformed into time signals.

Sign in / Sign up

Export Citation Format

Share Document