scholarly journals Contrastive Learning for 3D Point Clouds Classification and Shape Completion

Sensors ◽  
2021 ◽  
Vol 21 (21) ◽  
pp. 7392
Author(s):  
Danish Nazir ◽  
Muhammad Zeshan Afzal ◽  
Alain Pagani ◽  
Marcus Liwicki ◽  
Didier Stricker
Keyword(s):  
Point Cloud ◽  
Feature Learning ◽  
Point Clouds ◽  
Classification Performance ◽  
Feature Representations ◽  
3D Point Clouds ◽  
Chamfer Distance ◽  
Shape Completion ◽  
Number Of Classes

In this paper, we present the idea of Self Supervised learning on the shape completion and classification of point clouds. Most 3D shape completion pipelines utilize AutoEncoders to extract features from point clouds used in downstream tasks such as classification, segmentation, detection, and other related applications. Our idea is to add contrastive learning into AutoEncoders to encourage global feature learning of the point cloud classes. It is performed by optimizing triplet loss. Furthermore, local feature representations learning of point cloud is performed by adding the Chamfer distance function. To evaluate the performance of our approach, we utilize the PointNet classifier. We also extend the number of classes for evaluation from 4 to 10 to show the generalization ability of the learned features. Based on our results, embeddings generated from the contrastive AutoEncoder enhances shape completion and classification performance from 84.2% to 84.9% of point clouds achieving the state-of-the-art results with 10 classes.

scholarly journals Contrasitive Learning for 3D Point Clouds Classification and Shape Completion

2021 ◽  
Author(s):  
Danish Nazir ◽  
Muhammad Zeshan Afzal ◽  
Alain Pagani ◽  
Marcus Liwicki ◽  
Didier Stricker
Keyword(s):  
Point Clouds ◽  
Classification Performance ◽  
Local Feature ◽  
Feature Representations ◽  
3D Point Clouds ◽  
Chamfer Distance ◽  
Shape Completion ◽  
Number Of Classes ◽  
Global And Local

In this paper, we present the idea of Self Supervised learning on the Shape Completion and Classification of point clouds. Most 3D shape completion pipelines utilize autoencoders to extract features from point clouds used in downstream tasks such as Classification, Segmentation, Detection, and other related applications. Our idea is to add Contrastive Learning into Auto-Encoders to learn both global and local feature representations of point clouds. We use a combination of Triplet Loss and Chamfer distance to learn global and local feature representations. To evaluate the performance of embeddings for Classification, we utilize the PointNet classifier. We also extend the number of classes to evaluate our model from 4 to 10 to show the generalization ability of learned features. Based on our results, embedding generated from the Contrastive autoencoder enhances Shape Completion and Classification performance from 84.2% to 84.9% of point clouds achieving the state-of-the-art results with 10 classes.

scholarly journals CONTEXTUAL CLASSIFICATION OF POINT CLOUD DATA BY EXPLOITING INDIVIDUAL 3D NEIGBOURHOODS

2015 ◽  
Vol II-3/W4 ◽  
pp. 271-278 ◽  
Author(s):  
M. Weinmann ◽  
A. Schmidt ◽  
C. Mallet ◽  
S. Hinz ◽  
F. Rottensteiner ◽  
...  
Keyword(s):  
Feature Extraction ◽  
Point Cloud ◽  
Contextual Information ◽  
Point Clouds ◽  
Optimal Size ◽  
Point Cloud Data ◽  
Cloud Data ◽  
Semantic Class ◽  
3D Point Clouds

The fully automated analysis of 3D point clouds is of great importance in photogrammetry, remote sensing and computer vision. For reliably extracting objects such as buildings, road inventory or vegetation, many approaches rely on the results of a point cloud classification, where each 3D point is assigned a respective semantic class label. Such an assignment, in turn, typically involves statistical methods for feature extraction and machine learning. Whereas the different components in the processing workflow have extensively, but separately been investigated in recent years, the respective connection by sharing the results of crucial tasks across all components has not yet been addressed. This connection not only encapsulates the interrelated issues of neighborhood selection and feature extraction, but also the issue of how to involve spatial context in the classification step. In this paper, we present a novel and generic approach for 3D scene analysis which relies on (<i>i</i>) individually optimized 3D neighborhoods for (<i>ii</i>) the extraction of distinctive geometric features and (<i>iii</i>) the contextual classification of point cloud data. For a labeled benchmark dataset, we demonstrate the beneficial impact of involving contextual information in the classification process and that using individual 3D neighborhoods of optimal size significantly increases the quality of the results for both pointwise and contextual classification.

scholarly journals Structure-Aware Convolution for 3D Point Cloud Classification and Segmentation

2020 ◽  
Vol 12 (4) ◽  
pp. 634 ◽  
Author(s):  
Lei Wang ◽  
Yuxuan Liu ◽  
Shenman Zhang ◽  
Jixing Yan ◽  
Pengjie Tao
Keyword(s):  
Deep Learning ◽  
Template Matching ◽  
Point Cloud ◽  
Structure Learning ◽  
Feature Learning ◽  
Point Clouds ◽  
Learning Networks ◽  
Geometric Structures ◽  
Learning Capability ◽  
3D Point Clouds

Semantic feature learning on 3D point clouds is quite challenging because of their irregular and unordered data structure. In this paper, we propose a novel structure-aware convolution (SAC) to generalize deep learning on regular grids to irregular 3D point clouds. Similar to the template-matching process of convolution on 2D images, the key of our SAC is to match the point clouds’ neighborhoods with a series of 3D kernels, where each kernel can be regarded as a “geometric template” formed by a set of learnable 3D points. Thus, the interested geometric structures of the input point clouds can be activated by the corresponding kernels. To verify the effectiveness of the proposed SAC, we embedded it into three recently developed point cloud deep learning networks (PointNet, PointNet++, and KCNet) as a lightweight module, and evaluated its performance on both classification and segmentation tasks. Experimental results show that, benefiting from the geometric structure learning capability of our SAC, all these back-end networks achieved better classification and segmentation performance (e.g., +2.77% mean accuracy for classification and +4.99% mean intersection over union (IoU) for segmentation) with few additional parameters. Furthermore, results also demonstrate that the proposed SAC is helpful in improving the robustness of networks with the constraints of geometric structures.

scholarly journals Machine Learning-Based Supervised Classification of Point Clouds Using Multiscale Geometric Features

2021 ◽  
Vol 10 (3) ◽  
pp. 187
Author(s):  
Muhammed Enes Atik ◽  
Zaide Duran ◽  
Dursun Zafer Seker
Keyword(s):  
Machine Learning ◽  
Point Cloud ◽  
Supervised Classification ◽  
Point Clouds ◽  
Support Vector ◽  
Geometric Features ◽  
Mathematical Tool ◽  
City Area ◽  
3D Point Clouds

3D scene classification has become an important research field in photogrammetry, remote sensing, computer vision and robotics with the widespread usage of 3D point clouds. Point cloud classification, called semantic labeling, semantic segmentation, or semantic classification of point clouds is a challenging topic. Machine learning, on the other hand, is a powerful mathematical tool used to classify 3D point clouds whose content can be significantly complex. In this study, the classification performance of different machine learning algorithms in multiple scales was evaluated. The feature spaces of the points in the point cloud were created using the geometric features generated based on the eigenvalues of the covariance matrix. Eight supervised classification algorithms were tested in four different areas from three datasets (the Dublin City dataset, Vaihingen dataset and Oakland3D dataset). The algorithms were evaluated in terms of overall accuracy, precision, recall, F1 score and process time. The best overall results were obtained for four test areas with different algorithms. Dublin City Area 1 was obtained with Random Forest as 93.12%, Dublin City Area 2 was obtained with a Multilayer Perceptron algorithm as 92.78%, Vaihingen was obtained as 79.71% with Support Vector Machines and Oakland3D with Linear Discriminant Analysis as 97.30%.

scholarly journals INTERACTIVE CLASSIFICATION OF CONSTRUCTION MATERIALS: FEEDBACK DRIVEN FRAMEWORK FOR ANNOTATION AND ANALYSIS OF 3D POINT CLOUDS

2017 ◽  
Vol XLII-2/W5 ◽  
pp. 343-347
Author(s):  
M. R. Hess ◽  
V. Petrovic ◽  
F. Kuester
Keyword(s):  
Cultural Heritage ◽  
Point Cloud ◽  
Laser Scanning ◽  
Building Materials ◽  
Point Clouds ◽  
Normal Vector ◽  
Surface Geometry ◽  
Historical Structures ◽  
3D Point Clouds

Digital documentation of cultural heritage structures is increasingly more common through the application of different imaging techniques. Many works have focused on the application of laser scanning and photogrammetry techniques for the acquisition of threedimensional (3D) geometry detailing cultural heritage sites and structures. With an abundance of these 3D data assets, there must be a digital environment where these data can be visualized and analyzed. Presented here is a feedback driven visualization framework that seamlessly enables interactive exploration and manipulation of massive point cloud data. The focus of this work is on the classification of different building materials with the goal of building more accurate as-built information models of historical structures. User defined functions have been tested within the interactive point cloud visualization framework to evaluate automated and semi-automated classification of 3D point data. These functions include decisions based on observed color, laser intensity, normal vector or local surface geometry. Multiple case studies are presented here to demonstrate the flexibility and utility of the presented point cloud visualization framework to achieve classification objectives.

scholarly journals SEGMENTATION OF 3D PHOTOGRAMMETRIC POINT CLOUD FOR 3D BUILDING MODELING

2018 ◽  
Vol XLII-4/W10 ◽  
pp. 135-142 ◽  
Author(s):  
E. Özdemir ◽  
F. Remondino
Keyword(s):  
Point Cloud ◽  
Ground Level ◽  
Point Clouds ◽  
Energy Evaluation ◽  
3D Point Clouds ◽  
Building Modeling ◽  
3D Cadastre ◽  
Segmentation Methods ◽  
The One

<p><strong>Abstract.</strong> 3D city modeling has become important over the last decades as these models are being used in different studies including, energy evaluation, visibility analysis, 3D cadastre, urban planning, change detection, disaster management, etc. Segmentation and classification of photogrammetric or LiDAR data is important for 3D city models as these are the main data sources, and, these tasks are challenging due to their complexity. This study presents research in progress, which focuses on the segmentation and classification of 3D point clouds and orthoimages to generate 3D urban models. The aim is to classify photogrammetric-based point clouds (&amp;gt;<span class="thinspace"></span>30<span class="thinspace"></span>pts/sqm) in combination with aerial RGB orthoimages (~<span class="thinspace"></span>10<span class="thinspace"></span>cm, RGB image) in order to name buildings, ground level objects (GLOs), trees, grass areas, and other regions. If on the one hand the classification of aerial orthoimages is foreseen to be a fast approach to get classes and then transfer them from the image to the point cloud space, on the other hand, segmenting a point cloud is expected to be much more time consuming but to provide significant segments from the analyzed scene. For this reason, the proposed method combines segmentation methods on the two geoinformation in order to achieve better results.</p>

scholarly journals GENERATIVE NETWORKS FOR POINT CLOUD GENERATION IN CULTURAL HERITAGE DOMAIN

2021 ◽  
Author(s):  
Massimo Martini ◽  
Roberto Pierdicca ◽  
Marina Paolanti ◽  
Ramona Quattrini ◽  
Eva Savina Malinverni ◽  
...  
Keyword(s):  
Cultural Heritage ◽  
Point Cloud ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Historical Buildings ◽  
Architectural Elements ◽  
3D Point Clouds ◽  
Chamfer Distance ◽  
Segmentation Task ◽  
Jensen Shannon Divergence

In the Cultural Heritage (CH) domain, the semantic segmentation of 3D point clouds with Deep Learning (DL) techniques allows to recognize historical architectural elements, at a suitable level of detail, and hence expedite the process of modelling historical buildings for the development of BIM models from survey data. However, it is more difficult to collect a balanced dataset of labelled architectural elements for training a network. In fact, the CH objects are unique, and it is challenging for the network to recognize this kind of data. In recent years, Generative Networks have proven to be proper for generating new data. Starting from such premises, in this paper Generative Networks have been used for augmenting a CH dataset. In particular, the performances of three state-of-art Generative Networks such as PointGrow, PointFLow and PointGMM have been compared in terms of Jensen-Shannon Divergence (JSD), the Minimum Matching Distance-Chamfer Distance (MMD-CD) and the Minimum Matching Distance-Earth Mover’s Distance (MMD-EMD). The objects generated have been used for augmenting two classes of ArCH dataset, which are columns and windows. Then a DGCNN-Mod network was trained and tested for the semantic segmentation task, comparing the performance in the case of the ArCH dataset without and with augmentation.

scholarly journals Spectral Pattern Classification in Lidar Data for Rock Identification in Outcrops

2014 ◽  
Vol 2014 ◽  
pp. 1-10 ◽  
Author(s):  
Leonardo Campos Inocencio ◽  
Mauricio Roberto Veronez ◽  
Francisco Manoel Wohnrath Tognoli ◽  
Marcelo Kehl de Souza ◽  
Reginaldo Macedônio da Silva ◽  
...  
Keyword(s):  
Point Cloud ◽  
Laser Scanner ◽  
Point Clouds ◽  
Lidar Data ◽  
Spectral Signatures ◽  
Physical Chemical ◽  
Compositional Changes ◽  
Number Of Classes ◽  
Digital Outcrop

The present study aimed to develop and implement a method for detection and classification of spectral signatures in point clouds obtained from terrestrial laser scanner in order to identify the presence of different rocks in outcrops and to generate a digital outcrop model. To achieve this objective, a software based on cluster analysis was created, named K-Clouds. This software was developed through a partnership between UNISINOS and the company V3D. This tool was designed to begin with an analysis and interpretation of a histogram from a point cloud of the outcrop and subsequently indication of a number of classes provided by the user, to process the intensity return values. This classified information can then be interpreted by geologists, to provide a better understanding and identification from the existing rocks in the outcrop. Beyond the detection of different rocks, this work was able to detect small changes in the physical-chemical characteristics of the rocks, as they were caused by weathering or compositional changes.

scholarly journals SEMANTIC ENRICHMENT OF POINT CLOUD BY AUTOMATIC EXTRACTION AND ENHANCEMENT OF 360&#176; PANORAMAS

2019 ◽  
Vol XLII-2/W17 ◽  
pp. 355-362 ◽  
Author(s):  
A. Tabkha ◽  
R. Hajji ◽  
R. Billen ◽  
F. Poux
Keyword(s):  
Point Cloud ◽  
State Of The Art ◽  
Point Clouds ◽  
Automatic Process ◽  
Automatic Extraction ◽  
Learning Approaches ◽  
Semantic Enrichment ◽  
3D Point Clouds ◽  
Important Challenge

Abstract. The raw nature of point clouds is an important challenge for their direct exploitation in architecture, engineering and construction applications. Particularly, their lack of semantics hinders their utility for automatic workflows (Poux, 2019). In addition, the volume and the irregularity of the structure of point clouds makes it difficult to directly and automatically classify datasets efficiently, especially when compared to the state-of-the art 2D raster classification. Recently, with the advances in deep learning models such as convolutional neural networks (CNNs) , the performance of image-based classification of remote sensing scenes has improved considerably (Chen et al., 2018; Cheng et al., 2017). In this research, we examine a simple and innovative approach that represent large 3D point clouds through multiple 2D projections to leverage learning approaches based on 2D images. In other words, the approach in this study proposes an automatic process for extracting 360° panoramas, enhancing these to be able to leverage raster data to obtain domain-base semantic enrichment possibilities. Indeed, it is very important to obtain a rigorous characterization for use in the classification of a point cloud. Especially because there is a very large variety of 3D point cloud domain applications. In order to test the adequacy of the method and its potential for generalization, several tests were performed on different datasets. The developed semantic augmentation algorithm uses only the attributes X, Y, Z and camera positions as inputs.

Sign in / Sign up

Export Citation Format

Share Document