scholarly journals SEMANTIC SEGMENTATION FOR BUILDING FAÇADE 3D POINT CLOUD FROM 2D ORTHOPHOTO IMAGES USING TRANSFER LEARNING

2021 ◽  
Vol XLIII-B2-2021 ◽  
pp. 201-206
Author(s):  
A. Murtiyoso ◽  
C. Lhenry ◽  
T. Landes ◽  
P. Grussenmeyer ◽  
E. Alby
Keyword(s):  
Deep Learning ◽  
Transfer Learning ◽  
Point Cloud ◽  
Semantic Segmentation ◽  
3D Point Cloud ◽  
Semantic Classification ◽  
3D Data ◽  
The Neural Network ◽  
Building Facades ◽  
Open Question

Abstract. The task of semantic segmentation is an important one in the context of 3D building modelling. Indeed, developments in 3D generation techniques have rendered the point cloud ubiquitous. However pure data acquisition only captures geometric information and semantic classification remains to be performed, often manually, in order to give a tangible sense to the 3D data. Recently progress in computing power also opened the way for massive application of deep learning methods, including for semantic segmentation purposes. Although well established in the processing of 2D images, deep learning solutions remain an open question for 3D data. In this study, we aim to benefit from the vastly more developed 2D semantic segmentation by performing transfer learning on a photogrammetric orthoimage. The neural network was trained using labelled and rectified images of building façades. Another programme was then written to permit the passage between 2D orthoimage and 3D point cloud. Results show that the approach worked well and presents an alternative to help the automation process for point cloud semantic segmentation, at least in the case of photogrammetric data.

scholarly journals A BENCHMARK FOR LARGE-SCALE HERITAGE POINT CLOUD SEMANTIC SEGMENTATION

2020 ◽  
Vol XLIII-B2-2020 ◽  
pp. 1419-1426
Author(s):  
F. Matrone ◽  
A. Lingua ◽  
R. Pierdicca ◽  
E. S. Malinverni ◽  
M. Paolanti ◽  
...  
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Large Scale ◽  
Semantic Segmentation ◽  
Learning Approaches ◽  
Semantic Classification ◽  
Digital Heritage ◽  
3D Data ◽  
Digital Twins

Abstract. The lack of benchmarking data for the semantic segmentation of digital heritage scenarios is hampering the development of automatic classification solutions in this field. Heritage 3D data feature complex structures and uncommon classes that prevent the simple deployment of available methods developed in other fields and for other types of data. The semantic classification of heritage 3D data would support the community in better understanding and analysing digital twins, facilitate restoration and conservation work, etc. In this paper, we present the first benchmark with millions of manually labelled 3D points belonging to heritage scenarios, realised to facilitate the development, training, testing and evaluation of machine and deep learning methods and algorithms in the heritage field. The proposed benchmark, available at http://archdataset.polito.it/, comprises datasets and classification results for better comparisons and insights into the strengths and weaknesses of different machine and deep learning approaches for heritage point cloud semantic segmentation, in addition to promoting a form of crowdsourcing to enrich the already annotated database.

scholarly journals FUSION OF FEATURE BASED AND DEEP LEARNING METHODS FOR CLASSIFICATION OF MMS POINT CLOUDS

2019 ◽  
Vol XLII-2/W16 ◽  
pp. 235-242 ◽  
Author(s):  
D. Tosic ◽  
S. Tuttas ◽  
L. Hoegner ◽  
U. Stilla
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Local Geometry ◽  
Learning Approach ◽  
Semantic Classification ◽  
Feature Based ◽  
Urban Scene

<p><strong>Abstract.</strong> This work proposes an approach for semantic classification of an outdoor-scene point cloud acquired with a high precision Mobile Mapping System (MMS), with major goal to contribute to the automatic creation of High Definition (HD) Maps. The automatic point labeling is achieved by utilizing the combination of a feature-based approach for semantic classification of point clouds and a deep learning approach for semantic segmentation of images. Both, point cloud data, as well as the data from a multi-camera system are used for gaining spatial information in an urban scene. Two types of classification applied for this task are: 1) Feature-based approach, in which the point cloud is organized into a supervoxel structure for capturing geometric characteristics of points. Several geometric features are then extracted for appropriate representation of the local geometry, followed by removing the effect of local tendency for each supervoxel to enhance the distinction between similar structures. And lastly, the Random Forests (RF) algorithm is applied in the classification phase, for assigning labels to supervoxels and therefore to points within them. 2) The deep learning approach is employed for semantic segmentation of MMS images of the same scene. To achieve this, an implementation of Pyramid Scene Parsing Network is used. Resulting segmented images with each pixel containing a class label are then projected onto the point cloud, enabling label assignment for each point. At the end, experiment results are presented from a complex urban scene and the performance of this method is evaluated on a manually labeled dataset, for the deep learning and feature-based classification individually, as well as for the result of the labels fusion. The achieved overall accuracy with fusioned output is 0.87 on the final test set, which significantly outperforms the results of individual methods on the same point cloud. The labeled data is published on the TUM-PF Semantic-Labeling-Benchmark.</p>

scholarly journals Bottleneck Identification to Semantic Segmentation of Industrial 3D Point Cloud Scene via Deep Learning

2021 ◽  
Author(s):  
Romain Cazorla ◽  
Line Poinel ◽  
Panagiotis Papadakis ◽  
Cédric Buche
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Oil And Gas ◽  
Synthetic Data ◽  
Semantic Segmentation ◽  
Object Size ◽  
3D Point Cloud ◽  
Industrial Plants ◽  
Bottleneck Identification ◽  
Data Adaptive

Point cloud acquisition techniques are an essential tool for the digitization of industrial plants, yet the bulk of a designer's work remains manual. A first step to automatize drawing generation is to extract the semantics of the point cloud. Towards this goal, we investigate the use of deep learning to semantically segment oil and gas industrial scenes. We focus on domain characteristics such as high variation of object size, increased concavity and lack of annotated data, which hampers the use of conventional approaches. To address these issues, we advocate the use of synthetic data, adaptive downsampling and context sharing.

scholarly journals A 3D POINT CLOUD DEEP LEARNING APPROACH USING LIDAR TO IDENTIFY ANCIENT MAYA ARCHAEOLOGICAL SITES

2021 ◽  
Vol VIII-M-1-2021 ◽  
pp. 133-139
Author(s):  
H. Richards-Rissetto ◽  
D. Newton ◽  
A. Al Zadjali
Keyword(s):  
Deep Learning ◽  
Classification Accuracy ◽  
Point Cloud ◽  
Data Augmentation ◽  
World Heritage Site ◽  
Archaeological Sites ◽  
Lidar Data ◽  
3D Point Cloud ◽  
Anthropogenic Landscapes ◽  
3D Data

Abstract. Airborne light detection and ranging (LIDAR) systems allow archaeologists to capture 3D data of anthropogenic landscapes with a level of precision that permits the identification of archaeological sites in difficult to reach and inaccessible regions. These benefits have come with a deluge of LIDAR data that requires significant and costly manual labor to interpret and analyze. In order to address this challenge, researchers have explored the use of state-of-the-art automated object recognition algorithms from the field of deep learning with success. This previous research, however, has been limited to the exploration of deep learning processes that work with only 2D data, which excludes the use of available 3D data. Our research addresses this gap and contributes knowledge on the use of deep learning-based processes that can classify archaeological sites from LIDAR generated 3D point cloud datasets. LIDAR data from the UNESCO World Heritage Site of Copan, Honduras is used as the primary dataset to compare the classification accuracy of deep learning models using 2D and 3D data. The results demonstrate that models using 3D point cloud datasets provide the greatest classification accuracy in identifying Maya archaeological sites while requiring less data preparation. Further, the research contributes knowledge on the efficacy of data augmentation strategies when working with small 3D datasets.

Deep Learning on 3D Point Cloud for Semantic Segmentation

2021 ◽  
pp. 275-282
Author(s):  
Zhihan Ning ◽  
Linlin Tang ◽  
Shuhan Qi ◽  
Yang Liu
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Semantic Segmentation ◽  
3D Point Cloud

scholarly journals Label3DMaize: toolkit for 3D point cloud data annotation of maize shoots

GigaScience ◽  
2021 ◽  
Vol 10 (5) ◽  
Author(s):  
Teng Miao ◽  
Weiliang Wen ◽  
Yinglun Li ◽  
Sheng Wu ◽  
Chao Zhu ◽  
...  
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Training Dataset ◽  
Growth Stages ◽  
3D Point Cloud ◽  
Plant Structure ◽  
Data Annotation ◽  
Cloud Data ◽  
Point Cloud Segmentation ◽  
Different Growth Stages

Abstract Background The 3D point cloud is the most direct and effective data form for studying plant structure and morphology. In point cloud studies, the point cloud segmentation of individual plants to organs directly determines the accuracy of organ-level phenotype estimation and the reliability of the 3D plant reconstruction. However, highly accurate, automatic, and robust point cloud segmentation approaches for plants are unavailable. Thus, the high-throughput segmentation of many shoots is challenging. Although deep learning can feasibly solve this issue, software tools for 3D point cloud annotation to construct the training dataset are lacking. Results We propose a top-to-down point cloud segmentation algorithm using optimal transportation distance for maize shoots. We apply our point cloud annotation toolkit for maize shoots, Label3DMaize, to achieve semi-automatic point cloud segmentation and annotation of maize shoots at different growth stages, through a series of operations, including stem segmentation, coarse segmentation, fine segmentation, and sample-based segmentation. The toolkit takes ∼4–10 minutes to segment a maize shoot and consumes 10–20% of the total time if only coarse segmentation is required. Fine segmentation is more detailed than coarse segmentation, especially at the organ connection regions. The accuracy of coarse segmentation can reach 97.2% that of fine segmentation. Conclusion Label3DMaize integrates point cloud segmentation algorithms and manual interactive operations, realizing semi-automatic point cloud segmentation of maize shoots at different growth stages. The toolkit provides a practical data annotation tool for further online segmentation research based on deep learning and is expected to promote automatic point cloud processing of various plants.

scholarly journals Towards Scalable Economic Photovoltaic Potential Analysis Using Aerial Images and Deep Learning

Energies ◽  
2021 ◽  
Vol 14 (13) ◽  
pp. 3800
Author(s):  
Sebastian Krapf ◽  
Nils Kemmerzell ◽  
Syed Khawaja Haseeb Khawaja Haseeb Uddin ◽  
Manuel Hack Hack Vázquez ◽  
Fabian Netzler ◽  
...  
Keyword(s):  
Deep Learning ◽  
System Analysis ◽  
State Of The Art ◽  
Critical Role ◽  
Semantic Segmentation ◽  
Energy System ◽  
Aerial Images ◽  
Potential Analysis ◽  
3D Data ◽  
Challenges And Opportunities

Roof-mounted photovoltaic systems play a critical role in the global transition to renewable energy generation. An analysis of roof photovoltaic potential is an important tool for supporting decision-making and for accelerating new installations. State of the art uses 3D data to conduct potential analyses with high spatial resolution, limiting the study area to places with available 3D data. Recent advances in deep learning allow the required roof information from aerial images to be extracted. Furthermore, most publications consider the technical photovoltaic potential, and only a few publications determine the photovoltaic economic potential. Therefore, this paper extends state of the art by proposing and applying a methodology for scalable economic photovoltaic potential analysis using aerial images and deep learning. Two convolutional neural networks are trained for semantic segmentation of roof segments and superstructures and achieve an Intersection over Union values of 0.84 and 0.64, respectively. We calculated the internal rate of return of each roof segment for 71 buildings in a small study area. A comparison of this paper’s methodology with a 3D-based analysis discusses its benefits and disadvantages. The proposed methodology uses only publicly available data and is potentially scalable to the global level. However, this poses a variety of research challenges and opportunities, which are summarized with a focus on the application of deep learning, economic photovoltaic potential analysis, and energy system analysis.

scholarly journals Food Volume Estimation Based on Deep Learning View Synthesis from a Single Depth Map

Nutrients ◽  
2018 ◽  
Vol 10 (12) ◽  
pp. 2005 ◽  
Author(s):  
Frank Lo ◽  
Yingnan Sun ◽  
Jianing Qiu ◽  
Benny Lo
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Point Cloud ◽  
Volume Estimation ◽  
Assessment System ◽  
View Synthesis ◽  
Depth Image ◽  
3D Point Cloud ◽  
Viewing Angle ◽  
3D Point Clouds

An objective dietary assessment system can help users to understand their dietary behavior and enable targeted interventions to address underlying health problems. To accurately quantify dietary intake, measurement of the portion size or food volume is required. For volume estimation, previous research studies mostly focused on using model-based or stereo-based approaches which rely on manual intervention or require users to capture multiple frames from different viewing angles which can be tedious. In this paper, a view synthesis approach based on deep learning is proposed to reconstruct 3D point clouds of food items and estimate the volume from a single depth image. A distinct neural network is designed to use a depth image from one viewing angle to predict another depth image captured from the corresponding opposite viewing angle. The whole 3D point cloud map is then reconstructed by fusing the initial data points with the synthesized points of the object items through the proposed point cloud completion and Iterative Closest Point (ICP) algorithms. Furthermore, a database with depth images of food object items captured from different viewing angles is constructed with image rendering and used to validate the proposed neural network. The methodology is then evaluated by comparing the volume estimated by the synthesized 3D point cloud with the ground truth volume of the object items.

Review of Semantic Segmentation of Point Cloud Based on Deep Learning

2020 ◽  
Vol 57 (4) ◽  
pp. 040002
Author(s):  
张佳颖 Zhang Jiaying ◽  
赵晓丽 Zhao Xiaoli ◽  
陈正 Chen Zheng
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Semantic Segmentation
Sign in / Sign up

Export Citation Format

Share Document