scholarly journals DeepLabV3-Refiner-Based Semantic Segmentation Model for Dense 3D Point Clouds

2021 ◽  
Vol 13 (8) ◽  
pp. 1565
Author(s):  
Jeonghoon Kwak ◽  
Yunsick Sung
Keyword(s):  
Virtual Environment ◽  
Point Cloud ◽  
Three Dimensional ◽  
Fold Increase ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
3D Point Cloud ◽  
3D Point Clouds ◽  
Segmented Image ◽  
Human Object

Three-dimensional virtual environments can be configured as test environments of autonomous things, and remote sensing by 3D point clouds collected by light detection and range (LiDAR) can be used to detect virtual human objects by segmenting collected 3D point clouds in a virtual environment. The use of a traditional encoder-decoder model, such as DeepLabV3, improves the quality of the low-density 3D point clouds of human objects, where the quality is determined by the measurement gap of the LiDAR lasers. However, whenever a human object with a surrounding environment in a 3D point cloud is used by the traditional encoder-decoder model, it is difficult to increase the density fitting of the human object. This paper proposes a DeepLabV3-Refiner model, which is a model that refines the fit of human objects using human objects whose density has been increased through DeepLabV3. An RGB image that has a segmented human object is defined as a dense segmented image. DeepLabV3 is used to make predictions of dense segmented images and 3D point clouds for human objects in 3D point clouds. In the Refiner model, the results of DeepLabV3 are refined to fit human objects, and a dense segmented image fit to human objects is predicted. The dense 3D point cloud is calculated using the dense segmented image provided by the DeepLabV3-Refiner model. The 3D point clouds that were analyzed by the DeepLabV3-Refiner model had a 4-fold increase in density, which was verified experimentally. The proposed method had a 0.6% increase in density accuracy compared to that of DeepLabV3, and a 2.8-fold increase in the density corresponding to the human object. The proposed method was able to provide a 3D point cloud that increased the density to fit the human object. The proposed method can be used to provide an accurate 3D virtual environment by using the improved 3D point clouds.

scholarly journals TESSERAE3D: A BENCHMARK FOR TESSERAE SEMANTIC SEGMENTATION IN 3D POINT CLOUDS

2021 ◽  
Vol V-2-2021 ◽  
pp. 121-128
Author(s):  
A. Kharroubi ◽  
L. Van Wersch ◽  
R. Billen ◽  
F. Poux
Keyword(s):  
Point Cloud ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
3D Point Cloud ◽  
Learning Models ◽  
Semantic Classes ◽  
3D Point Clouds ◽  
Wide Range ◽  
Extraction Pattern ◽  
Automated Processes

Abstract. 3D point cloud of mosaic tesserae is used by heritage researchers, restorers and archaeologists for digital investigations. Information extraction, pattern analysis and semantic assignment are necessary to complement the geometric information. Automated processes that can speed up the task are highly sought after, especially new supervised approaches. However, the availability of labelled data necessary for training supervised learning models is a significant constraint. This paper introduces Tesserae3D, a 3D point cloud benchmark dataset for training and evaluating machine learning models, applied to mosaic tesserae segmentation. It is a publicly available, very high density and coloured dataset, accompanied by a standard multi-class semantic segmentation baseline. It consists of about 502 million points and contains 11 semantic classes covering a wide range of tesserae types. We propose a semantic segmentation baseline building on radiometric and covariance features fed to ensemble learning methods. The results delineate an achievable 89% F1-score and are made available under https://github.com/akharroubi/Tesserae3D, providing a simple interface to improve the score based on feedback from the research community.

scholarly journals DEEP LEARNING FOR SEMANTIC SEGMENTATION OF 3D POINT CLOUD

2019 ◽  
Vol XLII-2/W15 ◽  
pp. 735-742 ◽  
Author(s):  
E. S. Malinverni ◽  
R. Pierdicca ◽  
M. Paolanti ◽  
M. Martini ◽  
C. Morbidoni ◽  
...  
Keyword(s):  
Deep Learning ◽  
Cultural Heritage ◽  
Point Cloud ◽  
Cultural Landscapes ◽  
Three Dimensional ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Training Data ◽  
Historical Building ◽  
3D Point Clouds

<p><strong>Abstract.</strong> Cultural Heritage is a testimony of past human activity, and, as such, its objects exhibit great variety in their nature, size and complexity; from small artefacts and museum items to cultural landscapes, from historical building and ancient monuments to city centers and archaeological sites. Cultural Heritage around the globe suffers from wars, natural disasters and human negligence. The importance of digital documentation is well recognized and there is an increasing pressure to document our heritage both nationally and internationally. For this reason, the three-dimensional scanning and modeling of sites and artifacts of cultural heritage have remarkably increased in recent years. The semantic segmentation of point clouds is an essential step of the entire pipeline; in fact, it allows to decompose complex architectures in single elements, which are then enriched with meaningful information within Building Information Modelling software. Notwithstanding, this step is very time consuming and completely entrusted on the manual work of domain experts, far from being automatized. This work describes a method to label and cluster automatically a point cloud based on a supervised Deep Learning approach, using a state-of-the-art Neural Network called PointNet++. Despite other methods are known, we have choose PointNet++ as it reached significant results for classifying and segmenting 3D point clouds. PointNet++ has been tested and improved, by training the network with annotated point clouds coming from a real survey and to evaluate how performance changes according to the input training data. It can result of great interest for the research community dealing with the point cloud semantic segmentation, since it makes public a labelled dataset of CH elements for further tests.</p>

scholarly journals Automatic Point Cloud Semantic Segmentation of Complex Railway Environments

2021 ◽  
Vol 13 (12) ◽  
pp. 2332
Author(s):  
Daniel Lamas ◽  
Mario Soilán ◽  
Javier Grandío ◽  
Belén Riveiro
Keyword(s):  
Point Cloud ◽  
Three Dimensional ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Study Data ◽  
Cloud Data ◽  
Traffic Lights ◽  
Cloud Processing ◽  
3D Point Clouds ◽  
Point Cloud Processing

The growing development of data digitalisation methods has increased their demand and applications in the transportation infrastructure field. Currently, mobile mapping systems (MMSs) are one of the most popular technologies for the acquisition of infrastructure data, with three-dimensional (3D) point clouds as their main product. In this work, a heuristic-based workflow for semantic segmentation of complex railway environments is presented, in which their most relevant elements are classified, namely, rails, masts, wiring, droppers, traffic lights, and signals. This method takes advantage of existing methodologies in the field for point cloud processing and segmentation, taking into account the geometry and spatial context of each classified element in the railway environment. This method is applied to a 90-kilometre-long railway lane and validated against a manual reference on random sections of the case study data. The results are presented and discussed at the object level, differentiating the type of the element. The indicators F1 scores obtained for each element are superior to 85%, being higher than 99% in rails, the most significant element of the infrastructure. These metrics showcase the quality of the algorithm, which proves that this method is efficient for the classification of long and variable railway sections, and for the assisted labelling of point cloud data for future applications based on training supervised learning models.

scholarly journals Sloped Terrain Segmentation for Autonomous Drive Using Sparse 3D Point Cloud

2014 ◽  
Vol 2014 ◽  
pp. 1-9 ◽  
Author(s):  
Seoungjae Cho ◽  
Jonghyun Kim ◽  
Warda Ikram ◽  
Kyungeun Cho ◽  
Young-Sik Jeong ◽  
...  
Keyword(s):  
Real Time ◽  
Point Cloud ◽  
Data Exchange ◽  
Three Dimensional ◽  
Point Clouds ◽  
Two Dimensions ◽  
3D Point Cloud ◽  
3D Point Clouds ◽  
Ubiquitous Environment ◽  
Ground Segmentation

A ubiquitous environment for road travel that uses wireless networks requires the minimization of data exchange between vehicles. An algorithm that can segment the ground in real time is necessary to obtain location data between vehicles simultaneously executing autonomous drive. This paper proposes a framework for segmenting the ground in real time using a sparse three-dimensional (3D) point cloud acquired from undulating terrain. A sparse 3D point cloud can be acquired by scanning the geography using light detection and ranging (LiDAR) sensors. For efficient ground segmentation, 3D point clouds are quantized in units of volume pixels (voxels) and overlapping data is eliminated. We reduce nonoverlapping voxels to two dimensions by implementing a lowermost heightmap. The ground area is determined on the basis of the number of voxels in each voxel group. We execute ground segmentation in real time by proposing an approach to minimize the comparison between neighboring voxels. Furthermore, we experimentally verify that ground segmentation can be executed at about 19.31 ms per frame.

scholarly journals SEMANTIC SEGMENTATION OF INDOOR 3D POINT CLOUD WITH SLENET

2019 ◽  
Vol XLII-2/W13 ◽  
pp. 785-791
Author(s):  
Y. Ding ◽  
X. Zheng ◽  
H. Xiong ◽  
Y. Zhang
Keyword(s):  
Point Cloud ◽  
Graphical Model ◽  
Rapid Development ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
3D Point Cloud ◽  
3D Point Clouds ◽  
Cloud Models ◽  
Semantic Transfer ◽  
Class Segmentation

<p><strong>Abstract.</strong> With the rapid development of new indoor sensors and acquisition techniques, the amount of indoor three dimensional (3D) point cloud models was significantly increased. However, these massive “blind” point clouds are difficult to satisfy the demand of many location-based indoor applications and GIS analysis. The robust semantic segmentation of 3D point clouds remains a challenge. In this paper, a segmentation with layout estimation network (SLENet)-based 2D&amp;ndash;3D semantic transfer method is proposed for robust segmentation of image-based indoor 3D point clouds. Firstly, a SLENet is devised to simultaneously achieve the semantic labels and indoor spatial layout estimation from 2D images. A pixel labeling pool is then constructed to incorporate the visual graphical model to realize the efficient 2D&amp;ndash;3D semantic transfer for 3D point clouds, which avoids the time-consuming pixel-wise label transfer and the reprojection error. Finally, a 3D-contextual refinement, which explores the extra-image consistency with 3D constraints is developed to suppress the labeling contradiction caused by multi-superpixel aggregation. The experiments were conducted on an open dataset (NYUDv2 indoor dataset) and a local dataset. In comparison with the state-of-the-art methods in terms of 2D semantic segmentation, SLENet can both learn discriminative enough features for inter-class segmentation while preserving clear boundaries for intra-class segmentation. Based on the excellence of SLENet, the final 3D semantic segmentation tested on the point cloud created from the local image dataset can reach a total accuracy of 89.97%, with the object semantics and indoor structural information both expressed.</p>

scholarly journals A DIGITAL ARCHIVE OF BOROBUDUR BASED ON 3D POINT CLOUDS

2021 ◽  
Vol XLIII-B2-2021 ◽  
pp. 577-582
Author(s):  
M. Kawato ◽  
L. Li ◽  
K. Hasegawa ◽  
M. Adachi ◽  
H. Yamaguchi ◽  
...  
Keyword(s):  
Cultural Heritage ◽  
Point Cloud ◽  
Three Dimensional ◽  
Point Clouds ◽  
World Heritage Site ◽  
Digital Archive ◽  
First Person ◽  
3D Point Cloud ◽  
3D Point Clouds ◽  
The Temple

Abstract. Three-dimensional point clouds are becoming popular representations for digital archives of cultural heritage sites. The Borobudur Temple, located in Central Java, Indonesia, was built in the 8th century. Borobudur is considered one of the greatest Buddhist monuments in the world and was listed as a UNESCO World Heritage site. We are developing a virtual reality system as a digital archive of the Borobudur Temple. This research is a collaboration between Ritsumeikan University, Japan, the Indonesian Institute of Sciences (LIPI), and the Borobudur Conservation Office, Indonesia. In our VR system, the following three data sources are integrated to form a 3D point cloud: (1) a 3D point cloud of the overall shape of the temple acquired by photogrammetry using a camera carried by a UAV, (2) a 3D point cloud obtained from precise photogrammetric measurements of selected parts of the temple building, and (3) 3D data of the hidden relief panels recovered from the archived 2D monocular photos using deep learning. Our VR system supports both the first-person view and the bird’s eye view. The first-person view allows immersive observation and appreciation of the cultural heritage. The bird’s eye view is useful for understanding the whole picture. A user can easily switch between the two views by using a user-friendly VR user interface constructed by a 3D game engine.

scholarly journals Comparison of Depth Camera and Terrestrial Laser Scanner in Monitoring Structural Deflections

Sensors ◽  
2020 ◽  
Vol 21 (1) ◽  
pp. 201
Author(s):  
Michael Bekele Maru ◽  
Donghwan Lee ◽  
Kassahun Demissie Tola ◽  
Seunghee Park
Keyword(s):  
Optical Sensors ◽  
Point Cloud ◽  
Three Dimensional ◽  
Laser Scanner ◽  
Point Clouds ◽  
Depth Camera ◽  
Terrestrial Laser Scanner ◽  
Cloud Data ◽  
3D Point Clouds ◽  
3D Information

Modeling a structure in the virtual world using three-dimensional (3D) information enhances our understanding, while also aiding in the visualization, of how a structure reacts to any disturbance. Generally, 3D point clouds are used for determining structural behavioral changes. Light detection and ranging (LiDAR) is one of the crucial ways by which a 3D point cloud dataset can be generated. Additionally, 3D cameras are commonly used to develop a point cloud containing many points on the external surface of an object around it. The main objective of this study was to compare the performance of optical sensors, namely a depth camera (DC) and terrestrial laser scanner (TLS) in estimating structural deflection. We also utilized bilateral filtering techniques, which are commonly used in image processing, on the point cloud data for enhancing their accuracy and increasing the application prospects of these sensors in structure health monitoring. The results from these sensors were validated by comparing them with the outputs from a linear variable differential transformer sensor, which was mounted on the beam during an indoor experiment. The results showed that the datasets obtained from both the sensors were acceptable for nominal deflections of 3 mm and above because the error range was less than ±10%. However, the result obtained from the TLS were better than those obtained from the DC.

scholarly journals Fast and Robust Flight Altitude Estimation of Multirotor UAVs in Dynamic Unstructured Environments Using 3D Point Cloud Sensors

Aerospace ◽  
2018 ◽  
Vol 5 (3) ◽  
pp. 94 ◽  
Author(s):  
Hriday Bavle ◽  
Jose Sanchez-Lopez ◽  
Paloma Puente ◽  
Alejandro Rodriguez-Ramos ◽  
Carlos Sampedro ◽  
...  
Keyword(s):  
Point Cloud ◽  
Point Clouds ◽  
Estimation Algorithm ◽  
Indoor Environments ◽  
3D Point Cloud ◽  
Flight Altitude ◽  
Cloud Data ◽  
3D Point Clouds ◽  
Laser Altimeters ◽  
Altitude Estimation

This paper presents a fast and robust approach for estimating the flight altitude of multirotor Unmanned Aerial Vehicles (UAVs) using 3D point cloud sensors in cluttered, unstructured, and dynamic indoor environments. The objective is to present a flight altitude estimation algorithm, replacing the conventional sensors such as laser altimeters, barometers, or accelerometers, which have several limitations when used individually. Our proposed algorithm includes two stages: in the first stage, a fast clustering of the measured 3D point cloud data is performed, along with the segmentation of the clustered data into horizontal planes. In the second stage, these segmented horizontal planes are mapped based on the vertical distance with respect to the point cloud sensor frame of reference, in order to provide a robust flight altitude estimation even in presence of several static as well as dynamic ground obstacles. We validate our approach using the IROS 2011 Kinect dataset available in the literature, estimating the altitude of the RGB-D camera using the provided 3D point clouds. We further validate our approach using a point cloud sensor on board a UAV, by means of several autonomous real flights, closing its altitude control loop using the flight altitude estimated by our proposed method, in presence of several different static as well as dynamic ground obstacles. In addition, the implementation of our approach has been integrated in our open-source software framework for aerial robotics called Aerostack.

scholarly journals Voxel-based 3D Point Cloud Semantic Segmentation: Unsupervised Geometric and Relationship Featuring vs Deep Learning Methods

2019 ◽  
Vol 8 (5) ◽  
pp. 213 ◽  
Author(s):  
Florent Poux ◽  
Roland Billen
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Semantic Representation ◽  
Structural Connectivity ◽  
Three Dimensional ◽  
Strong Support ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Learning Methods ◽  
Cloud Data

Automation in point cloud data processing is central in knowledge discovery within decision-making systems. The definition of relevant features is often key for segmentation and classification, with automated workflows presenting the main challenges. In this paper, we propose a voxel-based feature engineering that better characterize point clusters and provide strong support to supervised or unsupervised classification. We provide different feature generalization levels to permit interoperable frameworks. First, we recommend a shape-based feature set (SF1) that only leverages the raw X, Y, Z attributes of any point cloud. Afterwards, we derive relationship and topology between voxel entities to obtain a three-dimensional (3D) structural connectivity feature set (SF2). Finally, we provide a knowledge-based decision tree to permit infrastructure-related classification. We study SF1/SF2 synergy on a new semantic segmentation framework for the constitution of a higher semantic representation of point clouds in relevant clusters. Finally, we benchmark the approach against novel and best-performing deep-learning methods while using the full S3DIS dataset. We highlight good performances, easy-integration, and high F1-score (> 85%) for planar-dominant classes that are comparable to state-of-the-art deep learning.

scholarly journals Multi-view attention-convolution pooling network for 3D point cloud classification

2021 ◽  
Author(s):  
Wenju Wang ◽  
Tao Wang ◽  
Yu Cai
Keyword(s):  
Point Cloud ◽  
Point Clouds ◽  
Information Loss ◽  
Feature Representation ◽  
3D Point Cloud ◽  
Current Output ◽  
3D Point Clouds ◽  
Cloud Classification ◽  
Feature Information ◽  
Point Cloud Classification

AbstractClassifying 3D point clouds is an important and challenging task in computer vision. Currently, classification methods using multiple views lose characteristic or detail information during the representation or processing of views. For this reason, we propose a multi-view attention-convolution pooling network framework for 3D point cloud classification tasks. This framework uses Res2Net to extract the features from multiple 2D views. Our attention-convolution pooling method finds more useful information in the input data related to the current output, effectively solving the problem of feature information loss caused by feature representation and the detail information loss during dimensionality reduction. Finally, we obtain the probability distribution of the model to be classified using a full connection layer and the softmax function. The experimental results show that our framework achieves higher classification accuracy and better performance than other contemporary methods using the ModelNet40 dataset.

Sign in / Sign up

Export Citation Format

Share Document