scholarly journals SEMANTIC SEGMENTATION OF INDOOR 3D POINT CLOUD WITH SLENET

2019 ◽  
Vol XLII-2/W13 ◽  
pp. 785-791
Author(s):  
Y. Ding ◽  
X. Zheng ◽  
H. Xiong ◽  
Y. Zhang
Keyword(s):  
Point Cloud ◽  
Graphical Model ◽  
Rapid Development ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
3D Point Cloud ◽  
3D Point Clouds ◽  
Cloud Models ◽  
Semantic Transfer ◽  
Class Segmentation

<p><strong>Abstract.</strong> With the rapid development of new indoor sensors and acquisition techniques, the amount of indoor three dimensional (3D) point cloud models was significantly increased. However, these massive “blind” point clouds are difficult to satisfy the demand of many location-based indoor applications and GIS analysis. The robust semantic segmentation of 3D point clouds remains a challenge. In this paper, a segmentation with layout estimation network (SLENet)-based 2D&amp;ndash;3D semantic transfer method is proposed for robust segmentation of image-based indoor 3D point clouds. Firstly, a SLENet is devised to simultaneously achieve the semantic labels and indoor spatial layout estimation from 2D images. A pixel labeling pool is then constructed to incorporate the visual graphical model to realize the efficient 2D&amp;ndash;3D semantic transfer for 3D point clouds, which avoids the time-consuming pixel-wise label transfer and the reprojection error. Finally, a 3D-contextual refinement, which explores the extra-image consistency with 3D constraints is developed to suppress the labeling contradiction caused by multi-superpixel aggregation. The experiments were conducted on an open dataset (NYUDv2 indoor dataset) and a local dataset. In comparison with the state-of-the-art methods in terms of 2D semantic segmentation, SLENet can both learn discriminative enough features for inter-class segmentation while preserving clear boundaries for intra-class segmentation. Based on the excellence of SLENet, the final 3D semantic segmentation tested on the point cloud created from the local image dataset can reach a total accuracy of 89.97%, with the object semantics and indoor structural information both expressed.</p>

scholarly journals Hole Repairing Algorithm for 3D Point Cloud Model of Symmetrical Objects Grasped by the Manipulator

Sensors ◽  
2021 ◽  
Vol 21 (22) ◽  
pp. 7558
Author(s):  
Linyan Cui ◽  
Guolong Zhang ◽  
Jinshen Wang
Keyword(s):  
Point Cloud ◽  
Cloud Model ◽  
Engineering Application ◽  
Point Clouds ◽  
Real Point ◽  
3D Point Cloud ◽  
Virtual Camera ◽  
3D Point Clouds ◽  
Cloud Models ◽  
Point Cloud Model

For the engineering application of manipulator grasping objects, mechanical arm occlusion and limited imaging angle produce various holes in the reconstructed 3D point clouds of objects. Acquiring a complete point cloud model of the grasped object plays a very important role in the subsequent task planning of the manipulator. This paper proposes a method with which to automatically detect and repair the holes in the 3D point cloud model of symmetrical objects grasped by the manipulator. With the established virtual camera coordinate system and boundary detection, repair and classification of holes, the closed boundaries for the nested holes were detected and classified into two kinds, which correspond to the mechanical claw holes caused by mechanical arm occlusion and the missing surface produced by limited imaging angle. These two kinds of holes were repaired based on surface reconstruction and object symmetry. Experiments on simulated and real point cloud models demonstrate that our approach outperforms the other state-of-the-art 3D point cloud hole repair algorithms.

scholarly journals TESSERAE3D: A BENCHMARK FOR TESSERAE SEMANTIC SEGMENTATION IN 3D POINT CLOUDS

2021 ◽  
Vol V-2-2021 ◽  
pp. 121-128
Author(s):  
A. Kharroubi ◽  
L. Van Wersch ◽  
R. Billen ◽  
F. Poux
Keyword(s):  
Point Cloud ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
3D Point Cloud ◽  
Learning Models ◽  
Semantic Classes ◽  
3D Point Clouds ◽  
Wide Range ◽  
Extraction Pattern ◽  
Automated Processes

Abstract. 3D point cloud of mosaic tesserae is used by heritage researchers, restorers and archaeologists for digital investigations. Information extraction, pattern analysis and semantic assignment are necessary to complement the geometric information. Automated processes that can speed up the task are highly sought after, especially new supervised approaches. However, the availability of labelled data necessary for training supervised learning models is a significant constraint. This paper introduces Tesserae3D, a 3D point cloud benchmark dataset for training and evaluating machine learning models, applied to mosaic tesserae segmentation. It is a publicly available, very high density and coloured dataset, accompanied by a standard multi-class semantic segmentation baseline. It consists of about 502 million points and contains 11 semantic classes covering a wide range of tesserae types. We propose a semantic segmentation baseline building on radiometric and covariance features fed to ensemble learning methods. The results delineate an achievable 89% F1-score and are made available under https://github.com/akharroubi/Tesserae3D, providing a simple interface to improve the score based on feedback from the research community.

scholarly journals Automated Progress Controlling and Monitoring Using Daily Site Images and Building Information Modelling

Buildings ◽  
2019 ◽  
Vol 9 (3) ◽  
pp. 70 ◽  
Author(s):  
Hadi Mahami ◽  
Farnad Nasirzadeh ◽  
Ali Hosseininaveh Ahmadabadian ◽  
Saeid Nahavandi
Keyword(s):  
Point Cloud ◽  
Construction Projects ◽  
Progress Monitoring ◽  
Point Clouds ◽  
3D Point Cloud ◽  
Project Duration ◽  
3D Point Clouds ◽  
Cloud Models ◽  
Building Information ◽  
Construction Progress Monitoring

This research presents a novel method for automated construction progress monitoring. Using the proposed method, an accurate and complete 3D point cloud is generated for automatic outdoor and indoor progress monitoring throughout the project duration. In this method, Structured-from-Motion (SFM) and Multi-View-Stereo (MVS) algorithms coupled with photogrammetric principles for the coded targets’ detection are exploited to generate as-built 3D point clouds. The coded targets are utilized to automatically resolve the scale and increase the accuracy of the point cloud generated using SFM and MVS methods. Having generated the point cloud, the CAD model is generated from the as-built point cloud and compared with the as-planned model. Finally, the quantity of the performed work is determined in two real case study projects. The proposed method is compared to the Structured-from-Motion (SFM)/Clustering Multi-Views Stereo (CMVS)/Patch-based Multi-View Stereo (PMVS) algorithm, as a common method for generating 3D point cloud models. The proposed photogrammetric Multi-View Stereo method reveals an accuracy of around 99 percent and the generated noises are less compared to the SFM/CMVS/PMVS algorithm. It is observed that the proposed method has extensively improved the accuracy of generated points cloud compared to the SFM/CMVS/PMVS algorithm. It is believed that the proposed method may present a novel and robust tool for automated progress monitoring in construction projects.

scholarly journals DeepLabV3-Refiner-Based Semantic Segmentation Model for Dense 3D Point Clouds

2021 ◽  
Vol 13 (8) ◽  
pp. 1565
Author(s):  
Jeonghoon Kwak ◽  
Yunsick Sung
Keyword(s):  
Virtual Environment ◽  
Point Cloud ◽  
Three Dimensional ◽  
Fold Increase ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
3D Point Cloud ◽  
3D Point Clouds ◽  
Segmented Image ◽  
Human Object

Three-dimensional virtual environments can be configured as test environments of autonomous things, and remote sensing by 3D point clouds collected by light detection and range (LiDAR) can be used to detect virtual human objects by segmenting collected 3D point clouds in a virtual environment. The use of a traditional encoder-decoder model, such as DeepLabV3, improves the quality of the low-density 3D point clouds of human objects, where the quality is determined by the measurement gap of the LiDAR lasers. However, whenever a human object with a surrounding environment in a 3D point cloud is used by the traditional encoder-decoder model, it is difficult to increase the density fitting of the human object. This paper proposes a DeepLabV3-Refiner model, which is a model that refines the fit of human objects using human objects whose density has been increased through DeepLabV3. An RGB image that has a segmented human object is defined as a dense segmented image. DeepLabV3 is used to make predictions of dense segmented images and 3D point clouds for human objects in 3D point clouds. In the Refiner model, the results of DeepLabV3 are refined to fit human objects, and a dense segmented image fit to human objects is predicted. The dense 3D point cloud is calculated using the dense segmented image provided by the DeepLabV3-Refiner model. The 3D point clouds that were analyzed by the DeepLabV3-Refiner model had a 4-fold increase in density, which was verified experimentally. The proposed method had a 0.6% increase in density accuracy compared to that of DeepLabV3, and a 2.8-fold increase in the density corresponding to the human object. The proposed method was able to provide a 3D point cloud that increased the density to fit the human object. The proposed method can be used to provide an accurate 3D virtual environment by using the improved 3D point clouds.

scholarly journals Fast and Robust Flight Altitude Estimation of Multirotor UAVs in Dynamic Unstructured Environments Using 3D Point Cloud Sensors

Aerospace ◽  
2018 ◽  
Vol 5 (3) ◽  
pp. 94 ◽  
Author(s):  
Hriday Bavle ◽  
Jose Sanchez-Lopez ◽  
Paloma Puente ◽  
Alejandro Rodriguez-Ramos ◽  
Carlos Sampedro ◽  
...  
Keyword(s):  
Point Cloud ◽  
Point Clouds ◽  
Estimation Algorithm ◽  
Indoor Environments ◽  
3D Point Cloud ◽  
Flight Altitude ◽  
Cloud Data ◽  
3D Point Clouds ◽  
Laser Altimeters ◽  
Altitude Estimation

This paper presents a fast and robust approach for estimating the flight altitude of multirotor Unmanned Aerial Vehicles (UAVs) using 3D point cloud sensors in cluttered, unstructured, and dynamic indoor environments. The objective is to present a flight altitude estimation algorithm, replacing the conventional sensors such as laser altimeters, barometers, or accelerometers, which have several limitations when used individually. Our proposed algorithm includes two stages: in the first stage, a fast clustering of the measured 3D point cloud data is performed, along with the segmentation of the clustered data into horizontal planes. In the second stage, these segmented horizontal planes are mapped based on the vertical distance with respect to the point cloud sensor frame of reference, in order to provide a robust flight altitude estimation even in presence of several static as well as dynamic ground obstacles. We validate our approach using the IROS 2011 Kinect dataset available in the literature, estimating the altitude of the RGB-D camera using the provided 3D point clouds. We further validate our approach using a point cloud sensor on board a UAV, by means of several autonomous real flights, closing its altitude control loop using the flight altitude estimated by our proposed method, in presence of several different static as well as dynamic ground obstacles. In addition, the implementation of our approach has been integrated in our open-source software framework for aerial robotics called Aerostack.

scholarly journals Multi-view attention-convolution pooling network for 3D point cloud classification

2021 ◽  
Author(s):  
Wenju Wang ◽  
Tao Wang ◽  
Yu Cai
Keyword(s):  
Point Cloud ◽  
Point Clouds ◽  
Information Loss ◽  
Feature Representation ◽  
3D Point Cloud ◽  
Current Output ◽  
3D Point Clouds ◽  
Cloud Classification ◽  
Feature Information ◽  
Point Cloud Classification

AbstractClassifying 3D point clouds is an important and challenging task in computer vision. Currently, classification methods using multiple views lose characteristic or detail information during the representation or processing of views. For this reason, we propose a multi-view attention-convolution pooling network framework for 3D point cloud classification tasks. This framework uses Res2Net to extract the features from multiple 2D views. Our attention-convolution pooling method finds more useful information in the input data related to the current output, effectively solving the problem of feature information loss caused by feature representation and the detail information loss during dimensionality reduction. Finally, we obtain the probability distribution of the model to be classified using a full connection layer and the softmax function. The experimental results show that our framework achieves higher classification accuracy and better performance than other contemporary methods using the ModelNet40 dataset.

scholarly journals IMAGE TO POINT CLOUD TRANSLATION USING CONDITIONAL GENERATIVE ADVERSARIAL NETWORK FOR AIRBORNE LIDAR DATA

2021 ◽  
Vol V-2-2021 ◽  
pp. 169-174
Author(s):  
T. Shinohara ◽  
H. Xiu ◽  
M. Matsuoka
Keyword(s):  
Point Cloud ◽  
Large Scale ◽  
Point Clouds ◽  
Airborne Lidar ◽  
Real Point ◽  
3D Point Cloud ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
3D Point Clouds ◽  
Latent Features

Abstract. This study introduces a novel image to a 3D point-cloud translation method with a conditional generative adversarial network that creates a large-scale 3D point cloud. This can generate supervised point clouds observed via airborne LiDAR from aerial images. The network is composed of an encoder to produce latent features of input images, generator to translate latent features to fake point clouds, and discriminator to classify false or real point clouds. The encoder is a pre-trained ResNet; to overcome the difficulty of generating 3D point clouds in an outdoor scene, we use a FoldingNet with features from ResNet. After a fixed number of iterations, our generator can produce fake point clouds that correspond to the input image. Experimental results show that our network can learn and generate certain point clouds using the data from the 2018 IEEE GRSS Data Fusion Contest.

scholarly journals Semantic Segmentation of 3D Point Cloud Based on Spatial Eight-Quadrant Kernel Convolution

2021 ◽  
Vol 13 (16) ◽  
pp. 3140
Author(s):  
Liman Liu ◽  
Jinjin Yu ◽  
Longyu Tan ◽  
Wanjuan Su ◽  
Lin Zhao ◽  
...  
Keyword(s):  
Point Cloud ◽  
Poor Performance ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Small Object ◽  
Fine Grained ◽  
3D Point Clouds ◽  
Kernel Convolution ◽  
Segmentation Accuracy ◽  
Indoor Scenes

In order to deal with the problem that some existing semantic segmentation networks for 3D point clouds generally have poor performance on small objects, a Spatial Eight-Quadrant Kernel Convolution (SEQKC) algorithm is proposed to enhance the ability of the network for extracting fine-grained features from 3D point clouds. As a result, the semantic segmentation accuracy of small objects in indoor scenes can be improved. To be specific, in the spherical space of the point cloud neighborhoods, a kernel point with attached weights is constructed in each octant, the distances between the kernel point and the points in its neighborhood are calculated, and the distance and the kernel points’ weights are used together to weight the point cloud features in the neighborhood space. In this case, the relationship between points are modeled, so that the local fine-grained features of the point clouds can be extracted by the SEQKC. Based on the SEQKC, we design a downsampling module for point clouds, and embed it into classical semantic segmentation networks (PointNet++, PointSIFT and PointConv) for semantic segmentation. Experimental results on benchmark dataset ScanNet V2 show that SEQKC-based PointNet++, PointSIFT and PointConv outperform the original networks about 1.35–2.12% in terms of MIoU, and they effectively improve the semantic segmentation performance of the networks for small objects of indoor scenes, e.g., the segmentation accuracy of small object “picture” is improved from 0.70% of PointNet++ to 10.37% of SEQKC-PointNet++.

An Improved Method for Restoring the Shape of 3D Point Cloud Surfaces

2018 ◽  
Vol 9 (2) ◽  
pp. 37-53
Author(s):  
Sinh Van Nguyen ◽  
Ha Manh Tran ◽  
Minh Khai Tran
Keyword(s):  
Geometric Modeling ◽  
Point Cloud ◽  
Point Clouds ◽  
Tangent Plane ◽  
Surface Patch ◽  
3D Point Cloud ◽  
Improved Method ◽  
Initial Shape ◽  
Cloud Data ◽  
3D Point Clouds

Building 3D objects or reconstructing their surfaces from 3D point cloud data are researched activities in the field of geometric modeling and computer graphics. In the recent years, they are also studied and used in some fields such as: graph models and simulation; image processing or restoration of digital heritages. This article presents an improved method for restoring the shape of 3D point cloud surfaces. The method is a combination of creating a Bezier surface patch and computing tangent plane of 3D points to fill holes on a surface of 3D point clouds. This method is described as follows: at first, a boundary for each hole on the surface is identified. The holes are then filled by computing Bezier curves of surface patches to find missing points. After that, the holes are refined based on two steps (rough and elaborate) to adjust the inserted points and preserve the local curvature of the holes. The contribution of the proposed method has been shown in processing time and the novelty of combined computation in this method has preserved the initial shape of the surface

Sign in / Sign up

Export Citation Format

Share Document