scholarly journals AttAN: Attention Adversarial Networks for 3D Point Cloud Semantic Segmentation

2020 ◽  
Author(s):  
Gege Zhang ◽  
Qinghua Ma ◽  
Licheng Jiao ◽  
Fang Liu ◽  
Qigong Sun
Keyword(s):  
Point Cloud ◽  
Dimensional Space ◽  
Three Dimensional ◽  
Semantic Segmentation ◽  
Autonomous Driving ◽  
3D Point Cloud ◽  
Adversarial Learning ◽  
Adversarial Networks ◽  
Segmentation Accuracy ◽  
Segmentation Task

3D point cloud semantic segmentation has attracted wide attention with its extensive applications in autonomous driving, AR/VR, and robot sensing fields. However, in existing methods, each point in the segmentation results is predicted independently from each other. This property causes the non-contiguity of label sets in three-dimensional space and produces many noisy label points, which hinders the improvement of segmentation accuracy. To address this problem, we first extend adversarial learning to this task and propose a novel framework Attention Adversarial Networks (AttAN). With high-order correlations in label sets learned from the adversarial learning, segmentation network can predict labels closer to the real ones and correct noisy results. Moreover, we design an additive attention block for the segmentation network, which is used to automatically focus on regions critical to the segmentation task by learning the correlation between multi-scale features. Adversarial learning, which explores the underlying relationship between labels in high-dimensional space, opens up a new way in 3D point cloud semantic segmentation. Experimental results on ScanNet and S3DIS datasets show that this framework effectively improves the segmentation quality and outperforms other state-of-the-art methods.

scholarly journals 3D Point Cloud of Single Tree Branches and Leaves Semantic Segmentation Based on Modified PointNet Network

2021 ◽  
Vol 2074 (1) ◽  
pp. 012026
Author(s):  
Renpeng Liu ◽  
Lisheng Ren ◽  
Fang Wang
Keyword(s):  
Point Cloud ◽  
Semantic Segmentation ◽  
Normal Vector ◽  
3D Point Cloud ◽  
Tree Model ◽  
Single Tree ◽  
Point Cloud Segmentation ◽  
Segmentation Accuracy ◽  
Tree Branch ◽  
The Impact

Abstract Semantic segmentation of single tree 3D point cloud is one of the key technologies in building tree model. It plays an important role in tree skeleton extraction, tree pruning, tree model reconstruction and other fields. Because the area of a single leaf is smaller than that of the whole tree, the segmentation of branches and leaves is a challenging problem. In view of the above problems, this paper first migrates PointNet to the tree branch and leaf point cloud segmentation, and proposes an automatic segmentation method based on improved PointNet. According to the difference of normal direction between leaves and branches, the point cloud information of three dimensions coordinates, color and normal vector is input into the point feature space. In data processing, increase the number of each block data, so that the network can better learn features. MLP is added to the original PointNet network to improve the ability of extracting and learning local features. In addition, in the process of feature extraction, jump connection is added to realize feature reuse and make full use of different levels of features. The original 1×1 filter of PointNet is replaced by 3×1 filter to improve the segmentation accuracy of tree point cloud. The focus loss function focal loss is introduced into the field of 3D point cloud to reduce the impact of the imbalance of point cloud samples on the results. The results show that the improved method improves the accuracy of tree branch point cloud segmentation compared with the original PointNet for branch and leaf segmentation. The segmentation accuracy of structural elements of branches and leaves is more than 88%, and MIoU is 48%.

scholarly journals DeepLabV3-Refiner-Based Semantic Segmentation Model for Dense 3D Point Clouds

2021 ◽  
Vol 13 (8) ◽  
pp. 1565
Author(s):  
Jeonghoon Kwak ◽  
Yunsick Sung
Keyword(s):  
Virtual Environment ◽  
Point Cloud ◽  
Three Dimensional ◽  
Fold Increase ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
3D Point Cloud ◽  
3D Point Clouds ◽  
Segmented Image ◽  
Human Object

Three-dimensional virtual environments can be configured as test environments of autonomous things, and remote sensing by 3D point clouds collected by light detection and range (LiDAR) can be used to detect virtual human objects by segmenting collected 3D point clouds in a virtual environment. The use of a traditional encoder-decoder model, such as DeepLabV3, improves the quality of the low-density 3D point clouds of human objects, where the quality is determined by the measurement gap of the LiDAR lasers. However, whenever a human object with a surrounding environment in a 3D point cloud is used by the traditional encoder-decoder model, it is difficult to increase the density fitting of the human object. This paper proposes a DeepLabV3-Refiner model, which is a model that refines the fit of human objects using human objects whose density has been increased through DeepLabV3. An RGB image that has a segmented human object is defined as a dense segmented image. DeepLabV3 is used to make predictions of dense segmented images and 3D point clouds for human objects in 3D point clouds. In the Refiner model, the results of DeepLabV3 are refined to fit human objects, and a dense segmented image fit to human objects is predicted. The dense 3D point cloud is calculated using the dense segmented image provided by the DeepLabV3-Refiner model. The 3D point clouds that were analyzed by the DeepLabV3-Refiner model had a 4-fold increase in density, which was verified experimentally. The proposed method had a 0.6% increase in density accuracy compared to that of DeepLabV3, and a 2.8-fold increase in the density corresponding to the human object. The proposed method was able to provide a 3D point cloud that increased the density to fit the human object. The proposed method can be used to provide an accurate 3D virtual environment by using the improved 3D point clouds.

scholarly journals Adversarial Data Augmentation on Breast MRI Segmentation

2021 ◽  
Vol 11 (10) ◽  
pp. 4554
Author(s):  
João F. Teixeira ◽  
Mariana Dias ◽  
Eva Batista ◽  
Joana Costa ◽  
Luís F. Teixeira ◽  
...  
Keyword(s):  
Data Augmentation ◽  
Medical Image Analysis ◽  
Synthetic Data ◽  
Breast Mri ◽  
Semantic Segmentation ◽  
Adversarial Networks ◽  
Augmentation Strategy ◽  
Magnetic Resonance Imaging Mri ◽  
Segmentation Task ◽  
The Impact

The scarcity of balanced and annotated datasets has been a recurring problem in medical image analysis. Several researchers have tried to fill this gap employing dataset synthesis with adversarial networks (GANs). Breast magnetic resonance imaging (MRI) provides complex, texture-rich medical images, with the same annotation shortage issues, for which, to the best of our knowledge, no previous work tried synthesizing data. Within this context, our work addresses the problem of synthesizing breast MRI images from corresponding annotations and evaluate the impact of this data augmentation strategy on a semantic segmentation task. We explored variations of image-to-image translation using conditional GANs, namely fitting the generator’s architecture with residual blocks and experimenting with cycle consistency approaches. We studied the impact of these changes on visual verisimilarity and how an U-Net segmentation model is affected by the usage of synthetic data. We achieved sufficiently realistic-looking breast MRI images and maintained a stable segmentation score even when completely replacing the dataset with the synthetic set. Our results were promising, especially when concerning to Pix2PixHD and Residual CycleGAN architectures.

scholarly journals Automatic Reconstruction of Multi-Level Indoor Spaces from Point Cloud and Trajectory

Sensors ◽  
2021 ◽  
Vol 21 (10) ◽  
pp. 3493
Author(s):  
Gahyeon Lim ◽  
Nakju Doh
Keyword(s):  
Point Cloud ◽  
Dimensional Space ◽  
Three Dimensional ◽  
Reconstruction Method ◽  
Complex Environments ◽  
Indoor Space ◽  
Multi Level ◽  
Level Building ◽  
Indoor Spaces ◽  
Data Term

Remarkable progress in the development of modeling methods for indoor spaces has been made in recent years with a focus on the reconstruction of complex environments, such as multi-room and multi-level buildings. Existing methods represent indoor structure models as a combination of several sub-spaces, which are constructed by room segmentation or horizontal slicing approach that divide the multi-room or multi-level building environments into several segments. In this study, we propose an automatic reconstruction method of multi-level indoor spaces with unique models, including inter-room and inter-floor connections from point cloud and trajectory. We construct structural points from registered point cloud and extract piece-wise planar segments from the structural points. Then, a three-dimensional space decomposition is conducted and water-tight meshes are generated with energy minimization using graph cut algorithm. The data term of the energy function is expressed as a difference in visibility between each decomposed space and trajectory. The proposed method allows modeling of indoor spaces in complex environments, such as multi-room, room-less, and multi-level buildings. The performance of the proposed approach is evaluated for seven indoor space datasets.

Automated Surface Inspection Using 3D Point Cloud Data in Manufacturing: A Case Study

2018 ◽  
Author(s):  
Romina Dastoorian ◽  
Ahmad E. Elhabashy ◽  
Wenmeng Tian ◽  
Lee J. Wells ◽  
Jaime A. Camelio
Keyword(s):  
Point Cloud ◽  
Surface Defect ◽  
Fault Location ◽  
Three Dimensional ◽  
Practical Case ◽  
3D Point Cloud ◽  
Point Cloud Data ◽  
Defect Inspection ◽  
Cloud Data

With the latest advancements in three-dimensional (3D) measurement technologies, obtaining 3D point cloud data for inspection purposes in manufacturing is becoming more common. While 3D point cloud data allows for better inspection capabilities, their analysis is typically challenging. Especially with unstructured 3D point cloud data, containing coordinates at random locations, the challenges increase with higher levels of noise and larger volumes of data. Hence, the objective of this paper is to extend the previously developed Adaptive Generalized Likelihood Ratio (AGLR) approach to handle unstructured 3D point cloud data used for automated surface defect inspection in manufacturing. More specifically, the AGLR approach was implemented in a practical case study to inspect twenty-seven samples, each with a unique fault. These faults were designed to cover an array of possible faults having three different sizes, three different magnitudes, and located in three different locations. The results show that the AGLR approach can indeed differentiate between non-faulty and a varying range of faulty surfaces while being able to pinpoint the fault location. This work also serves as a validation for the previously developed AGLR approach in a practical scenario.

scholarly journals Semantic Segmentation of Underwater Images Based on Improved Deeplab

2020 ◽  
Vol 8 (3) ◽  
pp. 188
Author(s):  
Fangfang Liu ◽  
Ming Fang
Keyword(s):  
Semantic Segmentation ◽  
Autonomous Driving ◽  
Correction Method ◽  
Target Object ◽  
Original Method ◽  
Indoor Navigation ◽  
Fine Tuning ◽  
Object Boundary ◽  
Current State ◽  
Segmentation Accuracy

Image semantic segmentation technology has been increasingly applied in many fields, for example, autonomous driving, indoor navigation, virtual reality and augmented reality. However, underwater scenes, where there is a huge amount of marine biological resources and irreplaceable biological gene banks that need to be researched and exploited, are limited. In this paper, image semantic segmentation technology is exploited to study underwater scenes. We extend the current state-of-the-art semantic segmentation network DeepLabv3 + and employ it as the basic framework. First, the unsupervised color correction method (UCM) module is introduced to the encoder structure of the framework to improve the quality of the image. Moreover, two up-sampling layers are added to the decoder structure to retain more target features and object boundary information. The model is trained by fine-tuning and optimizing relevant parameters. Experimental results indicate that the image obtained by our method demonstrates better performance in improving the appearance of the segmented target object and avoiding its pixels from mingling with other class’s pixels, enhancing the segmentation accuracy of the target boundaries and retaining more feature information. Compared with the original method, our method improves the segmentation accuracy by 3%.

scholarly journals Voxel-based 3D Point Cloud Semantic Segmentation: Unsupervised Geometric and Relationship Featuring vs Deep Learning Methods

2019 ◽  
Vol 8 (5) ◽  
pp. 213 ◽  
Author(s):  
Florent Poux ◽  
Roland Billen
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Semantic Representation ◽  
Structural Connectivity ◽  
Three Dimensional ◽  
Strong Support ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Learning Methods ◽  
Cloud Data

Automation in point cloud data processing is central in knowledge discovery within decision-making systems. The definition of relevant features is often key for segmentation and classification, with automated workflows presenting the main challenges. In this paper, we propose a voxel-based feature engineering that better characterize point clusters and provide strong support to supervised or unsupervised classification. We provide different feature generalization levels to permit interoperable frameworks. First, we recommend a shape-based feature set (SF1) that only leverages the raw X, Y, Z attributes of any point cloud. Afterwards, we derive relationship and topology between voxel entities to obtain a three-dimensional (3D) structural connectivity feature set (SF2). Finally, we provide a knowledge-based decision tree to permit infrastructure-related classification. We study SF1/SF2 synergy on a new semantic segmentation framework for the constitution of a higher semantic representation of point clouds in relevant clusters. Finally, we benchmark the approach against novel and best-performing deep-learning methods while using the full S3DIS dataset. We highlight good performances, easy-integration, and high F1-score (> 85%) for planar-dominant classes that are comparable to state-of-the-art deep learning.

scholarly journals MHNet: Multiscale Hierarchical Network for 3D Point Cloud Semantic Segmentation

IEEE Access ◽  
2019 ◽  
Vol 7 ◽  
pp. 173999-174012 ◽  
Author(s):  
Xiaoli Liang ◽  
Zhongliang Fu
Keyword(s):  
Point Cloud ◽  
Semantic Segmentation ◽  
3D Point Cloud ◽  
Hierarchical Network
Sign in / Sign up

Export Citation Format

Share Document