Estimating Reference Bony Shape Models for Orthognathic Surgical Planning Using 3D Point-Cloud Deep Learning

Abstract Background The 3D point cloud is the most direct and effective data form for studying plant structure and morphology. In point cloud studies, the point cloud segmentation of individual plants to organs directly determines the accuracy of organ-level phenotype estimation and the reliability of the 3D plant reconstruction. However, highly accurate, automatic, and robust point cloud segmentation approaches for plants are unavailable. Thus, the high-throughput segmentation of many shoots is challenging. Although deep learning can feasibly solve this issue, software tools for 3D point cloud annotation to construct the training dataset are lacking. Results We propose a top-to-down point cloud segmentation algorithm using optimal transportation distance for maize shoots. We apply our point cloud annotation toolkit for maize shoots, Label3DMaize, to achieve semi-automatic point cloud segmentation and annotation of maize shoots at different growth stages, through a series of operations, including stem segmentation, coarse segmentation, fine segmentation, and sample-based segmentation. The toolkit takes ∼4–10 minutes to segment a maize shoot and consumes 10–20% of the total time if only coarse segmentation is required. Fine segmentation is more detailed than coarse segmentation, especially at the organ connection regions. The accuracy of coarse segmentation can reach 97.2% that of fine segmentation. Conclusion Label3DMaize integrates point cloud segmentation algorithms and manual interactive operations, realizing semi-automatic point cloud segmentation of maize shoots at different growth stages. The toolkit provides a practical data annotation tool for further online segmentation research based on deep learning and is expected to promote automatic point cloud processing of various plants.

Download Full-text

Food Volume Estimation Based on Deep Learning View Synthesis from a Single Depth Map

Nutrients ◽

10.3390/nu10122005 ◽

2018 ◽

Vol 10 (12) ◽

pp. 2005 ◽

Cited By ~ 12

Author(s):

Frank Lo ◽

Yingnan Sun ◽

Jianing Qiu ◽

Benny Lo

Keyword(s):

Neural Network ◽

Deep Learning ◽

Point Cloud ◽

Volume Estimation ◽

Assessment System ◽

View Synthesis ◽

Depth Image ◽

3D Point Cloud ◽

Viewing Angle ◽

3D Point Clouds

An objective dietary assessment system can help users to understand their dietary behavior and enable targeted interventions to address underlying health problems. To accurately quantify dietary intake, measurement of the portion size or food volume is required. For volume estimation, previous research studies mostly focused on using model-based or stereo-based approaches which rely on manual intervention or require users to capture multiple frames from different viewing angles which can be tedious. In this paper, a view synthesis approach based on deep learning is proposed to reconstruct 3D point clouds of food items and estimate the volume from a single depth image. A distinct neural network is designed to use a depth image from one viewing angle to predict another depth image captured from the corresponding opposite viewing angle. The whole 3D point cloud map is then reconstructed by fusing the initial data points with the synthesized points of the object items through the proposed point cloud completion and Iterative Closest Point (ICP) algorithms. Furthermore, a database with depth images of food object items captured from different viewing angles is constructed with image rendering and used to validate the proposed neural network. The methodology is then evaluated by comparing the volume estimated by the synthesized 3D point cloud with the ground truth volume of the object items.

Download Full-text

Deep Learning Based Semantic Labelling of 3D Point Cloud in Visual SLAM

IOP Conference Series Materials Science and Engineering ◽

10.1088/1757-899x/428/1/012023 ◽

2018 ◽

Vol 428 ◽

pp. 012023 ◽

Cited By ~ 3

Author(s):

Xuxiang Qi ◽

Shaowu Yang ◽

Yuejin Yan

Keyword(s):

Deep Learning ◽

Point Cloud ◽

3D Point Cloud ◽

Visual Slam ◽

Semantic Labelling

Download Full-text

Point Encoder GAN: A deep learning model for 3D point cloud inpainting

Neurocomputing ◽

10.1016/j.neucom.2019.12.032 ◽

2020 ◽

Vol 384 ◽

pp. 192-199 ◽

Cited By ~ 2

Author(s):

Yikuan Yu ◽

Zitian Huang ◽

Fei Li ◽

Haodong Zhang ◽

Xinyi Le

Keyword(s):

Deep Learning ◽

Point Cloud ◽

Learning Model ◽

3D Point Cloud ◽

Deep Learning Model

Download Full-text

SEMANTIC SEGMENTATION FOR BUILDING FAÇADE 3D POINT CLOUD FROM 2D ORTHOPHOTO IMAGES USING TRANSFER LEARNING

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xliii-b2-2021-201-2021 ◽

2021 ◽

Vol XLIII-B2-2021 ◽

pp. 201-206

Author(s):

A. Murtiyoso ◽

C. Lhenry ◽

T. Landes ◽

P. Grussenmeyer ◽

E. Alby

Keyword(s):

Deep Learning ◽

Transfer Learning ◽

Point Cloud ◽

Semantic Segmentation ◽

3D Point Cloud ◽

Semantic Classification ◽

3D Data ◽

The Neural Network ◽

Building Facades ◽

Open Question

Abstract. The task of semantic segmentation is an important one in the context of 3D building modelling. Indeed, developments in 3D generation techniques have rendered the point cloud ubiquitous. However pure data acquisition only captures geometric information and semantic classification remains to be performed, often manually, in order to give a tangible sense to the 3D data. Recently progress in computing power also opened the way for massive application of deep learning methods, including for semantic segmentation purposes. Although well established in the processing of 2D images, deep learning solutions remain an open question for 3D data. In this study, we aim to benefit from the vastly more developed 2D semantic segmentation by performing transfer learning on a photogrammetric orthoimage. The neural network was trained using labelled and rectified images of building façades. Another programme was then written to permit the passage between 2D orthoimage and 3D point cloud. Results show that the approach worked well and presents an alternative to help the automation process for point cloud semantic segmentation, at least in the case of photogrammetric data.

Download Full-text

ROOFN3D: DEEP LEARNING TRAINING DATA FOR 3D BUILDING RECONSTRUCTION

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-2-1191-2018 ◽

2018 ◽

Vol XLII-2 ◽

pp. 1191-1198 ◽

Cited By ~ 5

Author(s):

A. Wichmann ◽

A. Agoub ◽

M. Kada

Keyword(s):

Machine Learning ◽

Deep Learning ◽

Point Cloud ◽

Geometric Model ◽

Training Data ◽

Computer Hardware ◽

Training Dataset ◽

3D Point Cloud ◽

Learning Methods ◽

Building Reconstruction

Machine learning methods have gained in importance through the latest development of artificial intelligence and computer hardware. Particularly approaches based on deep learning have shown that they are able to provide state-of-the-art results for various tasks. However, the direct application of deep learning methods to improve the results of 3D building reconstruction is often not possible due, for example, to the lack of suitable training data. To address this issue, we present RoofN3D which provides a new 3D point cloud training dataset that can be used to train machine learning models for different tasks in the context of 3D building reconstruction. It can be used, among others, to train semantic segmentation networks or to learn the structure of buildings and the geometric model construction. Further details about RoofN3D and the developed data preparation framework, which enables the automatic derivation of training data, are described in this paper. Furthermore, we provide an overview of other available 3D point cloud training data and approaches from current literature in which solutions for the application of deep learning to unstructured and not gridded 3D point cloud data are presented.

Download Full-text

Bottleneck Identification to Semantic Segmentation of Industrial 3D Point Cloud Scene via Deep Learning

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2021/670 ◽

2021 ◽

Author(s):

Romain Cazorla ◽

Line Poinel ◽

Panagiotis Papadakis ◽

Cédric Buche

Keyword(s):

Deep Learning ◽

Point Cloud ◽

Oil And Gas ◽

Synthetic Data ◽

Semantic Segmentation ◽

Object Size ◽

3D Point Cloud ◽

Industrial Plants ◽

Bottleneck Identification ◽

Data Adaptive

Point cloud acquisition techniques are an essential tool for the digitization of industrial plants, yet the bulk of a designer's work remains manual. A first step to automatize drawing generation is to extract the semantics of the point cloud. Towards this goal, we investigate the use of deep learning to semantically segment oil and gas industrial scenes. We focus on domain characteristics such as high variation of object size, increased concavity and lack of annotated data, which hampers the use of conventional approaches. To address these issues, we advocate the use of synthetic data, adaptive downsampling and context sharing.

Download Full-text