EpNet: A Deep Neural Network for Ear Detection in 3D Point Clouds

In recent years, Convolutional Neural Networks (CNN) have proven to be efficient analysis tools for processing point clouds, e.g., for reconstruction, segmentation, and classification. In this article, we focus on the classification of edges in point clouds, where both edges and their surrounding are described. We propose a new parameterization adding to each point a set of differential information on its surrounding shape reconstructed at different scales. These parameters, stored in a Scale-Space Matrix (SSM) , provide a well-suited information from which an adequate neural network can learn the description of edges and use it to efficiently detect them in acquired point clouds. After successfully applying a multi-scale CNN on SSMs for the efficient classification of edges and their neighborhood, we propose a new lightweight neural network architecture outperforming the CNN in learning time, processing time, and classification capabilities. Our architecture is compact, requires small learning sets, is very fast to train, and classifies millions of points in seconds.

Download Full-text

Urban Feature Extraction from Merged Airborne LiDAR Data and Digital Camera Data

Geoplanning Journal of Geomatics and Planning ◽

10.14710/geoplanning.7.2.57-74 ◽

2021 ◽

Vol 7 (2) ◽

pp. 57-74

Author(s):

Lamyaa Gamal EL-Deen Taha ◽

A. I. Ramzi ◽

A. Syarawi ◽

A. Bekheet

Keyword(s):

Neural Network ◽

Digital Camera ◽

Point Clouds ◽

Airborne Lidar ◽

Surface Model ◽

Digital Surface Model ◽

3D Point Clouds ◽

Digital Elevation ◽

Surface Models ◽

Elevation Model

Until recently, the most highly accurate digital surface models were obtained from airborne lidar. With the development of a new generation of large format digital photogrammetric aerial camera, a fully digital photogrammetric workflow became possible. Digital airborne images are sources for elevation extraction and orthophoto generation. This research concerned with the generation of digital surface models and orthophotos as applications from high-resolution images. In this research, the following steps were performed. A Benchmark data of LIDAR and digital aerial camera have been used. Firstly, image orientation, AT have been performed. Then the automatic digital surface model DSM generation has been produced from the digital aerial camera. Thirdly true digital ortho has been generated from the digital aerial camera also orthoimage will be generated using LIDAR digital elevation model (DSM). Leica Photogrammetric Suite (LPS) module of Erdsa Imagine 2014 software was utilized for processing. Then the resulted orthoimages from both techniques were mosaicked. The results show that automatic digital surface model DSM that been produced from digital aerial camera method has very high dense photogrammetric 3D point clouds compared to the LIDAR 3D point clouds. It was found that the true orthoimage produced from the second approach is better than the true orthoimage produced from the first approach. The five approaches were tested for classification of the best-orthorectified image mosaic using subpixel based (neural network) and pixel-based ( minimum distance and maximum likelihood).Multicues were extracted such as texture(entropy-mean),Digital elevation model, Digital surface model ,normalized digital surface model (nDSM) and intensity image. The contributions of the individual cues used in the classification have been evaluated. It was found that the best cue integration is intensity (pan) +nDSM+ entropy followed by intensity (pan) +nDSM+mean then intensity image +mean+ entropy after that DSM )image and two texture measures (mean and entropy) followed by the colour image. The integration with height data increases the accuracy. Also, it was found that the integration with entropy texture increases the accuracy. Resulted in fifteen cases of classification it was found that maximum likelihood classifier is the best followed by minimum distance then neural network classifier. We attribute this to the fine resolution of the digital camera image. Subpixel classifier (neural network) is not suitable for classifying aerial digital camera images.

Download Full-text

SLAM-OR: Simultaneous Localization, Mapping and Object Recognition Using Video Sensors Data in Open Environments from the Sparse Points Cloud

Sensors ◽

10.3390/s21144734 ◽

2021 ◽

Vol 21 (14) ◽

pp. 4734

Author(s):

Patryk Mazurek ◽

Tomasz Hachaj

Keyword(s):

Neural Network ◽

Deep Neural Network ◽

Clustering Algorithm ◽

State Of The Art ◽

Point Clouds ◽

Video Frame ◽

Current Frame ◽

Visual Sensors ◽

Open Environments ◽

Cloud Of Points

In this paper, we propose a novel approach that enables simultaneous localization, mapping (SLAM) and objects recognition using visual sensors data in open environments that is capable to work on sparse data point clouds. In the proposed algorithm the ORB-SLAM uses the current and previous monocular visual sensors video frame to determine observer position and to determine a cloud of points that represent objects in the environment, while the deep neural network uses the current frame to detect and recognize objects (OR). In the next step, the sparse point cloud returned from the SLAM algorithm is compared with the area recognized by the OR network. Because each point from the 3D map has its counterpart in the current frame, therefore the filtration of points matching the area recognized by the OR algorithm is performed. The clustering algorithm determines areas in which points are densely distributed in order to detect spatial positions of objects detected by OR. Then by using principal component analysis (PCA)—based heuristic we estimate bounding boxes of detected objects. The image processing pipeline that uses sparse point clouds generated by SLAM in order to determine positions of objects recognized by deep neural network and mentioned PCA heuristic are main novelties of our solution. In contrary to state-of-the-art approaches, our algorithm does not require any additional calculations like generation of dense point clouds for objects positioning, which highly simplifies the task. We have evaluated our research on large benchmark dataset using various state-of-the-art OR architectures (YOLO, MobileNet, RetinaNet) and clustering algorithms (DBSCAN and OPTICS) obtaining promising results. Both our source codes and evaluation data sets are available for download, so our results can be easily reproduced.

Download Full-text

Mapping Forest Vertical Structure in Sogwang-ri Forest from Full-Waveform Lidar Point Clouds Using Deep Neural Network

Remote Sensing ◽

10.3390/rs13183736 ◽

2021 ◽

Vol 13 (18) ◽

pp. 3736

Author(s):

Sung-Hwan Park ◽

Hyung-Sup Jung ◽

Sunmin Lee ◽

Eun-Sook Kim

Keyword(s):

Neural Network ◽

Deep Neural Network ◽

Vertical Structure ◽

Point Clouds ◽

Canopy Height ◽

Accurate Information ◽

Lidar Data ◽

Full Waveform ◽

Density Maps ◽

Point Density

The role of forests is increasing because of rapid land use changes worldwide that have implications on ecosystems and the carbon cycle. Therefore, it is necessary to obtain accurate information about forests and build forest inventories. However, it is difficult to assess the internal structure of the forest through 2D remote sensing techniques and fieldwork. In this aspect, we proposed a method for estimating the vertical structure of forests based on full-waveform light detection and ranging (FW LiDAR) data in this study. Voxel-based tree point density maps were generated by estimating the number of canopy height points in each voxel grid from the raster digital terrain model (DTM) and canopy height points after pre-processing the LiDAR point clouds. We applied an unsupervised classification algorithm to the voxel-based tree point density maps and identified seven classes by profile pattern analysis for the forest vertical types. The classification accuracy was found to be 72.73% from the validation from 11 field investigation sites, which was additionally confirmed through comparative analysis with aerial images. Based on this pre-classification reference map, which is assumed to be ground truths, the deep neural network (DNN) model was finally applied to perform the final classification. As a result of accuracy assessment, it showed accuracy of 92.72% with a good performance. These results demonstrate the potential of vertical structure estimation for extensive forests using FW LiDAR data and that the distinction between one-storied and two-storied forests can be clearly represented. This technique is expected to contribute to efficient and effective management of forests based on accurate information derived from the proposed method.

Download Full-text

Convolutional neural network for 3D point clouds matching

Applications of Digital Image Processing XLIV ◽

10.1117/12.2594573 ◽

2021 ◽

Author(s):

Sergei Voronin ◽

Artyom Makovetskii ◽

Aleksei Voronin ◽

Dmitrii Zhernov

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Point Clouds ◽

3D Point Clouds

Download Full-text

Local k-NNs pattern in Omni-Direction graph convolution neural network for 3D point clouds

Neurocomputing ◽

10.1016/j.neucom.2020.06.095 ◽

2020 ◽

Vol 413 ◽

pp. 487-498

Author(s):

Wenjing Zhang ◽

Songzhi Su ◽

Beizhan Wang ◽

Qingqi Hong ◽

Li Sun

Keyword(s):

Neural Network ◽

Point Clouds ◽

Convolution Neural Network ◽

3D Point Clouds

Download Full-text

Semantic Point Cloud Segmentation Using Fast Deep Neural Network and DCRF

Sensors ◽

10.3390/s21082731 ◽

2021 ◽

Vol 21 (8) ◽

pp. 2731

Author(s):

Yunbo Rao ◽

Menghan Zhang ◽

Zhanglin Cheng ◽

Junmin Xue ◽

Jiansu Pu ◽

...

Keyword(s):

Neural Network ◽

Point Cloud ◽

Deep Neural Network ◽

Conditional Random Field ◽

Semantic Segmentation ◽

Point Clouds ◽

Original Data ◽

Point Cloud Segmentation ◽

Semantic Point ◽

3D Scene

Accurate segmentation of entity categories is the critical step for 3D scene understanding. This paper presents a fast deep neural network model with Dense Conditional Random Field (DCRF) as a post-processing method, which can perform accurate semantic segmentation for 3D point cloud scene. On this basis, a compact but flexible framework is introduced for performing segmentation to the semantics of point clouds concurrently, contribute to more precise segmentation. Moreover, based on semantics labels, a novel DCRF model is elaborated to refine the result of segmentation. Besides, without any sacrifice to accuracy, we apply optimization to the original data of the point cloud, allowing the network to handle fewer data. In the experiment, our proposed method is conducted comprehensively through four evaluation indicators, proving the superiority of our method.

Download Full-text