Detection of Surface Defects in Logs Using Point Cloud Data and Deep Learning

Deep learning classification based on 3D point clouds has gained considerable research interest in recent years.The classification and quantitative analysis of wood defects are of great significance to the wood processing industry. In order to solve the problems of slow processing and low robustness of 3D data. This paper proposes an improvement based on littlepoint CNN lightweight deep learning network, adding BN layer. And based on the data set made by ourselves, the test is carried out. The new network bnlittlepoint CNN has been improved in speed and recognition rate. The correct rate of recognition for non defect log, non defect log and defect log as well as defect knot and dead knot can reach 95.6%.Finally, the "dead knot" and "loose knot" are quantitatively analyzed based on the "integral" idea, and the volume and surface area of the defect are obtained to a certain extent,the error is not more than 1.5% and the defect surface reconstruction is completed based on the triangulation idea.

Download Full-text

Parsing of Urban Facades from 3D Point Clouds Based on a Novel Multi-View Domain

Photogrammetric Engineering & Remote Sensing ◽

10.14358/pers.87.4.283 ◽

2021 ◽

Vol 87 (4) ◽

pp. 283-293

Author(s):

Wei Wang ◽

Yuan Xu ◽

Yingchao Ren ◽

Gang Wang

Keyword(s):

Deep Learning ◽

Prior Knowledge ◽

Performance Improvement ◽

Data Distribution ◽

Point Clouds ◽

Learning Models ◽

Data Set ◽

3D Point Clouds ◽

Segmentation Accuracy ◽

The Mean

Recently, performance improvement in facade parsing from 3D point clouds has been brought about by designing more complex network structures, which cost huge computing resources and do not take full advantage of prior knowledge of facade structure. Instead, from the perspective of data distribution, we construct a new hierarchical mesh multi-view data domain based on the characteristics of facade objects to achieve fusion of deep-learning models and prior knowledge, thereby significantly improving segmentation accuracy. We comprehensively evaluate the current mainstream method on the RueMonge 2014 data set and demonstrate the superiority of our method. The mean intersection-over-union index on the facade-parsing task reached 76.41%, which is 2.75% higher than the current best result. In addition, through comparative experiments, the reasons for the performance improvement of the proposed method are further analyzed.

Download Full-text

A 3D Shape Recognition Method Using Hybrid Deep Learning Network CNN–SVM

Electronics ◽

10.3390/electronics9040649 ◽

2020 ◽

Vol 9 (4) ◽

pp. 649

Author(s):

Long Hoang ◽

Suk-Hwan Lee ◽

Ki-Ryong Kwon

Keyword(s):

Feature Extraction ◽

Deep Learning ◽

3D Model ◽

Shape Recognition ◽

Point Clouds ◽

Support Vector ◽

Learning Network ◽

3D Shape ◽

3D Data ◽

Deep Learning Network

3D shape recognition becomes necessary due to the popularity of 3D data resources. This paper aims to introduce the new method, hybrid deep learning network convolution neural network–support vector machine (CNN–SVM), for 3D recognition. The vertices of the 3D mesh are interpolated to be converted into Point Clouds; those Point Clouds are rotated for 3D data augmentation. We obtain and store the 2D projection of this 3D augmentation data in a 32 × 32 × 12 matrix, the input data of CNN–SVM. An eight-layer CNN is used as the algorithm for feature extraction, then SVM is applied for classifying feature extraction. Two big datasets, ModelNet40 and ModelNet10, of the 3D model are used for model validation. Based on our numerical experimental results, CNN–SVM is more accurate and efficient than other methods. The proposed method is 13.48% more accurate than the PointNet method in ModelNet10 and 8.5% more precise than 3D ShapeNets for ModelNet40. The proposed method works with both the 3D model in the augmented/virtual reality system and in the 3D Point Clouds, an output of the LIDAR sensor in autonomously driving cars.

Download Full-text

Review: Deep Learning on 3D Point Clouds

Remote Sensing ◽

10.3390/rs12111729 ◽

2020 ◽

Vol 12 (11) ◽

pp. 1729 ◽

Cited By ~ 4

Author(s):

Saifullahi Aminu Bello ◽

Shangshu Yu ◽

Cheng Wang ◽

Jibril Muhmmad Adam ◽

Jonathan Li

Keyword(s):

Deep Learning ◽

Point Cloud ◽

General Structure ◽

Point Clouds ◽

Autonomous Driving ◽

Point Cloud Data ◽

3D Vision ◽

Cloud Data ◽

3D Point Clouds ◽

Learning Techniques

A point cloud is a set of points defined in a 3D metric space. Point clouds have become one of the most significant data formats for 3D representation and are gaining increased popularity as a result of the increased availability of acquisition devices, as well as seeing increased application in areas such as robotics, autonomous driving, and augmented and virtual reality. Deep learning is now the most powerful tool for data processing in computer vision and is becoming the most preferred technique for tasks such as classification, segmentation, and detection. While deep learning techniques are mainly applied to data with a structured grid, the point cloud, on the other hand, is unstructured. The unstructuredness of point clouds makes the use of deep learning for its direct processing very challenging. This paper contains a review of the recent state-of-the-art deep learning techniques, mainly focusing on raw point cloud data. The initial work on deep learning directly with raw point cloud data did not model local regions; therefore, subsequent approaches model local regions through sampling and grouping. More recently, several approaches have been proposed that not only model the local regions but also explore the correlation between points in the local regions. From the survey, we conclude that approaches that model local regions and take into account the correlation between points in the local regions perform better. Contrary to existing reviews, this paper provides a general structure for learning with raw point clouds, and various methods were compared based on the general structure. This work also introduces the popular 3D point cloud benchmark datasets and discusses the application of deep learning in popular 3D vision tasks, including classification, segmentation, and detection.

Download Full-text

DETERMINATION OF PARKING SPACE AND ITS CONCURRENT USAGE OVER TIME USING SEMANTICALLY SEGMENTED MOBILE MAPPING DATA

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xliii-b2-2021-185-2021 ◽

2021 ◽

Vol XLIII-B2-2021 ◽

pp. 185-192

Author(s):

A. Leichter ◽

U. Feuerhake ◽

M. Sester

Keyword(s):

Public Space ◽

Point Cloud ◽

Point Clouds ◽

Parking Space ◽

3D Point Cloud ◽

Cloud Data ◽

3D Point Clouds ◽

3D Data ◽

Rgb Images ◽

Bounding Boxes

Abstract. Public space is a scarce good in cities. There are many concurrent usages, which makes an adequate allocation of space both difficult and highly attractive. A lot of space is allocated by parking cars – even if the parking spaces are not occupied by cars all the time. In this work, we analyze space demand and usage by parking cars, in order to evaluate, when this space could be used for other purposes. The analysis is based on 3D point clouds acquired at several times during a day. We propose a processing pipeline to extract car bounding boxes from a given 3D point cloud. For the car extraction we utilize a label transfer technique for transfers from semantically segmented 2D RGB images to 3D point cloud data. This semantically segmented 3D data allows us to identify car instances. Subsequently, we aggregate and analyze information about parking cars. We present an exemplary analysis of the urban area where we extracted 15.000 cars at five different points in time. Based on this aggregated we present analytical results for time dependent parking behavior, parking space availability and utilization.

Download Full-text

BIM Supported Surveying and Imaging Combination for Heritage Conservation

Remote Sensing ◽

10.3390/rs13081584 ◽

2021 ◽

Vol 13 (8) ◽

pp. 1584

Author(s):

Pedro Martín-Lerones ◽

David Olmedo ◽

Ana López-Vidal ◽

Jaime Gómez-García-Bermejo ◽

Eduardo Zalama

Keyword(s):

Laser Scanning ◽

Point Clouds ◽

Cloud Data ◽

Industry Foundation Classes ◽

3D Point Clouds ◽

Conservation Actions ◽

Heritage Building ◽

2D Imaging ◽

Digital Survey ◽

Level 2

As the basis for analysis and management of heritage assets, 3D laser scanning and photogrammetric 3D reconstruction have been probed as adequate techniques for point cloud data acquisition. The European Directive 2014/24/EU imposes BIM Level 2 for government centrally procured projects as a collaborative process of producing federated discipline-specific models. Although BIM software resources are intensified and increasingly growing, distinct specifications for heritage (H-BIM) are essential to driving particular processes and tools to efficiency shifting from point clouds to meaningful information ready to be exchanged using non-proprietary formats, such as Industry Foundation Classes (IFC). This paper details a procedure for processing enriched 3D point clouds into the REVIT software package due to its worldwide popularity and how closely it integrates with the BIM concept. The procedure will be additionally supported by a tailored plug-in to make high-quality 3D digital survey datasets usable together with 2D imaging, enhancing the capability to depict contextualized important graphical data to properly planning conservation actions. As a practical example, a 2D/3D enhanced combination is worked to accurately include into a BIM project, the length, orientation, and width of a big crack on the walls of the Castle of Torrelobatón (Spain) as a representative heritage building.

Download Full-text

Multi-Dimensional Underwater Point Cloud Detection Based on Deep Learning

Sensors ◽

10.3390/s21030884 ◽

2021 ◽

Vol 21 (3) ◽

pp. 884

Author(s):

Chia-Ming Tsai ◽

Yi-Horng Lai ◽

Yung-Da Sun ◽

Yu-Jen Chung ◽

Jau-Woei Perng

Keyword(s):

Deep Learning ◽

Point Cloud ◽

Three Dimensional ◽

Point Clouds ◽

Training Data ◽

Network Architectures ◽

Point Cloud Data ◽

Data Types ◽

Raw Data ◽

Cloud Data

Numerous sensors can obtain images or point cloud data on land, however, the rapid attenuation of electromagnetic signals and the lack of light in water have been observed to restrict sensing functions. This study expands the utilization of two- and three-dimensional detection technologies in underwater applications to detect abandoned tires. A three-dimensional acoustic sensor, the BV5000, is used in this study to collect underwater point cloud data. Some pre-processing steps are proposed to remove noise and the seabed from raw data. Point clouds are then processed to obtain two data types: a 2D image and a 3D point cloud. Deep learning methods with different dimensions are used to train the models. In the two-dimensional method, the point cloud is transferred into a bird’s eye view image. The Faster R-CNN and YOLOv3 network architectures are used to detect tires. Meanwhile, in the three-dimensional method, the point cloud associated with a tire is cut out from the raw data and is used as training data. The PointNet and PointConv network architectures are then used for tire classification. The results show that both approaches provide good accuracy.

Download Full-text

Comparison of Depth Camera and Terrestrial Laser Scanner in Monitoring Structural Deflections

Sensors ◽

10.3390/s21010201 ◽

2020 ◽

Vol 21 (1) ◽

pp. 201

Author(s):

Michael Bekele Maru ◽

Donghwan Lee ◽

Kassahun Demissie Tola ◽

Seunghee Park

Keyword(s):

Optical Sensors ◽

Point Cloud ◽

Three Dimensional ◽

Laser Scanner ◽

Point Clouds ◽

Depth Camera ◽

Terrestrial Laser Scanner ◽

Cloud Data ◽

3D Point Clouds ◽

3D Information

Modeling a structure in the virtual world using three-dimensional (3D) information enhances our understanding, while also aiding in the visualization, of how a structure reacts to any disturbance. Generally, 3D point clouds are used for determining structural behavioral changes. Light detection and ranging (LiDAR) is one of the crucial ways by which a 3D point cloud dataset can be generated. Additionally, 3D cameras are commonly used to develop a point cloud containing many points on the external surface of an object around it. The main objective of this study was to compare the performance of optical sensors, namely a depth camera (DC) and terrestrial laser scanner (TLS) in estimating structural deflection. We also utilized bilateral filtering techniques, which are commonly used in image processing, on the point cloud data for enhancing their accuracy and increasing the application prospects of these sensors in structure health monitoring. The results from these sensors were validated by comparing them with the outputs from a linear variable differential transformer sensor, which was mounted on the beam during an indoor experiment. The results showed that the datasets obtained from both the sensors were acceptable for nominal deflections of 3 mm and above because the error range was less than ±10%. However, the result obtained from the TLS were better than those obtained from the DC.

Download Full-text

Pedestrian detection algorithm in traffic scene based on weakly supervised hierarchical deep model

International Journal of Advanced Robotic Systems ◽

10.1177/1729881417692311 ◽

2016 ◽

Vol 14 (1) ◽

pp. 172988141769231 ◽

Cited By ~ 2

Author(s):

Yingfeng Cai ◽

Youguo He ◽

Hai Wang ◽

Xiaoqiang Sun ◽

Long Chen ◽

...

Keyword(s):

Deep Learning ◽

Pedestrian Detection ◽

Recognition Rate ◽

Detection Algorithm ◽

Training Methods ◽

Two Dimensional ◽

Data Set ◽

Deep Model ◽

Unsupervised Training ◽

Weakly Supervised

The emergence and development of deep learning theory in machine learning field provide new method for visual-based pedestrian recognition technology. To achieve better performance in this application, an improved weakly supervised hierarchical deep learning pedestrian recognition algorithm with two-dimensional deep belief networks is proposed. The improvements are made by taking into consideration the weaknesses of structure and training methods of existing classifiers. First, traditional one-dimensional deep belief network is expanded to two-dimensional that allows image matrix to be loaded directly to preserve more information of a sample space. Then, a determination regularization term with small weight is added to the traditional unsupervised training objective function. By this modification, original unsupervised training is transformed to weakly supervised training. Subsequently, that gives the extracted features discrimination ability. Multiple sets of comparative experiments show that the performance of the proposed algorithm is better than other deep learning algorithms in recognition rate and outperforms most of the existing state-of-the-art methods in non-occlusion pedestrian data set while performs fair in weakly and heavily occlusion data set.

Download Full-text

Fast and Robust Flight Altitude Estimation of Multirotor UAVs in Dynamic Unstructured Environments Using 3D Point Cloud Sensors

Aerospace ◽

10.3390/aerospace5030094 ◽

2018 ◽

Vol 5 (3) ◽

pp. 94 ◽

Cited By ~ 5

Author(s):

Hriday Bavle ◽

Jose Sanchez-Lopez ◽

Paloma Puente ◽

Alejandro Rodriguez-Ramos ◽

Carlos Sampedro ◽

...

Keyword(s):

Point Cloud ◽

Point Clouds ◽

Estimation Algorithm ◽

Indoor Environments ◽

3D Point Cloud ◽

Flight Altitude ◽

Cloud Data ◽

3D Point Clouds ◽

Laser Altimeters ◽

Altitude Estimation

This paper presents a fast and robust approach for estimating the flight altitude of multirotor Unmanned Aerial Vehicles (UAVs) using 3D point cloud sensors in cluttered, unstructured, and dynamic indoor environments. The objective is to present a flight altitude estimation algorithm, replacing the conventional sensors such as laser altimeters, barometers, or accelerometers, which have several limitations when used individually. Our proposed algorithm includes two stages: in the first stage, a fast clustering of the measured 3D point cloud data is performed, along with the segmentation of the clustered data into horizontal planes. In the second stage, these segmented horizontal planes are mapped based on the vertical distance with respect to the point cloud sensor frame of reference, in order to provide a robust flight altitude estimation even in presence of several static as well as dynamic ground obstacles. We validate our approach using the IROS 2011 Kinect dataset available in the literature, estimating the altitude of the RGB-D camera using the provided 3D point clouds. We further validate our approach using a point cloud sensor on board a UAV, by means of several autonomous real flights, closing its altitude control loop using the flight altitude estimated by our proposed method, in presence of several different static as well as dynamic ground obstacles. In addition, the implementation of our approach has been integrated in our open-source software framework for aerial robotics called Aerostack.

Download Full-text

3DLEB-Net: Label-Efficient Deep Learning-Based Semantic Segmentation of Building Point Clouds at LoD3 Level

Applied Sciences ◽

10.3390/app11198996 ◽

2021 ◽

Vol 11 (19) ◽

pp. 8996

Author(s):

Yuwei Cao ◽

Marco Scaioni

Keyword(s):

Deep Learning ◽

Semantic Segmentation ◽

Point Clouds ◽

Training Data ◽

Second Step ◽

Point Cloud Data ◽

Dynamic Graph ◽

Cloud Data ◽

Supervised Methods ◽

Global And Local

In current research, fully supervised Deep Learning (DL) techniques are employed to train a segmentation network to be applied to point clouds of buildings. However, training such networks requires large amounts of fine-labeled buildings’ point-cloud data, presenting a major challenge in practice because they are difficult to obtain. Consequently, the application of fully supervised DL for semantic segmentation of buildings’ point clouds at LoD3 level is severely limited. In order to reduce the number of required annotated labels, we proposed a novel label-efficient DL network that obtains per-point semantic labels of LoD3 buildings’ point clouds with limited supervision, named 3DLEB-Net. In general, it consists of two steps. The first step (Autoencoder, AE) is composed of a Dynamic Graph Convolutional Neural Network (DGCNN) encoder and a folding-based decoder. It is designed to extract discriminative global and local features from input point clouds by faithfully reconstructing them without any label. The second step is the semantic segmentation network. By supplying a small amount of task-specific supervision, a segmentation network is proposed for semantically segmenting the encoded features acquired from the pre-trained AE. Experimentally, we evaluated our approach based on the Architectural Cultural Heritage (ArCH) dataset. Compared to the fully supervised DL methods, we found that our model achieved state-of-the-art results on the unseen scenes, with only 10% of labeled training data from fully supervised methods as input. Moreover, we conducted a series of ablation studies to show the effectiveness of the design choices of our model.

Download Full-text