scholarly journals FUSED 3D TRANSPARENT VISUALIZATION FOR LARGE-SCALE CULTURAL HERITAGE USING DEEP LEARNING-BASED MONOCULAR RECONSTRUCTION

2020 ◽  
Vol V-2-2020 ◽  
pp. 989-996
Author(s):  
J. Pan ◽  
L. Li ◽  
H. Yamaguchi ◽  
K. Hasegawa ◽  
F. I. Thufail ◽  
...  
Keyword(s):  
Deep Learning ◽  
3D Reconstruction ◽  
Cultural Heritage ◽  
Point Cloud ◽  
Large Scale ◽  
Reconstruction Method ◽  
Visualization Method ◽  
Point Cloud Data ◽  
Complete Collection ◽  
Cloud Data

Abstract. This paper proposes a fused 3D transparent visualization method with the aim of achieving see-through imaging of large-scale cultural heritage by combining photogrammetry point cloud data and 3D reconstructed models. 3D reconstructed models are efficiently reconstructed from a single monocular photo using deep learning. It is demonstrated that the proposed method can be widely applied, particularly to instances of incomplete cultural heritages. In this study, the proposed method is applied to a typical example, the Borobudur temple in Indonesia. The Borobudur temple possesses the most complete collection of Buddhist reliefs. However, some parts of the Borobudur reliefs have been hidden by stone walls and became not visible following the reinforcements during the Dutch rule. Today, only gray-scale monocular photos of those hidden parts are displayed in the Borobudur Museum. In this paper, the visible parts of the temple are first digitized into point cloud data by photogrammetry scanning. For the hidden parts, a 3D reconstruction method based on deep learning is proposed to reconstruct the invisible parts into point cloud data directly from single monocular photos from the museum. The proposed 3D reconstruction method achieves 95% accuracy of the reconstructed point cloud on average. With the point cloud data of both the visible parts and the hidden parts, the proposed transparent visualization method called the stochastic point-based rendering is applied to achieve a fused 3D transparent visualization of the valuable temple.

scholarly journals Fused Transparent Visualization of Point Cloud Data and Background Photographic Image for Tangible Cultural Heritage Assets

2019 ◽  
Vol 8 (8) ◽  
pp. 343 ◽  
Author(s):  
Li ◽  
Hasegawa ◽  
Nii ◽  
Tanaka
Keyword(s):  
Cultural Heritage ◽  
Point Cloud ◽  
Laser Scanning ◽  
Large Scale ◽  
Three Dimensional ◽  
Point Clouds ◽  
Point Cloud Data ◽  
Cloud Data ◽  
Photographic Image ◽  
Scale Point

Digital archiving of three-dimensional cultural heritage assets has increased the demand for visualization of large-scale point clouds of cultural heritage assets acquired by laser scanning. We proposed a fused transparent visualization method that visualizes a point cloud of a cultural heritage asset in an environment using a photographic image as the background. We also proposed lightness adjustment and color enhancement methods to deal with the reduced visibility caused by the fused visualization. We applied the proposed method to a laser-scanned point cloud of a high-valued cultural festival float with complex inner and outer structures. Experimental results demonstrate that the proposed method enables high-quality transparent visualization of the cultural asset in its surrounding environment.

scholarly journals Multi-Dimensional Underwater Point Cloud Detection Based on Deep Learning

Sensors ◽  
2021 ◽  
Vol 21 (3) ◽  
pp. 884
Author(s):  
Chia-Ming Tsai ◽  
Yi-Horng Lai ◽  
Yung-Da Sun ◽  
Yu-Jen Chung ◽  
Jau-Woei Perng
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Three Dimensional ◽  
Point Clouds ◽  
Training Data ◽  
Network Architectures ◽  
Point Cloud Data ◽  
Data Types ◽  
Raw Data ◽  
Cloud Data

Numerous sensors can obtain images or point cloud data on land, however, the rapid attenuation of electromagnetic signals and the lack of light in water have been observed to restrict sensing functions. This study expands the utilization of two- and three-dimensional detection technologies in underwater applications to detect abandoned tires. A three-dimensional acoustic sensor, the BV5000, is used in this study to collect underwater point cloud data. Some pre-processing steps are proposed to remove noise and the seabed from raw data. Point clouds are then processed to obtain two data types: a 2D image and a 3D point cloud. Deep learning methods with different dimensions are used to train the models. In the two-dimensional method, the point cloud is transferred into a bird’s eye view image. The Faster R-CNN and YOLOv3 network architectures are used to detect tires. Meanwhile, in the three-dimensional method, the point cloud associated with a tire is cut out from the raw data and is used as training data. The PointNet and PointConv network architectures are then used for tire classification. The results show that both approaches provide good accuracy.

scholarly journals LiDAR Point Cloud Recognition and Visualization with Deep Learning for Overhead Contact Inspection

Sensors ◽  
2020 ◽  
Vol 20 (21) ◽  
pp. 6387 ◽  
Author(s):  
Xiaohan Tu ◽  
Cheng Xu ◽  
Siping Liu ◽  
Shuai Lin ◽  
Lipei Chen ◽  
...  
Keyword(s):  
Deep Learning ◽  
Computational Complexity ◽  
Point Cloud ◽  
High Speed ◽  
Point Cloud Data ◽  
Embedded Devices ◽  
Cloud Data ◽  
Manual Inspection ◽  
Order Of Magnitude ◽  
Supply Systems

As overhead contact (OC) is an essential part of power supply systems in high-speed railways, it is necessary to regularly inspect and repair abnormal OC components. Relative to manual inspection, applying LiDAR (light detection and ranging) to OC inspection can improve efficiency, accuracy, and safety, but it faces challenges to efficiently and effectively segment LiDAR point cloud data and identify catenary components. Recent deep learning-based recognition methods are rarely employed to recognize OC components, because they have high computational complexity, while their accuracy needs to be improved. To track these problems, we first propose a lightweight model, RobotNet, with depthwise and pointwise convolutions and an attention module to recognize the point cloud. Second, we optimize RobotNet to accelerate its recognition speed on embedded devices using an existing compilation tool. Third, we design software to facilitate the visualization of point cloud data. Our software can not only display a large amount of point cloud data, but also visualize the details of OC components. Extensive experiments demonstrate that RobotNet recognizes OC components more accurately and efficiently than others. The inference speed of the optimized RobotNet increases by an order of magnitude. RobotNet has lower computational complexity than other studies. The visualization results also show that our recognition method is effective.

scholarly journals A Deep Learning-Based Perception Algorithm Using 3D LiDAR for Autonomous Driving: Simultaneous Segmentation and Detection Network (SSADNet)

2020 ◽  
Vol 10 (13) ◽  
pp. 4486 ◽  
Author(s):  
Yongbeom Lee ◽  
Seongkeun Park
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Autonomous Driving ◽  
Light Detection And Ranging ◽  
Point Cloud Data ◽  
Cloud Data ◽  
3D Lidar ◽  
Top View ◽  
Single Neural Network ◽  
Reflection Intensity

In this paper, we propose a deep learning-based perception method in autonomous driving systems using a Light Detection and Ranging(LiDAR) point cloud data, which is called a simultaneous segmentation and detection network (SSADNet). SSADNet can be used to recognize both drivable areas and obstacles, which is necessary for autonomous driving. Unlike the previous methods, where separate networks were needed for segmentation and detection, SSADNet can perform segmentation and detection simultaneously based on a single neural network. The proposed method uses point cloud data obtained from a 3D LiDAR for network input to generate a top view image consisting of three channels of distance, height, and reflection intensity. The structure of the proposed network includes a branch for segmentation and a branch for detection as well as a bridge connecting the two parts. The KITTI dataset, which is often used for experiments on autonomous driving, was used for training. The experimental results show that segmentation and detection can be performed simultaneously for drivable areas and vehicles at a quick inference speed, which is appropriate for autonomous driving systems.

scholarly journals TAILORED FEATURES FOR SEMANTIC SEGMENTATION WITH A DGCNN USING FREE TRAINING SAMPLES OF A COLORED AIRBORNE POINT CLOUD

2020 ◽  
Vol XLIII-B2-2020 ◽  
pp. 339-346
Author(s):  
E. Widyaningrum ◽  
M. K. Fajari ◽  
R. C. Lindenbergh ◽  
M. Hahn
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Semantic Segmentation ◽  
Blue Color ◽  
Point Cloud Data ◽  
Dynamic Graph ◽  
Cloud Data ◽  
Training Samples ◽  
Map Generation ◽  
Fast Development

Abstract. Automation of 3D LiDAR point cloud processing is expected to increase the production rate of many applications including automatic map generation. Fast development on high-end hardware has boosted the expansion of deep learning research for 3D classification and segmentation. However, deep learning requires large amount of high quality training samples. The generation of training samples for accurate classification results, especially for airborne point cloud data, is still problematic. Moreover, which customized features should be used best for segmenting airborne point cloud data is still unclear. This paper proposes semi-automatic point cloud labelling and examines the potential of combining different tailor-made features for pointwise semantic segmentation of an airborne point cloud. We implement a Dynamic Graph CNN (DGCNN) approach to classify airborne point cloud data into four land cover classes: bare-land, trees, buildings and roads. The DGCNN architecture is chosen as this network relates two approaches, PointNet and graph CNNs, to exploit the geometric relationships between points. For experiments, we train an airborne point cloud and co-aligned orthophoto of the Surabaya city area of Indonesia to DGCNN using three different tailor-made feature combinations: points with RGB (Red, Green, Blue) color, points with original LiDAR features (Intensity, Return number, Number of returns) so-called IRN, and points with two spectral colors and Intensity (Red, Green, Intensity) so-called RGI. The overall accuracy of the testing area indicates that using RGB information gives the best segmentation results of 81.05% while IRN and RGI gives accuracy values of 76.13%, and 79.81%, respectively.

scholarly journals 3D COMPARISON TOWARDS A COMPREHENSIVE ANALYSIS OF A BUILDING IN CULTURAL HERITAGE

2018 ◽  
Vol XLII-2 ◽  
pp. 1061-1066 ◽  
Author(s):  
I. Selvaggi ◽  
M. Dellapasqua ◽  
F. Franci ◽  
A. Spangher ◽  
D. Visintini ◽  
...  
Keyword(s):  
Cultural Heritage ◽  
Point Cloud ◽  
Laser Scanning ◽  
Three Dimensional ◽  
Deformation Analysis ◽  
Point Cloud Data ◽  
Cloud Data ◽  
Site Restoration ◽  
Close Range ◽  
Remote Sensing Techniques

Terrestrial remote sensing techniques, including both Terrestrial Laser Scanning (TLS) and Close-Range Photogrammetry (CRP), have been recently used in multiple applications and projects with particular reference to the documentation/inspection of a wide variety of Cultural Heritage structures.<br> The high density of TLS point cloud data allows to perform structure survey in an unprecedented level of detail, providing a direct solution for the digital three-dimensional modelling, the site restoration and the analysis of the structural conditions. Textural information provided by CRP can be used for the photorealistic representation of the surveyed structure. With respect to many studies, the combination of TLS and CRP techniques produces the best results for Cultural Heritage documentation purposes. Moreover, TLS and CRP point cloud data have been proved to be useful in the field of deformation analysis and structural health monitoring. They can be the input data for the Finite Element Method (FEM), providing some prior knowledge concerning the material and the boundary conditions such as constraints and loading.<br> The paper investigates the capabilities and advantages of TLS and CRP data integration for the three-dimensional modelling compared to a simplified geometric reconstruction. This work presents some results concerning the Baptistery of Aquileia in Italy, characterized by an octagonal plan and walls composed by masonry stones with good texture.

Three-Dimensional Reconstruction of Maize Plants and Extraction of Phenotypic Parameters Based On Laser Point Cloud Data

2021 ◽  
Author(s):  
Chengxin Ju ◽  
Yuanyuan Zhao ◽  
Fengfeng Wu ◽  
Rui Li ◽  
Tianle Yang ◽  
...  
Keyword(s):  
3D Reconstruction ◽  
Plant Height ◽  
Point Cloud ◽  
Agricultural Research ◽  
Three Dimensional ◽  
Laser Scanner ◽  
Point Cloud Data ◽  
Scanning Method ◽  
Cloud Data ◽  
Maize Plants

Abstract Background: Three-dimensional (3D) laser scanning technology could rapidly extract the surface geometric features of maize plants to achieve non-destructive monitoring of maize phenotypes. However, extracting the phenotypic parameters of maize plants based on laser point cloud data is challenging.Methods: In this paper, a rotational scanning method was used to collect the data of potted maize point cloud from different perspectives by using a laser scanner. Maize point cloud data were grid-reconstructed and aligned based on greedy projection triangulation algorithm and iterative closest point (ICP) algorithm, and the random sampling consistency algorithm was used to segment the stem and leaf point clouds of single maize plant to obtain the plant height and leaf parameters.Results: The results showed that the R2 between the predicted plant height and the measured plant height was above 0.95, and the R2 of the predicted leaf length, leaf width and leaf area were 0.938, 0878 and 0.956 respectively when compared with the measured values.Conclusions: The 3D reconstruction of maize plants using the laser scanner showed a good performance, and the phenotypic parameters obtained based on the reconstructed 3D model had high accuracy. The results were helpful to the practical application of plant 3D reconstruction and provided guidance for plant parameter acquisition and theoretical methods for intelligent agricultural research.

Sign in / Sign up

Export Citation Format

Share Document