scholarly journals Semantic 3D Reconstruction with Learning MVS and 2D Segmentation of Aerial Images

2020 ◽  
Vol 10 (4) ◽  
pp. 1275
Author(s):  
Zizhuang Wei ◽  
Yao Wang ◽  
Hongwei Yi ◽  
Yisong Chen ◽  
Guoping Wang
Keyword(s):  
Point Cloud ◽  
Large Scale ◽  
Point Clouds ◽  
Aerial Images ◽  
Experimental Result ◽  
Fusion Procedure ◽  
3D Geometry ◽  
Dense Point ◽  
Semantic Point ◽  
Projection Error

Semantic modeling is a challenging task that has received widespread attention in recent years. With the help of mini Unmanned Aerial Vehicles (UAVs), multi-view high-resolution aerial images of large-scale scenes can be conveniently collected. In this paper, we propose a semantic Multi-View Stereo (MVS) method to reconstruct 3D semantic models from 2D images. Firstly, 2D semantic probability distribution is obtained by Convolutional Neural Network (CNN). Secondly, the calibrated cameras poses are determined by Structure from Motion (SfM), while the depth maps are estimated by learning MVS. Combining 2D segmentation and 3D geometry information, dense point clouds with semantic labels are generated by a probability-based semantic fusion method. In the final stage, the coarse 3D semantic point cloud is optimized by both local and global refinements. By making full use of the multi-view consistency, the proposed method efficiently produces a fine-level 3D semantic point cloud. The experimental result evaluated by re-projection maps achieves 88.4% Pixel Accuracy on the Urban Drone Dataset (UDD). In conclusion, our graph-based semantic fusion procedure and refinement based on local and global information can suppress and reduce the re-projection error.

scholarly journals ANALYSIS OF 3D BUILDING MODELS ACCURACY BASED ON THE AIRBORNE LASER SCANNING POINT CLOUDS

2018 ◽  
Vol XLII-2 ◽  
pp. 797-804 ◽  
Author(s):  
W. Ostrowski ◽  
M. Pilarska ◽  
J. Charyton ◽  
K. Bakuła
Keyword(s):  
Point Cloud ◽  
Laser Scanning ◽  
Large Scale ◽  
Point Clouds ◽  
3D Models ◽  
Airborne Laser Scanning ◽  
Aerial Images ◽  
Statistical Parameters ◽  
Airborne Laser ◽  
Building Models

Creating 3D building models in large scale is becoming more popular and finds many applications. Nowadays, a wide term “3D building models” can be applied to several types of products: well-known CityGML solid models (available on few Levels of Detail), which are mainly generated from Airborne Laser Scanning (ALS) data, as well as 3D mesh models that can be created from both nadir and oblique aerial images. City authorities and national mapping agencies are interested in obtaining the 3D building models. Apart from the completeness of the models, the accuracy aspect is also important. Final accuracy of a building model depends on various factors (accuracy of the source data, complexity of the roof shapes, etc.). In this paper the methodology of inspection of dataset containing 3D models is presented. The proposed approach check all building in dataset with comparison to ALS point clouds testing both: accuracy and level of details. Using analysis of statistical parameters for normal heights for reference point cloud and tested planes and segmentation of point cloud provides the tool that can indicate which building and which roof plane in do not fulfill requirement of model accuracy and detail correctness. Proposed method was tested on two datasets: solid and mesh model.

scholarly journals TUM-MLS-2016: An Annotated Mobile LiDAR Dataset of the TUM City Campus for Semantic Point Cloud Interpretation in Urban Areas

2020 ◽  
Vol 12 (11) ◽  
pp. 1875 ◽  
Author(s):  
Jingwei Zhu ◽  
Joachim Gehrung ◽  
Rong Huang ◽  
Björn Borgmann ◽  
Zhenghao Sun ◽  
...  
Keyword(s):  
Test Data ◽  
Urban Areas ◽  
Point Cloud ◽  
Large Scale ◽  
Point Clouds ◽  
Semantic Interpretation ◽  
3D Point Clouds ◽  
Semantic Labeling ◽  
Benchmark Datasets ◽  
Semantic Point

In the past decade, a vast amount of strategies, methods, and algorithms have been developed to explore the semantic interpretation of 3D point clouds for extracting desirable information. To assess the performance of the developed algorithms or methods, public standard benchmark datasets should invariably be introduced and used, which serve as an indicator and ruler in the evaluation and comparison. In this work, we introduce and present large-scale Mobile LiDAR point clouds acquired at the city campus of the Technical University of Munich, which have been manually annotated and can be used for the evaluation of related algorithms and methods for semantic point cloud interpretation. We created three datasets from a measurement campaign conducted in April 2016, including a benchmark dataset for semantic labeling, test data for instance segmentation, and test data for annotated single 360 ° laser scans. These datasets cover an urban area of approximately 1 km long roadways and include more than 40 million annotated points with eight classes of objects labeled. Moreover, experiments were carried out with results from several baseline methods compared and analyzed, revealing the quality of this dataset and its effectiveness when using it for performance evaluation.

scholarly journals A REGISTRATION METHOD OF POINT CLOUDS COLLECTED BY MOBILE LIDAR USING SOLELY STANDARD LAS FILES INFORMATION

2017 ◽  
Vol XLII-1/W1 ◽  
pp. 121-128 ◽  
Author(s):  
L. Gézero ◽  
C. Antunes
Keyword(s):  
Point Cloud ◽  
Large Scale ◽  
Auxiliary Information ◽  
Point Clouds ◽  
Registration Method ◽  
Digital Terrain ◽  
Terrain Models ◽  
Dense Point ◽  
Fully Automatic ◽  
Method Point

In the last few years, LiDAR sensors installed in terrestrial vehicles have been revealed as an efficient method to collect very dense 3D georeferenced information. The possibility of creating very dense point clouds representing the surface surrounding the sensor, at a given moment, in a very fast, detailed and easy way, shows the potential of this technology to be used for cartography and digital terrain models production in large scale. However, there are still some limitations associated with the use of this technology. When several acquisitions of the same area with the same device, are made, differences between the clouds can be observed. The range of that differences can go from few centimetres to some several tens of centimetres, mainly in urban and high vegetation areas where the occultation of the GNSS system introduces a degradation of the georeferenced trajectory. Along this article a different method point cloud registration is proposed. In addition to the efficiency and speed of execution, the main advantages of the method are related to the fact that the adjustment is continuously made over the trajectory, based on the GPS time. The process is fully automatic and only information recorded in the standard LAS files is used, without the need for any auxiliary information, in particular regarding the trajectory.

scholarly journals 3D THERMAL MAPPING OF BUILDING ROOFS BASED ON FUSION OF THERMAL AND VISIBLE POINT CLOUDS IN UAV IMAGERY

2019 ◽  
Vol XLII-4/W18 ◽  
pp. 271-277
Author(s):  
M. Dahaghin ◽  
F. Samadzadegan ◽  
F. Dadras Javan
Keyword(s):  
Camera Calibration ◽  
Point Cloud ◽  
Point Clouds ◽  
Control Points ◽  
Thermal Camera ◽  
Dense Point ◽  
Normal Vectors ◽  
Projection Error ◽  
Orientation Parameters ◽  
Vegetation Layer

Abstract. Thermography is a robust method for detecting thermal irregularities on the roof of the buildings as one of the main energy dissipation parts. Recently, UAVs are presented to be useful in gathering 3D thermal data of the building roofs. In this topic, the low spatial resolution of thermal imagery is a challenge which leads to a sparse resolution in point clouds. This paper suggests the fusion of visible and thermal point clouds to generate a high-resolution thermal point cloud of the building roofs. For the purpose, camera calibration is performed to obtain internal orientation parameters, and then thermal point clouds and visible point clouds are generated. In the next step, both two point clouds are geo-referenced by control points. To extract building roofs from the visible point cloud, CSF ground filtering is applied, and the vegetation layer is removed by RGBVI index. Afterward, a predefined threshold is applied to the normal vectors in the z-direction in order to separate facets of roofs from the walls. Finally, the visible point cloud of the building roofs and registered thermal point cloud are combined and generate a fused dense point cloud. Results show mean re-projection error of 0.31 pixels for thermal camera calibration and mean absolute distance of 0.2 m for point clouds registration. The final product is a fused point cloud, which its density improves up to twice of the initial thermal point cloud density and it has the spatial accuracy of visible point cloud along with thermal information of the building roofs.

scholarly journals Ground-distance segmentation of 3D LiDAR point cloud toward autonomous driving

2020 ◽  
Vol 9 ◽  
Author(s):  
Jian Wu ◽  
Qingxiong Yang
Keyword(s):  
Point Cloud ◽  
Large Scale ◽  
Ground Plane ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Autonomous Driving ◽  
Urban Environments ◽  
Cloud Data ◽  
Dense Point ◽  
3D Lidar

In this paper, we study the semantic segmentation of 3D LiDAR point cloud data in urban environments for autonomous driving, and a method utilizing the surface information of the ground plane was proposed. In practice, the resolution of a LiDAR sensor installed in a self-driving vehicle is relatively low and thus the acquired point cloud is indeed quite sparse. While recent work on dense point cloud segmentation has achieved promising results, the performance is relatively low when directly applied to sparse point clouds. This paper is focusing on semantic segmentation of the sparse point clouds obtained from 32-channel LiDAR sensor with deep neural networks. The main contribution is the integration of the ground information which is used to group ground points far away from each other. Qualitative and quantitative experiments on two large-scale point cloud datasets show that the proposed method outperforms the current state-of-the-art.

scholarly journals EXPERIMENTS WITH UAS IMAGERY FOR AUTOMATIC MODELING OF POWER LINE 3D GEOMETRY

2015 ◽  
Vol XL-1/W4 ◽  
pp. 403-409 ◽  
Author(s):  
G. Jóźków ◽  
B. Vander Jagt ◽  
C. Toth
Keyword(s):  
Transmission Line ◽  
Transmission Lines ◽  
Large Scale ◽  
Point Clouds ◽  
Airborne Lidar ◽  
Aerial Images ◽  
Automatic Modeling ◽  
Dense Matching ◽  
3D Geometry ◽  
Mapping Technology

The ideal mapping technology for transmission line inspection is the airborne LiDAR executed from helicopter platforms. It allows for full 3D geometry extraction in highly automated manner. Large scale aerial images can be also used for this purpose, however, automation is possible only for finding transmission line positions (2D geometry), and the sag needs to be estimated manually. For longer lines, these techniques are less expensive than ground surveys, yet they are still expensive. UAS technology has the potential to reduce these costs, especially if using inexpensive platforms with consumer grade cameras. This study investigates the potential of using high resolution UAS imagery for automatic modeling of transmission line 3D geometry. <br><br> The key point of this experiment was to employ dense matching algorithms to appropriately acquired UAS images to have points created also on wires. This allowed to model the 3D geometry of transmission lines similarly to LiDAR acquired point clouds. Results showed that the transmission line modeling is possible with a high internal accuracy for both, horizontal and vertical directions, even when wires were represented by a partial (sparse) point cloud.

scholarly journals GEOREFERENCING AND REPROJECTION ERROR INVESTIGATION ON IMAGE BASED 3D DIGITIZATION AND MAPPING OF HISTORICAL BUILDINGS

2019 ◽  
Vol XLII-2/W11 ◽  
pp. 71-75
Author(s):  
C. Altuntas
Keyword(s):  
Point Cloud ◽  
Large Scale ◽  
Cloud Model ◽  
Low Cost ◽  
Three Dimensional ◽  
Control Points ◽  
Historical Buildings ◽  
Reprojection Error ◽  
Dense Point ◽  
Point Cloud Model

<p><strong>Abstract.</strong> Image based dense point cloud creation is easy and low-cost application for three dimensional digitization of small and large scale objects and surfaces. It is especially attractive method for cultural heritage documentation. Reprojection error on conjugate keypoints indicates accuracy of the model and keypoint localisation in this method. In addition, sequential registration of the images from large scale historical buildings creates big cumulative registration error. Thus, accuracy of the model should be increased with the control points or loop close imaging. The registration of point point cloud model into the georeference system is performed using control points. In this study historical Sultan Selim Mosque that was built in sixteen century by Great Architect Sinan was modelled via photogrammetric dense point cloud. The reprojection error and number of keypoints were evaluated for different base/length ratio. In addition, georeferencing accuracy was evaluated with many configuration of control points with loop and without loop closure imaging.</p>

scholarly journals EXPLORING ALS AND DIM DATA FOR SEMANTIC SEGMENTATION USING CNNS

2018 ◽  
Vol XLII-1 ◽  
pp. 347-354 ◽  
Author(s):  
F. Politz ◽  
M. Sester
Keyword(s):  
Point Cloud ◽  
Laser Scanning ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Good Alternative ◽  
Aerial Images ◽  
Learning Approaches ◽  
Advantages And Disadvantages ◽  
Sensing Applications ◽  
High Level

<p><strong>Abstract.</strong> Over the past years, the algorithms for dense image matching (DIM) to obtain point clouds from aerial images improved significantly. Consequently, DIM point clouds are now a good alternative to the established Airborne Laser Scanning (ALS) point clouds for remote sensing applications. In order to derive high-level applications such as digital terrain models or city models, each point within a point cloud must be assigned a class label. Usually, ALS and DIM are labelled with different classifiers due to their varying characteristics. In this work, we explore both point cloud types in a fully convolutional encoder-decoder network, which learns to classify ALS as well as DIM point clouds. As input, we project the point clouds onto a 2D image raster plane and calculate the minimal, average and maximal height values for each raster cell. The network then differentiates between the classes ground, non-ground, building and no data. We test our network in six training setups using only one point cloud type, both point clouds as well as several transfer-learning approaches. We quantitatively and qualitatively compare all results and discuss the advantages and disadvantages of all setups. The best network achieves an overall accuracy of 96<span class="thinspace"></span>% in an ALS and 83<span class="thinspace"></span>% in a DIM test set.</p>

scholarly journals A Simple Semantic-Based Data Storage Layout for Querying Point Clouds

2020 ◽  
Vol 9 (2) ◽  
pp. 72 ◽  
Author(s):  
Sami El-Mahgary ◽  
Juho-Pekka Virtanen ◽  
Hannu Hyyppä
Keyword(s):  
Data Storage ◽  
Point Cloud ◽  
Point Clouds ◽  
Database Management System ◽  
Programming Approach ◽  
Point Cloud Data ◽  
Data Layout ◽  
Cloud Data ◽  
Semantic Point ◽  
Relational Database Management

The importance of being able to separate the semantics from the actual (X,Y,Z) coordinates in a point cloud has been actively brought up in recent research. However, there is still no widely used or accepted data layout paradigm on how to efficiently store and manage such semantic point cloud data. In this paper, we present a simple data layout that makes use the semantics and that allows for quick queries. The underlying idea is especially suited for a programming approach (e.g., queries programmed via Python) but we also present an even simpler implementation of the underlying technique on a well known relational database management system (RDBMS), namely, PostgreSQL. The obtained query results suggest that the presented approach can be successfully used to handle point and range queries on large points clouds.

scholarly journals Evaluating Thermal Attribute Mapping Strategies for Oblique Airborne Photogrammetric System AOS-Tx8

2019 ◽  
Vol 12 (1) ◽  
pp. 112 ◽  
Author(s):  
Dong Lin ◽  
Lutz Bannehr ◽  
Christoph Ulrich ◽  
Hans-Gerd Maas
Keyword(s):  
Point Cloud ◽  
Large Scale ◽  
Texture Mapping ◽  
Point Clouds ◽  
Registration Accuracy ◽  
Point Cloud Registration ◽  
Thermal Imagery ◽  
Processing Scheme ◽  
Data Registration ◽  
Thermal Cameras

Thermal imagery is widely used in various fields of remote sensing. In this study, a novel processing scheme is developed to process the data acquired by the oblique airborne photogrammetric system AOS-Tx8 consisting of four thermal cameras and four RGB cameras with the goal of large-scale area thermal attribute mapping. In order to merge 3D RGB data and 3D thermal data, registration is conducted in four steps: First, thermal and RGB point clouds are generated independently by applying structure from motion (SfM) photogrammetry to both the thermal and RGB imagery. Next, a coarse point cloud registration is performed by the support of georeferencing data (global positioning system, GPS). Subsequently, a fine point cloud registration is conducted by octree-based iterative closest point (ICP). Finally, three different texture mapping strategies are compared. Experimental results showed that the global image pose refinement outperforms the other two strategies at registration accuracy between thermal imagery and RGB point cloud. Potential building thermal leakages in large areas can be fast detected in the generated texture mapping results. Furthermore, a combination of the proposed workflow and the oblique airborne system allows for a detailed thermal analysis of building roofs and facades.

Sign in / Sign up

Export Citation Format

Share Document