scholarly journals SegMap: Segment-based mapping and localization using data-driven descriptors

2019 ◽  
Vol 39 (2-3) ◽  
pp. 339-355 ◽  
Author(s):  
Renaud Dubé ◽  
Andrei Cramariuc ◽  
Daniel Dugas ◽  
Hannes Sommer ◽  
Marcin Dymczyk ◽  
...  
Keyword(s):  
Large Scale ◽  
Structural Changes ◽  
Mobile Robotics ◽  
State Of The Art ◽  
Point Clouds ◽  
Open Loop ◽  
Data Driven ◽  
Global Localization ◽  
3D Point Clouds ◽  
Coarse Information

Precisely estimating a robot’s pose in a prior, global map is a fundamental capability for mobile robotics, e.g., autonomous driving or exploration in disaster zones. This task, however, remains challenging in unstructured, dynamic environments, where local features are not discriminative enough and global scene descriptors only provide coarse information. We therefore present SegMap: a map representation solution for localization and mapping based on the extraction of segments in 3D point clouds. Working at the level of segments offers increased invariance to view-point and local structural changes, and facilitates real-time processing of large-scale 3D data. SegMap exploits a single compact data-driven descriptor for performing multiple tasks: global localization, 3D dense map reconstruction, and semantic information extraction. The performance of SegMap is evaluated in multiple urban driving and search and rescue experiments. We show that the learned SegMap descriptor has superior segment retrieval capabilities, compared with state-of-the-art handcrafted descriptors. As a consequence, we achieve a higher localization accuracy and a 6% increase in recall over state-of-the-art handcrafted descriptors. These segment-based localizations allow us to reduce the open-loop odometry drift by up to 50%. SegMap is open-source available along with easy to run demonstrations.

scholarly journals Scene Wireframes Sketching for Drones

Proceedings ◽  
2018 ◽  
Vol 2 (18) ◽  
pp. 1193
Author(s):  
Roi Santos ◽  
Xose Pardo ◽  
Xose Fdez-Vidal
Keyword(s):  
State Of The Art ◽  
Point Clouds ◽  
Outdoor Environment ◽  
Point Matching ◽  
3D Point Clouds ◽  
Complementary Approach ◽  
Straight Line ◽  
Scene Representation ◽  
Feature Point Matching ◽  
3D Scene

The increasing use of autonomous UAVs inside buildings and around human-made structures demands new accurate and comprehensive representation of their operation environments. Most of the 3D scene abstraction methods use invariant feature point matching, nevertheless some sparse 3D point clouds do not concisely represent the structure of the environment. Likewise, line clouds constructed by short and redundant segments with inaccurate directions limit the understanding of scenes as those that include environments with poor texture, or whose texture resembles a repetitive pattern. The presented approach is based on observation and representation models using the straight line segments, whose resemble the limits of an urban indoor or outdoor environment. The goal of the work is to get a full method based on the matching of lines that provides a complementary approach to state-of-the-art methods when facing 3D scene representation of poor texture environments for future autonomous UAV.

scholarly journals Automated Visual Recognizability Evaluation of Traffic Sign Based on 3D LiDAR Point Clouds

2019 ◽  
Vol 11 (12) ◽  
pp. 1453 ◽  
Author(s):  
Shanxin Zhang ◽  
Cheng Wang ◽  
Lili Lin ◽  
Chenglu Wen ◽  
Chenhui Yang ◽  
...  
Keyword(s):  
Traffic Safety ◽  
Laser Scanning ◽  
Large Scale ◽  
Evaluation Method ◽  
Evaluation Model ◽  
Point Clouds ◽  
Traffic Sign ◽  
Traffic Signs ◽  
3D Point Clouds ◽  
Traffic Environment

Maintaining the high visual recognizability of traffic signs for traffic safety is a key matter for road network management. Mobile Laser Scanning (MLS) systems provide efficient way of 3D measurement over large-scale traffic environment. This paper presents a quantitative visual recognizability evaluation method for traffic signs in large-scale traffic environment based on traffic recognition theory and MLS 3D point clouds. We first propose the Visibility Evaluation Model (VEM) to quantitatively describe the visibility of traffic sign from any given viewpoint, then we proposed the concept of visual recognizability field and Traffic Sign Visual Recognizability Evaluation Model (TSVREM) to measure the visual recognizability of a traffic sign. Finally, we present an automatic TSVREM calculation algorithm for MLS 3D point clouds. Experimental results on real MLS 3D point clouds show that the proposed method is feasible and efficient.

scholarly journals Supervised Detection of Façade Openings in 3D Point Clouds with Thermal Attributes

2020 ◽  
Vol 12 (3) ◽  
pp. 543 ◽  
Author(s):  
Małgorzata Jarząbek-Rychard ◽  
Dong Lin ◽  
Hans-Gerd Maas
Keyword(s):  
Large Scale ◽  
Conditional Random Field ◽  
Contextual Information ◽  
Point Clouds ◽  
Thermal Infrared ◽  
Classification Performance ◽  
Space Images ◽  
3D Point Clouds ◽  
3D Scene ◽  
And Control

Targeted energy management and control is becoming an increasing concern in the building sector. Automatic analyses of thermal data, which minimize the subjectivity of the assessment and allow for large-scale inspections, are therefore of high interest. In this study, we propose an approach for a supervised extraction of façade openings (windows and doors) from photogrammetric 3D point clouds attributed to RGB and thermal infrared (TIR) information. The novelty of the proposed approach is in the combination of thermal information with other available characteristics of data for a classification performed directly in 3D space. Images acquired in visible and thermal infrared spectra serve as input data for the camera pose estimation and the reconstruction of 3D scene geometry. To investigate the relevance of different information types to the classification performance, a Random Forest algorithm is applied to various sets of computed features. The best feature combination is then used as an input for a Conditional Random Field that enables us to incorporate contextual information and consider the interaction between the points. The evaluation executed on a per-point level shows that the fusion of all available information types together with context consideration allows us to extract objects with 90% completeness and 95% correctness. A respective assessment executed on a per-object level shows 97% completeness and 88% accuracy.

scholarly journals TUM-MLS-2016: An Annotated Mobile LiDAR Dataset of the TUM City Campus for Semantic Point Cloud Interpretation in Urban Areas

2020 ◽  
Vol 12 (11) ◽  
pp. 1875 ◽  
Author(s):  
Jingwei Zhu ◽  
Joachim Gehrung ◽  
Rong Huang ◽  
Björn Borgmann ◽  
Zhenghao Sun ◽  
...  
Keyword(s):  
Test Data ◽  
Urban Areas ◽  
Point Cloud ◽  
Large Scale ◽  
Point Clouds ◽  
Semantic Interpretation ◽  
3D Point Clouds ◽  
Semantic Labeling ◽  
Benchmark Datasets ◽  
Semantic Point

In the past decade, a vast amount of strategies, methods, and algorithms have been developed to explore the semantic interpretation of 3D point clouds for extracting desirable information. To assess the performance of the developed algorithms or methods, public standard benchmark datasets should invariably be introduced and used, which serve as an indicator and ruler in the evaluation and comparison. In this work, we introduce and present large-scale Mobile LiDAR point clouds acquired at the city campus of the Technical University of Munich, which have been manually annotated and can be used for the evaluation of related algorithms and methods for semantic point cloud interpretation. We created three datasets from a measurement campaign conducted in April 2016, including a benchmark dataset for semantic labeling, test data for instance segmentation, and test data for annotated single 360 ° laser scans. These datasets cover an urban area of approximately 1 km long roadways and include more than 40 million annotated points with eight classes of objects labeled. Moreover, experiments were carried out with results from several baseline methods compared and analyzed, revealing the quality of this dataset and its effectiveness when using it for performance evaluation.

scholarly journals Tunnel Deformation Inspection via Global Spatial Axis Extraction from 3D Raw Point Cloud

Sensors ◽  
2020 ◽  
Vol 20 (23) ◽  
pp. 6815
Author(s):  
Cheng Yi ◽  
Dening Lu ◽  
Qian Xie ◽  
Jinxuan Xu ◽  
Jun Wang
Keyword(s):  
Cross Sections ◽  
Point Cloud ◽  
Large Scale ◽  
Spline Approximation ◽  
Point Clouds ◽  
Driving Safety ◽  
Inspection Method ◽  
B Spline ◽  
3D Point Clouds ◽  
Spatial Axis

Global inspection of large-scale tunnels is a fundamental yet challenging task to ensure the structural stability of tunnels and driving safety. Advanced LiDAR scanners, which sample tunnels into 3D point clouds, are making their debut in the Tunnel Deformation Inspection (TDI). However, the acquired raw point clouds inevitably possess noticeable occlusions, missing areas, and noise/outliers. Considering the tunnel as a geometrical sweeping feature, we propose an effective tunnel deformation inspection algorithm by extracting the global spatial axis from the poor-quality raw point cloud. Essentially, we convert tunnel axis extraction into an iterative fitting optimization problem. Specifically, given the scanned raw point cloud of a tunnel, the initial design axis is sampled to generate a series of normal planes within the corresponding Frenet frame, followed by intersecting those planes with the tunnel point cloud to yield a sequence of cross sections. By fitting cross sections with circles, the fitted circle centers are approximated with a B-Spline curve, which is considered as an updated axis. The procedure of “circle fitting and B-SPline approximation” repeats iteratively until convergency, that is, the distance of each fitted circle center to the current axis is smaller than a given threshold. By this means, the spatial axis of the tunnel can be accurately obtained. Subsequently, according to the practical mechanism of tunnel deformation, we design a segmentation approach to partition cross sections into meaningful pieces, based on which various inspection parameters can be automatically computed regarding to tunnel deformation. A variety of practical experiments have demonstrated the feasibility and effectiveness of our inspection method.

scholarly journals EVALUATING UNMANNED AERIAL PLATFORMS FOR CULTURAL HERITAGE LARGE SCALE MAPPING

2016 ◽  
Vol XLI-B5 ◽  
pp. 355-362 ◽  
Author(s):  
A. Georgopoulos ◽  
C. Oikonomou ◽  
E. Adamopoulos ◽  
E. K. Stathopoulou
Keyword(s):  
Cultural Heritage ◽  
Optical Sensors ◽  
Large Scale ◽  
Point Clouds ◽  
Short Review ◽  
Data Set ◽  
Heritage Sites ◽  
3D Point Clouds ◽  
Aerial Platforms ◽  
3D Information

When it comes to large scale mapping of limited areas especially for cultural heritage sites, things become critical. Optical and non-optical sensors are developed to such sizes and weights that can be lifted by such platforms, like e.g. LiDAR units. At the same time there is an increase in emphasis on solutions that enable users to get access to 3D information faster and cheaper. Considering the multitude of platforms, cameras and the advancement of algorithms in conjunction with the increase of available computing power this challenge should and indeed is further investigated. In this paper a short review of the UAS technologies today is attempted. A discussion follows as to their applicability and advantages, depending on their specifications, which vary immensely. The on-board cameras available are also compared and evaluated for large scale mapping. Furthermore a thorough analysis, review and experimentation with different software implementations of Structure from Motion and Multiple View Stereo algorithms, able to process such dense and mostly unordered sequence of digital images is also conducted and presented. As test data set, we use a rich optical and thermal data set from both fixed wing and multi-rotor platforms over an archaeological excavation with adverse height variations and using different cameras. Dense 3D point clouds, digital terrain models and orthophotos have been produced and evaluated for their radiometric as well as metric qualities.

scholarly journals Utilizing UAV and 3D Computer Vision for Visual Inspection of a Large Gravity Dam

2018 ◽  
Author(s):  
Ali Khaloo ◽  
David Lattanzi ◽  
Adam Jachimowicz
Keyword(s):  
Large Scale ◽  
Critical Infrastructure ◽  
Three Dimensional ◽  
Point Clouds ◽  
Gravity Dam ◽  
Concrete Gravity Dam ◽  
3D Computer Vision ◽  
3D Point Clouds ◽  
Safety Risks ◽  
Specialized Equipment

Dams are a critical infrastructure system for many communities, but they are also one of the most challenging to inspect. Dams are typically very large and complex structures, and the result is that inspections are often time-intensive and require expensive, specialized equipment and training to provide inspectors with comprehensive access to the structure. The scale and nature of dam inspections also introduces additional safety risks to the inspectors. Unmanned aerial vehicles (UAV) have the potential to address many of these challenges, particularly when used as a data acquisition platform for photogrammetric three-dimensional (3D) reconstruction and analysis, though the nature of both UAV and modern photogrammetric methods necessitates careful planning and coordination for integration. This paper presents a case study on one such integration at the Brighton Dam, a large-scale concrete gravity dam in Maryland, USA. A combination of multiple UAV platforms and multi-scale photogrammetry was used to create two comprehensive and high-resolution 3D point clouds of the dam and surrounding environment at intervals. These models were then assessed for their overall quality, as well as their ability to resolve flaws and defects that were artificially applied to the structure between inspection intervals. The results indicate that the integrated process is capable of generating models that accurately render a variety of defect types with sub-millimeter accuracy. Recommendations for mission planning and imaging specifications are provided as well.

scholarly journals FAST PROBABILISTIC FUSION OF 3D POINT CLOUDS VIA OCCUPANCY GRIDS FOR SCENE CLASSIFICATION

2016 ◽  
Vol III-3 ◽  
pp. 325-332
Author(s):  
Andreas Kuhn ◽  
Hai Huang ◽  
Martin Drauschke ◽  
Helmut Mayer
Keyword(s):  
High Resolution ◽  
Point Cloud ◽  
State Of The Art ◽  
Point Clouds ◽  
Automatic Generation ◽  
Scene Classification ◽  
Semantic Classification ◽  
3D Point Clouds ◽  
Occupancy Grids ◽  
High Scalability

High resolution consumer cameras on Unmanned Aerial Vehicles (UAVs) allow for cheap acquisition of highly detailed images, e.g., of urban regions. Via image registration by means of Structure from Motion (SfM) and Multi View Stereo (MVS) the automatic generation of huge amounts of 3D points with a relative accuracy in the centimeter range is possible. Applications such as semantic classification have a need for accurate 3D point clouds, but do not benefit from an extremely high resolution/density. In this paper, we, therefore, propose a fast fusion of high resolution 3D point clouds based on occupancy grids. The result is used for semantic classification. In contrast to state-of-the-art classification methods, we accept a certain percentage of outliers, arguing that they can be considered in the classification process when a per point belief is determined in the fusion process. To this end, we employ an octree-based fusion which allows for the derivation of outlier probabilities. The probabilities give a belief for every 3D point, which is essential for the semantic classification to consider measurement noise. For an example point cloud with half a billion 3D points (cf. Figure 1), we show that our method can reduce runtime as well as improve classification accuracy and offers high scalability for large datasets.

Sign in / Sign up

Export Citation Format

Share Document