Visual Simultaneous Localization and Mapping with Applications to Monitoring of Underground Transportation Infrastructure

2020 ◽  
pp. 036119812096310
Author(s):  
Fred Daneshgaran ◽  
Antonio Marangi ◽  
Nicola Bruno ◽  
Fausto Lizzio ◽  
Marina Mondin ◽  
...  
Keyword(s):  
Real Time ◽  
Simultaneous Localization And Mapping ◽  
Transportation Infrastructure ◽  
Sensor Data ◽  
Processing Unit ◽  
Localization System ◽  
The Core ◽  
Time Operation ◽  
Localization And Mapping ◽  
Underground Transportation

This paper presents the results of the development, design, and implementation of a visual simultaneous localization and mapping (SLAM) system for autonomous real-time localization with application to underground transportation infrastructure (UTI) such as tunnels. Localization is achieved in the absence of any global positioning system (GPS) or auxiliary system. The indoor localization system is a necessary element of a fully autonomous platform for the detection of cracks and other anomalies on the interior surfaces of tunnels and other UTI. It can be used for tagging of high-resolution sensor data obtained with low-cost prototype data acquisition platforms previously developed. Visual based SLAM has been used as the core element in an architecture employing a commercial off-the-shelf (COTS) ZED stereo camera from Stereolabs. To achieve real-time operation, an NVIDIA Jetson TX2 massively parallel graphics processing unit (GPU) was used as the core computational engine employing two different software libraries. We achieved localization at 5 frames per second (FPS) using ORBSLAM2 open-source software library, and the much lighter, but proprietary, ZED SDK was able to deliver a performance at nearly 60 FPS. To assess the accuracy of the relative localization system, we conducted several tests at 30 FPS and reported on the resulting error variances that were found to be consistently very small. Finally, we conducted several tests in a tunnel in the Los Angeles county area and confirmed the applicability of the method for monitoring UTI.

scholarly journals JD-SLAM: Joint camera pose estimation and moving object segmentation for simultaneous localization and mapping in dynamic scenes

2021 ◽  
Vol 18 (1) ◽  
pp. 172988142199444
Author(s):  
Yujia Zhai ◽  
Baoli Lu ◽  
Weijun Li ◽  
Jian Xu ◽  
Shuangyi Ma
Keyword(s):  
Object Detection ◽  
Real Time ◽  
Object Segmentation ◽  
Three Dimensional ◽  
Simultaneous Localization And Mapping ◽  
Processing Unit ◽  
Dynamic Scenes ◽  
The Real ◽  
Localization And Mapping ◽  
Camera Pose

As a fundamental assumption in simultaneous localization and mapping, the static scenes hypothesis can be hardly fulfilled in applications of indoor/outdoor navigation or localization. Recent works about simultaneous localization and mapping in dynamic scenes commonly use heavy pixel-level segmentation net to distinguish dynamic objects, which brings enormous calculations and limits the real-time performance of the system. That restricts the application of simultaneous localization and mapping on the mobile terminal. In this article, we present a lightweight system for monocular simultaneous localization and mapping in dynamic scenes, which can run in real time on central processing unit (CPU) and generate a semantic probability map. The pixel-wise semantic segmentation net is replaced with a lightweight object detection net combined with three-dimensional segmentation based on motion clustering. And a framework integrated with an improved weighted-random sample consensus solver is proposed to jointly solve the camera pose and perform three-dimensional object segmentation, which enables high accuracy and efficiency. Besides, the prior information of the generated map and the object detection results is introduced for better estimation. The experiments on the public data set, and in the real-world demonstrate that our method obtains an outstanding improvement in both accuracy and speed compared to state-of-the-art methods.

scholarly journals Real-Time Expanded Field-of-View for Minimally Invasive Surgery Using Multi-Camera Visual Simultaneous Localization and Mapping

Sensors ◽  
2021 ◽  
Vol 21 (6) ◽  
pp. 2106
Author(s):  
Ahmed Afifi ◽  
Chisato Takada ◽  
Yuichiro Yoshimura ◽  
Toshiya Nakaguchi
Keyword(s):  
Minimally Invasive Surgery ◽  
Minimally Invasive ◽  
Real Time ◽  
Invasive Surgery ◽  
Simultaneous Localization And Mapping ◽  
Field Of View ◽  
Time Dynamic ◽  
Matrix Estimation ◽  
Localization And Mapping

Minimally invasive surgery is widely used because of its tremendous benefits to the patient. However, there are some challenges that surgeons face in this type of surgery, the most important of which is the narrow field of view. Therefore, we propose an approach to expand the field of view for minimally invasive surgery to enhance surgeons’ experience. It combines multiple views in real-time to produce a dynamic expanded view. The proposed approach extends the monocular Oriented features from an accelerated segment test and Rotated Binary robust independent elementary features—Simultaneous Localization And Mapping (ORB-SLAM) to work with a multi-camera setup. The ORB-SLAM’s three parallel threads, namely tracking, mapping and loop closing, are performed for each camera and new threads are added to calculate the relative cameras’ pose and to construct the expanded view. A new algorithm for estimating the optimal inter-camera correspondence matrix from a set of corresponding 3D map points is presented. This optimal transformation is then used to produce the final view. The proposed approach was evaluated using both human models and in vivo data. The evaluation results of the proposed correspondence matrix estimation algorithm prove its ability to reduce the error and to produce an accurate transformation. The results also show that when other approaches fail, the proposed approach can produce an expanded view. In this work, a real-time dynamic field-of-view expansion approach that can work in all situations regardless of images’ overlap is proposed. It outperforms the previous approaches and can also work at 21 fps.

scholarly journals REINFORCEMENT LEARNING HELPS SLAM: LEARNING TO BUILD MAPS

2020 ◽  
Vol XLIII-B4-2020 ◽  
pp. 329-335
Author(s):  
N. Botteghi ◽  
B. Sirmacek ◽  
R. Schulte ◽  
M. Poel ◽  
C. Brune
Keyword(s):  
Reinforcement Learning ◽  
Real Time ◽  
A Priori ◽  
Simultaneous Localization And Mapping ◽  
Indoor Environments ◽  
Robot Localization ◽  
Robust Solution ◽  
Localization And Mapping ◽  
Reward Functions ◽  
Slam Algorithm

Abstract. In this research, we investigate the use of Reinforcement Learning (RL) for an effective and robust solution for exploring unknown and indoor environments and reconstructing their maps. We benefit from a Simultaneous Localization and Mapping (SLAM) algorithm for real-time robot localization and mapping. Three different reward functions are compared and tested in different environments with growing complexity. The performances of the three different RL-based path planners are assessed not only on the training environments, but also on an a priori unseen environment to test the generalization properties of the policies. The results indicate that RL-based planners trained to maximize the coverage of the map are able to consistently explore and construct the maps of different indoor environments.

scholarly journals Intensity-Assisted ICP for Fast Registration of 2D-LIDAR

Sensors ◽  
2019 ◽  
Vol 19 (9) ◽  
pp. 2124 ◽  
Author(s):  
Yingzhong Tian ◽  
Xining Liu ◽  
Long Li ◽  
Wenbin Wang
Keyword(s):  
Real Time ◽  
Computational Cost ◽  
Target Function ◽  
Picard Iteration ◽  
Processing Unit ◽  
Central Processing ◽  
Localization And Mapping ◽  
Initial Transformation ◽  
Comparative Results ◽  
Rigid Body Transformation

Iterative closest point (ICP) is a method commonly used to perform scan-matching and registration. To be a simple and robust algorithm, it is still computationally expensive, and it has been regarded as having a crucial challenge especially in a real-time application as used for the simultaneous localization and mapping (SLAM) problem. For these reasons, this paper presents a new method for the acceleration of ICP with an assisted intensity. Unlike the conventional ICP, this method is proposed to reduce the computational cost and avoid divergences. An initial transformation guess is computed with an assisted intensity for their relative rigid-body transformation. Moreover, a target function is proposed to determine the best initial transformation guess based on the statistic of their spatial distances and intensity residuals. Additionally, this method is also proposed to reduce the iteration number. The Anderson acceleration is utilized for increasing the iteration speed which has better ability than the Picard iteration procedure. The proposed algorithm is operated in real time with a single core central processing unit (CPU) thread. Hence, it is suitable for the robot which has limited computation resources. To validate the novelty, this proposed method is evaluated on the SEMANTIC3D.NET benchmark dataset. According to comparative results, the proposed method is declared as having better accuracy and robustness than the conventional ICP methods.

scholarly journals Real-Time RGB-D Simultaneous Localization and Mapping Guided by Terrestrial LiDAR Point Cloud for Indoor 3-D Reconstruction and Camera Pose Estimation

2019 ◽  
Vol 9 (16) ◽  
pp. 3264 ◽  
Author(s):  
Xujie Kang ◽  
Jing Li ◽  
Xiangtao Fan ◽  
Wenhui Wan
Keyword(s):  
Real Time ◽  
Point Cloud ◽  
Closed Loop ◽  
Simultaneous Localization And Mapping ◽  
Motion Blur ◽  
Long Distance ◽  
Terrestrial Lidar ◽  
Map Construction ◽  
Localization And Mapping ◽  
Camera Pose

In recent years, low-cost and lightweight RGB and depth (RGB-D) sensors, such as Microsoft Kinect, have made available rich image and depth data, making them very popular in the field of simultaneous localization and mapping (SLAM), which has been increasingly used in robotics, self-driving vehicles, and augmented reality. The RGB-D SLAM constructs 3D environmental models of natural landscapes while simultaneously estimating camera poses. However, in highly variable illumination and motion blur environments, long-distance tracking can result in large cumulative errors and scale shifts. To address this problem in actual applications, in this study, we propose a novel multithreaded RGB-D SLAM framework that incorporates a highly accurate prior terrestrial Light Detection and Ranging (LiDAR) point cloud, which can mitigate cumulative errors and improve the system’s robustness in large-scale and challenging scenarios. First, we employed deep learning to achieve system automatic initialization and motion recovery when tracking is lost. Next, we used terrestrial LiDAR point cloud to obtain prior data of the landscape, and then we applied the point-to-surface inductively coupled plasma (ICP) iterative algorithm to realize accurate camera pose control from the previously obtained LiDAR point cloud data, and finally expanded its control range in the local map construction. Furthermore, an innovative double window segment-based map optimization method is proposed to ensure consistency, better real-time performance, and high accuracy of map construction. The proposed method was tested for long-distance tracking and closed-loop in two different large indoor scenarios. The experimental results indicated that the standard deviation of the 3D map construction is 10 cm in a mapping distance of 100 m, compared with the LiDAR ground truth. Further, the relative cumulative error of the camera in closed-loop experiments is 0.09%, which is twice less than that of the typical SLAM algorithm (3.4%). Therefore, the proposed method was demonstrated to be more robust than the ORB-SLAM2 algorithm in complex indoor environments.

Sign in / Sign up

Export Citation Format

Share Document