Real-Time RGB-D Simultaneous Localization and Mapping Guided by Terrestrial LiDAR Point Cloud for Indoor 3-D Reconstruction and Camera Pose Estimation

In recent years, low-cost and lightweight RGB and depth (RGB-D) sensors, such as Microsoft Kinect, have made available rich image and depth data, making them very popular in the field of simultaneous localization and mapping (SLAM), which has been increasingly used in robotics, self-driving vehicles, and augmented reality. The RGB-D SLAM constructs 3D environmental models of natural landscapes while simultaneously estimating camera poses. However, in highly variable illumination and motion blur environments, long-distance tracking can result in large cumulative errors and scale shifts. To address this problem in actual applications, in this study, we propose a novel multithreaded RGB-D SLAM framework that incorporates a highly accurate prior terrestrial Light Detection and Ranging (LiDAR) point cloud, which can mitigate cumulative errors and improve the system’s robustness in large-scale and challenging scenarios. First, we employed deep learning to achieve system automatic initialization and motion recovery when tracking is lost. Next, we used terrestrial LiDAR point cloud to obtain prior data of the landscape, and then we applied the point-to-surface inductively coupled plasma (ICP) iterative algorithm to realize accurate camera pose control from the previously obtained LiDAR point cloud data, and finally expanded its control range in the local map construction. Furthermore, an innovative double window segment-based map optimization method is proposed to ensure consistency, better real-time performance, and high accuracy of map construction. The proposed method was tested for long-distance tracking and closed-loop in two different large indoor scenarios. The experimental results indicated that the standard deviation of the 3D map construction is 10 cm in a mapping distance of 100 m, compared with the LiDAR ground truth. Further, the relative cumulative error of the camera in closed-loop experiments is 0.09%, which is twice less than that of the typical SLAM algorithm (3.4%). Therefore, the proposed method was demonstrated to be more robust than the ORB-SLAM2 algorithm in complex indoor environments.

Download Full-text

JD-SLAM: Joint camera pose estimation and moving object segmentation for simultaneous localization and mapping in dynamic scenes

International Journal of Advanced Robotic Systems ◽

10.1177/1729881421994447 ◽

2021 ◽

Vol 18 (1) ◽

pp. 172988142199444

Author(s):

Yujia Zhai ◽

Baoli Lu ◽

Weijun Li ◽

Jian Xu ◽

Shuangyi Ma

Keyword(s):

Object Detection ◽

Real Time ◽

Object Segmentation ◽

Three Dimensional ◽

Simultaneous Localization And Mapping ◽

Processing Unit ◽

Dynamic Scenes ◽

The Real ◽

Localization And Mapping ◽

Camera Pose

As a fundamental assumption in simultaneous localization and mapping, the static scenes hypothesis can be hardly fulfilled in applications of indoor/outdoor navigation or localization. Recent works about simultaneous localization and mapping in dynamic scenes commonly use heavy pixel-level segmentation net to distinguish dynamic objects, which brings enormous calculations and limits the real-time performance of the system. That restricts the application of simultaneous localization and mapping on the mobile terminal. In this article, we present a lightweight system for monocular simultaneous localization and mapping in dynamic scenes, which can run in real time on central processing unit (CPU) and generate a semantic probability map. The pixel-wise semantic segmentation net is replaced with a lightweight object detection net combined with three-dimensional segmentation based on motion clustering. And a framework integrated with an improved weighted-random sample consensus solver is proposed to jointly solve the camera pose and perform three-dimensional object segmentation, which enables high accuracy and efficiency. Besides, the prior information of the generated map and the object detection results is introduced for better estimation. The experiments on the public data set, and in the real-world demonstrate that our method obtains an outstanding improvement in both accuracy and speed compared to state-of-the-art methods.

Download Full-text

LeGO-LOAM-SC: An Improved Simultaneous Localization and Mapping Method Fusing LeGO-LOAM and Scan Context for Underground Coalmine

Sensors ◽

10.3390/s22020520 ◽

2022 ◽

Vol 22 (2) ◽

pp. 520

Author(s):

Guanghui Xue ◽

Jinbo Wei ◽

Ruixue Li ◽

Jian Cheng

Keyword(s):

Real Time ◽

Point Cloud ◽

Sequence Data ◽

Simultaneous Localization And Mapping ◽

Mapping Method ◽

Mapping Accuracy ◽

Time Performance ◽

Mobile Vehicle ◽

Localization And Mapping ◽

Mean Square Errors

Simultaneous localization and mapping (SLAM) is one of the key technologies for coal mine underground operation vehicles to build complex environment maps and positioning and to realize unmanned and autonomous operation. Many domestic and foreign scholars have studied many SLAM algorithms, but the mapping accuracy and real-time performance still need to be further improved. This paper presents a SLAM algorithm integrating scan context and Light weight and Ground-Optimized LiDAR Odometry and Mapping (LeGO-LOAM), LeGO-LOAM-SC. The algorithm uses the global descriptor extracted by scan context for loop detection, adds pose constraints to Georgia Tech Smoothing and Mapping (GTSAM) by Iterative Closest Points (ICP) for graph optimization, and constructs point cloud map and an output estimated pose of the mobile vehicle. The test with KITTI dataset 00 sequence data and the actual test in 2-storey underground parking lots are carried out. The results show that the proposed improved algorithm makes up for the drift of the point cloud map, has a higher mapping accuracy, a better real-time performance, a lower resource occupancy, a higher coincidence between trajectory estimation and real trajectory, smoother loop, and 6% reduction in CPU occupancy, the mean square errors of absolute trajectory error (ATE) and relative pose error (RPE) are reduced by 55.7% and 50.3% respectively; the translation and rotation accuracy are improved by about 5%, and the time consumption is reduced by 2~4%. Accurate map construction and low drift pose estimation can be performed.

Download Full-text

Real-Time Expanded Field-of-View for Minimally Invasive Surgery Using Multi-Camera Visual Simultaneous Localization and Mapping

Sensors ◽

10.3390/s21062106 ◽

2021 ◽

Vol 21 (6) ◽

pp. 2106

Author(s):

Ahmed Afifi ◽

Chisato Takada ◽

Yuichiro Yoshimura ◽

Toshiya Nakaguchi

Keyword(s):

Minimally Invasive Surgery ◽

Minimally Invasive ◽

Real Time ◽

Invasive Surgery ◽

Simultaneous Localization And Mapping ◽

Field Of View ◽

Time Dynamic ◽

Matrix Estimation ◽

Localization And Mapping

Minimally invasive surgery is widely used because of its tremendous benefits to the patient. However, there are some challenges that surgeons face in this type of surgery, the most important of which is the narrow field of view. Therefore, we propose an approach to expand the field of view for minimally invasive surgery to enhance surgeons’ experience. It combines multiple views in real-time to produce a dynamic expanded view. The proposed approach extends the monocular Oriented features from an accelerated segment test and Rotated Binary robust independent elementary features—Simultaneous Localization And Mapping (ORB-SLAM) to work with a multi-camera setup. The ORB-SLAM’s three parallel threads, namely tracking, mapping and loop closing, are performed for each camera and new threads are added to calculate the relative cameras’ pose and to construct the expanded view. A new algorithm for estimating the optimal inter-camera correspondence matrix from a set of corresponding 3D map points is presented. This optimal transformation is then used to produce the final view. The proposed approach was evaluated using both human models and in vivo data. The evaluation results of the proposed correspondence matrix estimation algorithm prove its ability to reduce the error and to produce an accurate transformation. The results also show that when other approaches fail, the proposed approach can produce an expanded view. In this work, a real-time dynamic field-of-view expansion approach that can work in all situations regardless of images’ overlap is proposed. It outperforms the previous approaches and can also work at 21 fps.

Download Full-text

Real-time Simultaneous Localization and Mapping (SLAM) for Vision-based Autonomous Navigation

Transactions of the Korean Society of Mechanical Engineers A ◽

10.3795/ksme-a.2015.39.5.483 ◽

2015 ◽

Vol 39 (5) ◽

pp. 483-489

Author(s):

Hyon Lim ◽

Jongwoo Lim ◽

H. Jin Kim

Keyword(s):

Real Time ◽

Autonomous Navigation ◽

Simultaneous Localization And Mapping ◽

Localization And Mapping

Download Full-text

Research and Optimization of Real-time Simultaneous Localization and Mapping of Indoor Robot Based on Binocular Vision

Journal of Physics Conference Series ◽

10.1088/1742-6596/1267/1/012039 ◽

2019 ◽

Vol 1267 ◽

pp. 012039

Author(s):

Qiwei Zhang ◽

Guanglu Zhou ◽

Pengpeng Wang ◽

Fengguang Wu

Keyword(s):

Real Time ◽

Binocular Vision ◽

Simultaneous Localization And Mapping ◽

Localization And Mapping

Download Full-text

REINFORCEMENT LEARNING HELPS SLAM: LEARNING TO BUILD MAPS

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xliii-b4-2020-329-2020 ◽

2020 ◽

Vol XLIII-B4-2020 ◽

pp. 329-335

Author(s):

N. Botteghi ◽

B. Sirmacek ◽

R. Schulte ◽

M. Poel ◽

C. Brune

Keyword(s):

Reinforcement Learning ◽

Real Time ◽

A Priori ◽

Simultaneous Localization And Mapping ◽

Indoor Environments ◽

Robot Localization ◽

Robust Solution ◽

Localization And Mapping ◽

Reward Functions ◽

Slam Algorithm

Abstract. In this research, we investigate the use of Reinforcement Learning (RL) for an effective and robust solution for exploring unknown and indoor environments and reconstructing their maps. We benefit from a Simultaneous Localization and Mapping (SLAM) algorithm for real-time robot localization and mapping. Three different reward functions are compared and tested in different environments with growing complexity. The performances of the three different RL-based path planners are assessed not only on the training environments, but also on an a priori unseen environment to test the generalization properties of the policies. The results indicate that RL-based planners trained to maximize the coverage of the map are able to consistently explore and construct the maps of different indoor environments.

Download Full-text