scholarly journals REINFORCEMENT LEARNING HELPS SLAM: LEARNING TO BUILD MAPS

2020 ◽  
Vol XLIII-B4-2020 ◽  
pp. 329-335
Author(s):  
N. Botteghi ◽  
B. Sirmacek ◽  
R. Schulte ◽  
M. Poel ◽  
C. Brune
Keyword(s):  
Reinforcement Learning ◽  
Real Time ◽  
A Priori ◽  
Simultaneous Localization And Mapping ◽  
Indoor Environments ◽  
Robot Localization ◽  
Robust Solution ◽  
Localization And Mapping ◽  
Reward Functions ◽  
Slam Algorithm

Abstract. In this research, we investigate the use of Reinforcement Learning (RL) for an effective and robust solution for exploring unknown and indoor environments and reconstructing their maps. We benefit from a Simultaneous Localization and Mapping (SLAM) algorithm for real-time robot localization and mapping. Three different reward functions are compared and tested in different environments with growing complexity. The performances of the three different RL-based path planners are assessed not only on the training environments, but also on an a priori unseen environment to test the generalization properties of the policies. The results indicate that RL-based planners trained to maximize the coverage of the map are able to consistently explore and construct the maps of different indoor environments.

scholarly journals CURIOSITY-DRIVEN REINFORCEMENT LEARNING AGENT FOR MAPPING UNKNOWN INDOOR ENVIRONMENTS

2021 ◽  
Vol V-1-2021 ◽  
pp. 129-136
Author(s):  
N. Botteghi ◽  
R. Schulte ◽  
B. Sirmacek ◽  
M. Poel ◽  
C. Brune
Keyword(s):  
Artificial Intelligence ◽  
Reinforcement Learning ◽  
Mobile Robot ◽  
Indoor Environments ◽  
Reward Function ◽  
Learning Framework ◽  
Learning Agent ◽  
The World ◽  
Reward Functions ◽  
Slam Algorithm

Abstract. Autonomously exploring and mapping is one of the open challenges of robotics and artificial intelligence. Especially when the environments are unknown, choosing the optimal navigation directive is not straightforward. In this paper, we propose a reinforcement learning framework for navigating, exploring, and mapping unknown environments. The reinforcement learning agent is in charge of selecting the commands for steering the mobile robot, while a SLAM algorithm estimates the robot pose and maps the environments. The agent, to select optimal actions, is trained to be curious about the world. This concept translates into the introduction of a curiosity-driven reward function that encourages the agent to steer the mobile robot towards unknown and unseen areas of the world and the map. We test our approach in explorations challenges in different indoor environments. The agent trained with the proposed reward function outperforms the agents trained with reward functions commonly used in the literature for solving such tasks.

A two-level optimized graph-based simultaneous localization and mapping algorithm

2018 ◽  
Vol 45 (6) ◽  
pp. 758-765
Author(s):  
Hui Xiong ◽  
Youping Chen ◽  
Xiaoping Li ◽  
Bing Chen
Keyword(s):  
Simultaneous Localization And Mapping ◽  
Environmental Information ◽  
Indoor Environments ◽  
Construction Process ◽  
Loop Closure ◽  
Content Type ◽  
Mapping Algorithm ◽  
Localization And Mapping ◽  
Slam Algorithm ◽  
Estimated Error

PurposeBecause submaps including a subset of the global map contain more environmental information, submap-based graph simultaneous localization and mapping (SLAM) has been studied by many researchers. In most of those studies, helpful environmental information was not taken into consideration when designed the termination criterion of the submap construction process. After optimizing the graph, cumulative error within the submaps was also ignored. To address those problems, this paper aims to propose a two-level optimized graph-based SLAM algorithm.Design/methodology/approachSubmaps are updated by extended Kalman filter SLAM while no geometric-shaped landmark models are needed; raw laser scans are treated as landmarks. A more reasonable criterion called the uncertainty index is proposed to combine with the size of the submap to terminate the submap construction process. After a submap is completed and a loop closure is found, a two-level optimization process is performed to minimize the loop closure error and the accumulated error within the submaps.FindingsSimulation and experimental results indicate that the estimated error of the proposed algorithm is small, and the maps generated are consistent whether in global or local.Practical implicationsThe proposed method is robust to sparse pedestrians and can be adapted to most indoor environments.Originality/valueIn this paper, a two-level optimized graph-based SLAM algorithm is proposed.

scholarly journals An Adaptive Augmented Vision-Based Ellipsoidal SLAM for Indoor Environments

Sensors ◽  
2019 ◽  
Vol 19 (12) ◽  
pp. 2795
Author(s):  
Lahemer ◽  
Rad
Keyword(s):  
Real Time ◽  
Indoor Environment ◽  
Vision System ◽  
Computational Effort ◽  
Monocular Vision ◽  
Indoor Environments ◽  
Simulation Studies ◽  
Localization And Mapping ◽  
Landmark Identification ◽  
Slam Algorithm

In this paper, the problem of Simultaneous Localization And Mapping (SLAM) is addressed via a novel augmented landmark vision-based ellipsoidal SLAM. The algorithm is implemented on a NAO humanoid robot and is tested in an indoor environment. The main feature of the system is the implementation of SLAM with a monocular vision system. Distinguished landmarks referred to as NAOmarks are employed to localize the robot via its monocular vision system. We henceforth introduce the notion of robotic augmented reality (RAR) and present a monocular Extended Kalman Filter (EKF)/ellipsoidal SLAM in order to improve the performance and alleviate the computational effort, to provide landmark identification, and to simplify the data association problem. The proposed SLAM algorithm is implemented in real-time to further calibrate the ellipsoidal SLAM parameters, noise bounding, and to improve its overall accuracy. The augmented EKF/ellipsoidal SLAM algorithms are compared with the regular EKF/ellipsoidal SLAM methods and the merits of each algorithm is also discussed in the paper. The real-time experimental and simulation studies suggest that the adaptive augmented ellipsoidal SLAM is more accurate than the conventional EKF/ellipsoidal SLAMs.

scholarly journals Real-Time Expanded Field-of-View for Minimally Invasive Surgery Using Multi-Camera Visual Simultaneous Localization and Mapping

Sensors ◽  
2021 ◽  
Vol 21 (6) ◽  
pp. 2106
Author(s):  
Ahmed Afifi ◽  
Chisato Takada ◽  
Yuichiro Yoshimura ◽  
Toshiya Nakaguchi
Keyword(s):  
Minimally Invasive Surgery ◽  
Minimally Invasive ◽  
Real Time ◽  
Invasive Surgery ◽  
Simultaneous Localization And Mapping ◽  
Field Of View ◽  
Time Dynamic ◽  
Matrix Estimation ◽  
Localization And Mapping

Minimally invasive surgery is widely used because of its tremendous benefits to the patient. However, there are some challenges that surgeons face in this type of surgery, the most important of which is the narrow field of view. Therefore, we propose an approach to expand the field of view for minimally invasive surgery to enhance surgeons’ experience. It combines multiple views in real-time to produce a dynamic expanded view. The proposed approach extends the monocular Oriented features from an accelerated segment test and Rotated Binary robust independent elementary features—Simultaneous Localization And Mapping (ORB-SLAM) to work with a multi-camera setup. The ORB-SLAM’s three parallel threads, namely tracking, mapping and loop closing, are performed for each camera and new threads are added to calculate the relative cameras’ pose and to construct the expanded view. A new algorithm for estimating the optimal inter-camera correspondence matrix from a set of corresponding 3D map points is presented. This optimal transformation is then used to produce the final view. The proposed approach was evaluated using both human models and in vivo data. The evaluation results of the proposed correspondence matrix estimation algorithm prove its ability to reduce the error and to produce an accurate transformation. The results also show that when other approaches fail, the proposed approach can produce an expanded view. In this work, a real-time dynamic field-of-view expansion approach that can work in all situations regardless of images’ overlap is proposed. It outperforms the previous approaches and can also work at 21 fps.

scholarly journals Deep Reinforcement Learning by Balancing Offline Monte Carlo and Online Temporal Difference Use Based on Environment Experiences

Symmetry ◽  
2020 ◽  
Vol 12 (10) ◽  
pp. 1685 ◽  
Author(s):  
Chayoung Kim
Keyword(s):  
Monte Carlo ◽  
Reinforcement Learning ◽  
Real Time ◽  
Temporal Difference ◽  
Q Learning ◽  
State Action ◽  
Proposed Model ◽  
Reward Functions ◽  
And Performance ◽  
The Internet Of Things

Owing to the complexity involved in training an agent in a real-time environment, e.g., using the Internet of Things (IoT), reinforcement learning (RL) using a deep neural network, i.e., deep reinforcement learning (DRL) has been widely adopted on an online basis without prior knowledge and complicated reward functions. DRL can handle a symmetrical balance between bias and variance—this indicates that the RL agents are competently trained in real-world applications. The approach of the proposed model considers the combinations of basic RL algorithms with online and offline use based on the empirical balances of bias–variance. Therefore, we exploited the balance between the offline Monte Carlo (MC) technique and online temporal difference (TD) with on-policy (state-action–reward-state-action, Sarsa) and an off-policy (Q-learning) in terms of a DRL. The proposed balance of MC (offline) and TD (online) use, which is simple and applicable without a well-designed reward, is suitable for real-time online learning. We demonstrated that, for a simple control task, the balance between online and offline use without an on- and off-policy shows satisfactory results. However, in complex tasks, the results clearly indicate the effectiveness of the combined method in improving the convergence speed and performance in a deep Q-network.

Localization and Mapping for Indoor Navigation

2020 ◽  
pp. 930-954 ◽  
Author(s):  
Heba Gaber ◽  
Mohamed Marey ◽  
Safaa Amin ◽  
Mohamed F. Tolba
Keyword(s):  
System Architecture ◽  
Large Range ◽  
Simultaneous Localization And Mapping ◽  
Indoor Navigation ◽  
Indoor Environments ◽  
Challenging Problem ◽  
Unknown Environments ◽  
Localization And Mapping ◽  
Robotic Missions

Mapping and exploration for the purpose of navigation in unknown or partially unknown environments is a challenging problem, especially in indoor environments where GPS signals can't give the required accuracy. This chapter discusses the main aspects for designing a Simultaneous Localization and Mapping (SLAM) system architecture with the ability to function in situations where map information or current positions are initially unknown or partially unknown and where environment modifications are possible. Achieving this capability makes these systems significantly more autonomous and ideal for a large range of applications, especially indoor navigation for humans and for robotic missions. This chapter surveys the existing algorithms and technologies used for localization and mapping and highlights on using SLAM algorithms for indoor navigation. Also the proposed approach for the current research is presented.

scholarly journals Active SLAM for Autonomous Underwater Exploration

2019 ◽  
Vol 11 (23) ◽  
pp. 2827 ◽  
Author(s):  
Narcís Palomeras ◽  
Marc Carreras ◽  
Juan Andrade-Cetto
Keyword(s):  
Autonomous Underwater Vehicles ◽  
A Priori ◽  
Underwater Vehicles ◽  
View Planning ◽  
Localization And Mapping ◽  
Entropy Reduction ◽  
Graph Slam ◽  
Slam Algorithm ◽  
Iterative Closest Point Algorithm ◽  
Active Localization

Exploration of a complex underwater environment without an a priori map is beyond the state of the art for autonomous underwater vehicles (AUVs). Despite several efforts regarding simultaneous localization and mapping (SLAM) and view planning, there is no exploration framework, tailored to underwater vehicles, that faces exploration combining mapping, active localization, and view planning in a unified way. We propose an exploration framework, based on an active SLAM strategy, that combines three main elements: a view planner, an iterative closest point algorithm (ICP)-based pose-graph SLAM algorithm, and an action selection mechanism that makes use of the joint map and state entropy reduction. To demonstrate the benefits of the active SLAM strategy, several tests were conducted with the Girona 500 AUV, both in simulation and in the real world. The article shows how the proposed framework makes it possible to plan exploratory trajectories that keep the vehicle’s uncertainty bounded; thus, creating more consistent maps.

scholarly journals An Ant–Based Filtering Random–Finite–Set Approach to Simultaneous Localization and Mapping

2018 ◽  
Vol 28 (3) ◽  
pp. 505-519
Author(s):  
Demeng Li ◽  
Jihong Zhua ◽  
Benlian Xu ◽  
Mingli Lu ◽  
Mingyue Li
Keyword(s):  
Simultaneous Localization And Mapping ◽  
Posterior Density ◽  
Operation System ◽  
Feature Map ◽  
Localization And Mapping ◽  
Finite Set ◽  
Feature Based ◽  
Vehicle Trajectory ◽  
Slam Algorithm ◽  
Random Finite Sets

Abstract Inspired by ant foraging, as well as modeling of the feature map and measurements as random finite sets, a novel formulation in an ant colony framework is proposed to jointly estimate the map and the vehicle trajectory so as to solve a feature-based simultaneous localization and mapping (SLAM) problem. This so-called ant-PHD-SLAM algorithm allows decomposing the recursion for the joint map-trajectory posterior density into a jointly propagated posterior density of the vehicle trajectory and the posterior density of the feature map conditioned on the vehicle trajectory. More specifically, an ant-PHD filter is proposed to jointly estimate the number of map features and their locations, namely, using the powerful search ability and collective cooperation of ants to complete the PHD-SLAM filter time prediction and data update process. Meanwhile, a novel fast moving ant estimator (F-MAE) is utilized to estimate the maneuvering vehicle trajectory. Evaluation and comparison using several numerical examples show a performance improvement over recently reported approaches. Moreover, the experimental results based on the robot operation system (ROS) platform validate the consistency with the results obtained from numerical simulations.

Sign in / Sign up

Export Citation Format

Share Document