Navigation system with SLAM-based trajectory topological map and reinforcement learning-based local planner

Extensive studies have shown that many animals’ capability of forming spatial representations for self-localization, path planning, and navigation relies on the functionalities of place and head-direction (HD) cells in the hippocampus. Although there are numerous hippocampal modeling approaches, only a few span the wide functionalities ranging from processing raw sensory signals to planning and action generation. This paper presents a vision-based navigation system that involves generating place and HD cells through learning from visual images, building topological maps based on learned cell representations and performing navigation using hierarchical reinforcement learning. First, place and HD cells are trained from sequences of visual stimuli in an unsupervised learning fashion. A modified Slow Feature Analysis (SFA) algorithm is proposed to learn different cell types in an intentional way by restricting their learning to separate phases of the spatial exploration. Then, to extract the encoded metric information from these unsupervised learning representations, a self-organized learning algorithm is adopted to learn over the emerged cell activities and to generate topological maps that reveal the topology of the environment and information about a robot’s head direction, respectively. This enables the robot to perform self-localization and orientation detection based on the generated maps. Finally, goal-directed navigation is performed using reinforcement learning in continuous state spaces which are represented by the population activities of place cells. In particular, considering that the topological map provides a natural hierarchical representation of the environment, hierarchical reinforcement learning (HRL) is used to exploit this hierarchy to accelerate learning. The HRL works on different spatial scales, where a high-level policy learns to select subgoals and a low-level policy learns over primitive actions to specialize on the selected subgoals. Experimental results demonstrate that our system is able to navigate a robot to the desired position effectively, and the HRL shows a much better learning performance than the standard RL in solving our navigation tasks.

Download Full-text

Algorithm of Topological Map Generation for the EDN Navigation System

IFAC Proceedings Volumes ◽

10.1016/s1474-6670(17)34308-2 ◽

2001 ◽

Vol 34 (4) ◽

pp. 275-280 ◽

Cited By ~ 1

Author(s):

R. Barber ◽

V. Egido ◽

M.A. Salichs

Keyword(s):

Navigation System ◽

Topological Map ◽

Map Generation

Download Full-text

Topological Navigation and Localization in Tunnels—Application to Autonomous Load-Haul-Dump Vehicles Operating in Underground Mines

Applied Sciences ◽

10.3390/app11146547 ◽

2021 ◽

Vol 11 (14) ◽

pp. 6547

Author(s):

Mauricio Mascaró ◽

Isao Parra-Tsunekawa ◽

Carlos Tampier ◽

Javier Ruiz-del-Solar

Keyword(s):

Navigation System ◽

Autonomous Vehicles ◽

Underground Mines ◽

Topological Map ◽

Automation Systems ◽

Scale Models ◽

Controlled Environments ◽

Mining Vehicles ◽

Topological Navigation ◽

Localization Systems

Mobile robots are no longer used exclusively in research laboratories and indoor controlled environments, but are now also used in dynamic industrial environments, including outdoor sites. Mining is one industry where robots and autonomous vehicles are increasingly used to increase the safety of the workers, as well as to augment the productivity, efficiency, and predictability of the processes. Since autonomous vehicles navigate inside tunnels in underground mines, this kind of navigation has different precision requirements than navigating in an open environment. When driving inside tunnels, it is not relevant to have accurate self-localization, but it is necessary for autonomous vehicles to be able to move safely through the tunnel and to make appropriate decisions at its intersections and access points in the tunnel. To address these needs, a topological navigation system for mining vehicles operating in tunnels is proposed and validated in this paper. This system was specially designed to be used by Load-Haul-Dump (LHD) vehicles, also known as scoop trams, operating in underground mines. In addition, a localization system, specifically designed to be used with the topological navigation system and its associated topological map, is also proposed. The proposed topological navigation and localization systems were validated using a commercial LHD during several months at a copper sub-level stoping mine located in the Coquimbo Region in the northern part of Chile. An important aspect to be addressed when working with heavy-duty machinery, such as LHDs, is the way in which automation systems are developed and tested. For this reason, the development and testing methodology, which includes the use of simulators, scale-models of LHDs, validation, and testing using a commercial LHD in test-fields, and its final validation in a mine, are described.

Download Full-text