scholarly journals Fast Transfer Navigation for Autonomous Robots

2021 ◽  
Vol 2021 ◽  
pp. 1-7
Author(s):  
Chen Wang ◽  
Xudong Li ◽  
Xiaolin Tao ◽  
Kai Ling ◽  
Quhui Liu ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Indoor Environment ◽  
Autonomous Robots ◽  
Indoor Navigation ◽  
Robotic Navigation ◽  
Navigation Algorithm ◽  
Proposed Model ◽  
Interior Environment

Navigation technology enables indoor robots to arrive at their destinations safely. Generally, the varieties of the interior environment contribute to the difficulty of robotic navigation and hurt their performance. This paper proposes a transfer navigation algorithm and improves its generalization by leveraging deep reinforcement learning and a self-attention module. To simulate the unfurnished indoor environment, we build the virtual indoor navigation (VIN) environment to compare our model and its competitors. In the VIN environment, our method outperforms other algorithms by adapting to an unseen indoor environment. The code of the proposed model and the virtual indoor navigation environment will be released.

scholarly journals Data Model for IndoorGML Extension to Support Indoor Navigation of People with Mobility Disabilities

2020 ◽  
Vol 9 (2) ◽  
pp. 66 ◽  
Author(s):  
Seula Park ◽  
Kiyun Yu ◽  
Jiyoung Kim
Keyword(s):  
Data Model ◽  
Indoor Environment ◽  
Spatial Information ◽  
People With Disabilities ◽  
Indoor Navigation ◽  
Shopping Mall ◽  
Data Generation ◽  
Proposed Model ◽  
Model Features ◽  
Indoor Spaces

The increasing complexity of modern buildings has challenged the mobility of people with disabilities (PWD) in the indoor environment. To help overcome this problem, this paper proposes a data model that can be easily applied to indoor spatial information services for people with disabilities. In the proposed model, features are defined based on relevant regulations that stipulate significant mobility factors for people with disabilities. To validate the model’s capability to describe the indoor spaces in terms that are relevant to people with mobility disabilities, the model was used to generate data in a path planning application, considering two different cases in a shopping mall. The application confirmed that routes for people with mobility disabilities are significantly different from those of ordinary pedestrians, in a way that reflects features and attributes defined in the proposed data model. The latter can be inserted as an IndoorGML extension, and is thus expected to facilitate relevant data generation for the design of various services for people with disabilities.

An adaptive localization system for outdoor/indoor navigation for autonomous robots

2006 ◽  
Author(s):  
E. B. Pacis ◽  
B. Sights ◽  
G. Ahuja ◽  
G. Kogut ◽  
H. R. Everett
Keyword(s):  
Autonomous Robots ◽  
Indoor Navigation ◽  
Localization System

scholarly journals A strategy learning model for autonomous agents based on classification

2015 ◽  
Vol 25 (3) ◽  
pp. 471-482 ◽  
Author(s):  
Bartłomiej Śnieżyński
Keyword(s):  
Reinforcement Learning ◽  
Supervised Learning ◽  
Learning Process ◽  
Autonomous Agents ◽  
Good Alternative ◽  
Learning Model ◽  
Learning Method ◽  
Complex Environments ◽  
Agent Based ◽  
Proposed Model

AbstractIn this paper we propose a strategy learning model for autonomous agents based on classification. In the literature, the most commonly used learning method in agent-based systems is reinforcement learning. In our opinion, classification can be considered a good alternative. This type of supervised learning can be used to generate a classifier that allows the agent to choose an appropriate action for execution. Experimental results show that this model can be successfully applied for strategy generation even if rewards are delayed. We compare the efficiency of the proposed model and reinforcement learning using the farmer-pest domain and configurations of various complexity. In complex environments, supervised learning can improve the performance of agents much faster that reinforcement learning. If an appropriate knowledge representation is used, the learned knowledge may be analyzed by humans, which allows tracking the learning process

scholarly journals Multi-Agent Reinforcement Learning for Optimizing Traffic Signal Timing

2021 ◽  
Author(s):  
Areej Salaymeh ◽  
Loren Schwiebert ◽  
Stephen Remias
Keyword(s):  
Reinforcement Learning ◽  
Traffic Signals ◽  
Transportation Systems ◽  
Traffic Signal ◽  
Urban Traffic ◽  
Signal Timing ◽  
Model Free ◽  
Proposed Model ◽  
Multi Agent ◽  
Traffic Signal Timing

Designing efficient transportation systems is crucial to save time and money for drivers and for the economy as whole. One of the most important components of traffic systems are traffic signals. Currently, most traffic signal systems are configured using fixed timing plans, which are based on limited vehicle count data. Past research has introduced and designed intelligent traffic signals; however, machine learning and deep learning have only recently been used in systems that aim to optimize the timing of traffic signals in order to reduce travel time. A very promising field in Artificial Intelligence is Reinforcement Learning. Reinforcement learning (RL) is a data driven method that has shown promising results in optimizing traffic signal timing plans to reduce traffic congestion. However, model-based and centralized methods are impractical here due to the high dimensional state-action space in complex urban traffic network. In this paper, a model-free approach is used to optimize signal timing for complicated multiple four-phase signalized intersections. We propose a multi-agent deep reinforcement learning framework that aims to optimize traffic flow using data within traffic signal intersections and data coming from other intersections in a Multi-Agent Environment in what is called Multi-Agent Reinforcement Learning (MARL). The proposed model consists of state-of-art techniques such as Double Deep Q-Network and Hindsight Experience Replay (HER). This research uses HER to allow our framework to quickly learn on sparse reward settings. We tested and evaluated our proposed model via a Simulation of Urban MObility simulation (SUMO). Our results show that the proposed method is effective in reducing congestion in both peak and off-peak times.

Robot hand-eye cooperation based on improved inverse reinforcement learning

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Ning Yu ◽  
Lin Nan ◽  
Tao Ku
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Visual Information ◽  
Industrial Robots ◽  
Robot Hand ◽  
Inverse Reinforcement Learning ◽  
Generative Adversarial Network ◽  
Content Type ◽  
Coordination Model ◽  
Proposed Model

Purpose How to make accurate action decisions based on visual information is one of the important research directions of industrial robots. The purpose of this paper is to design a highly optimized hand-eye coordination model of the robot to improve the robots’ on-site decision-making ability. Design/methodology/approach The combination of inverse reinforcement learning (IRL) algorithm and generative adversarial network can effectively reduce the dependence on expert samples and robots can obtain the decision-making performance that the degree of optimization is not lower than or even higher than that of expert samples. Findings The performance of the proposed model is verified in the simulation environment and real scene. By monitoring the reward distribution of the reward function and the trajectory of the robot, the proposed model is compared with other existing methods. The experimental results show that the proposed model has better decision-making performance in the case of less expert data. Originality/value A robot hand-eye cooperation model based on improved IRL is proposed and verified. Empirical investigations on real experiments reveal that overall, the proposed approach tends to improve the real efficiency by more than 10% when compared to alternative hand-eye cooperation methods.

scholarly journals Learning an Efficient Gait Cycle of a Biped Robot Based on Reinforcement Learning and Artificial Neural Networks

2019 ◽  
Vol 9 (3) ◽  
pp. 502 ◽  
Author(s):  
Cristyan Gil ◽  
Hiram Calvo ◽  
Humberto Sossa
Keyword(s):  
Reinforcement Learning ◽  
Gait Cycle ◽  
Biped Robot ◽  
Q Learning ◽  
Nao Robot ◽  
Simulated Environment ◽  
Proposed Model ◽  
Normal Speed ◽  
Multi Level ◽  
Efficient Gait

Programming robots for performing different activities requires calculating sequences of values of their joints by taking into account many factors, such as stability and efficiency, at the same time. Particularly for walking, state of the art techniques to approximate these sequences are based on reinforcement learning (RL). In this work we propose a multi-level system, where the same RL method is used first to learn the configuration of robot joints (poses) that allow it to stand with stability, and then in the second level, we find the sequence of poses that let it reach the furthest distance in the shortest time, while avoiding falling down and keeping a straight path. In order to evaluate this, we focus on measuring the time it takes for the robot to travel a certain distance. To our knowledge, this is the first work focusing both on speed and precision of the trajectory at the same time. We implement our model in a simulated environment using q-learning. We compare with the built-in walking modes of an NAO robot by improving normal-speed and enhancing robustness in fast-speed. The proposed model can be extended to other tasks and is independent of a particular robot model.

scholarly journals Deep Reinforcement Learning by Balancing Offline Monte Carlo and Online Temporal Difference Use Based on Environment Experiences

Symmetry ◽  
2020 ◽  
Vol 12 (10) ◽  
pp. 1685 ◽  
Author(s):  
Chayoung Kim
Keyword(s):  
Monte Carlo ◽  
Reinforcement Learning ◽  
Real Time ◽  
Temporal Difference ◽  
Q Learning ◽  
State Action ◽  
Proposed Model ◽  
Reward Functions ◽  
And Performance ◽  
The Internet Of Things

Owing to the complexity involved in training an agent in a real-time environment, e.g., using the Internet of Things (IoT), reinforcement learning (RL) using a deep neural network, i.e., deep reinforcement learning (DRL) has been widely adopted on an online basis without prior knowledge and complicated reward functions. DRL can handle a symmetrical balance between bias and variance—this indicates that the RL agents are competently trained in real-world applications. The approach of the proposed model considers the combinations of basic RL algorithms with online and offline use based on the empirical balances of bias–variance. Therefore, we exploited the balance between the offline Monte Carlo (MC) technique and online temporal difference (TD) with on-policy (state-action–reward-state-action, Sarsa) and an off-policy (Q-learning) in terms of a DRL. The proposed balance of MC (offline) and TD (online) use, which is simple and applicable without a well-designed reward, is suitable for real-time online learning. We demonstrated that, for a simple control task, the balance between online and offline use without an on- and off-policy shows satisfactory results. However, in complex tasks, the results clearly indicate the effectiveness of the combined method in improving the convergence speed and performance in a deep Q-network.

scholarly journals Deep Learning-Based Positioning of Visually Impaired People in Indoor Environments

Sensors ◽  
2020 ◽  
Vol 20 (21) ◽  
pp. 6238
Author(s):  
Payal Mahida ◽  
Seyed Shahrestani ◽  
Hon Cheung
Keyword(s):  
Navigation System ◽  
Visually Impaired ◽  
Indoor Environment ◽  
Inertial Sensor ◽  
User Interaction ◽  
Indoor Navigation ◽  
Sensor Data ◽  
Indoor Environments ◽  
Visually Impaired People ◽  
Indoor Navigation System

Wayfinding and navigation can present substantial challenges to visually impaired (VI) people. Some of the significant aspects of these challenges arise from the difficulty of knowing the location of a moving person with enough accuracy. Positioning and localization in indoor environments require unique solutions. Furthermore, positioning is one of the critical aspects of any navigation system that can assist a VI person with their independent movement. The other essential features of a typical indoor navigation system include pathfinding, obstacle avoidance, and capabilities for user interaction. This work focuses on the positioning of a VI person with enough precision for their use in indoor navigation. We aim to achieve this by utilizing only the capabilities of a typical smartphone. More specifically, our proposed approach is based on the use of the accelerometer, gyroscope, and magnetometer of a smartphone. We consider the indoor environment to be divided into microcells, with the vertex of each microcell being assigned two-dimensional local coordinates. A regression-based analysis is used to train a multilayer perceptron neural network to map the inertial sensor measurements to the coordinates of the vertex of the microcell corresponding to the position of the smartphone. In order to test our proposed solution, we used IPIN2016, a publicly-available multivariate dataset that divides the indoor environment into cells tagged with the inertial sensor data of a smartphone, in order to generate the training and validating sets. Our experiments show that our proposed approach can achieve a remarkable prediction accuracy of more than 94%, with a 0.65 m positioning error.

scholarly journals Optimizing the Pairs-Trading Strategy Using Deep Reinforcement Learning with Trading and Stop-Loss Boundaries

Complexity ◽  
2019 ◽  
Vol 2019 ◽  
pp. 1-20 ◽  
Author(s):  
Taewook Kim ◽  
Ha Young Kim
Keyword(s):  
Reinforcement Learning ◽  
Trading Strategy ◽  
Trading Strategies ◽  
Cointegration Test ◽  
Optimum Level ◽  
Pairs Trading ◽  
Proposed Model ◽  
The Mean ◽  
Stop Loss ◽  
The Given

Many researchers have tried to optimize pairs trading as the numbers of opportunities for arbitrage profit have gradually decreased. Pairs trading is a market-neutral strategy; it profits if the given condition is satisfied within a given trading window, and if not, there is a risk of loss. In this study, we propose an optimized pairs-trading strategy using deep reinforcement learning—particularly with the deep Q-network—utilizing various trading and stop-loss boundaries. More specifically, if spreads hit trading thresholds and reverse to the mean, the agent receives a positive reward. However, if spreads hit stop-loss thresholds or fail to reverse to the mean after hitting the trading thresholds, the agent receives a negative reward. The agent is trained to select the optimum level of discretized trading and stop-loss boundaries given a spread to maximize the expected sum of discounted future profits. Pairs are selected from stocks on the S&P 500 Index using a cointegration test. We compared our proposed method with traditional pairs-trading strategies which use constant trading and stop-loss boundaries. We find that our proposed model is trained well and outperforms traditional pairs-trading strategies.

Sign in / Sign up

Export Citation Format

Share Document