Improving heuristic search for RTS-game unit micromanagement using reinforcement learning

2015 ◽  
Author(s):  
Supaphon Kamon ◽  
Tung Due Nguyen ◽  
Tomohiro Harada ◽  
Ruck Thawonmas ◽  
Ikuko Nishikawa
Keyword(s):  
Reinforcement Learning ◽  
Heuristic Search ◽  
Rts Game

scholarly journals Explaining Reinforcement Learning to Mere Mortals: An Empirical Study

2019 ◽  
Author(s):  
Andrew Anderson ◽  
Jonathan Dodge ◽  
Amrita Sadarangani ◽  
Zoe Juozapaitis ◽  
Evan Newman ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Empirical Study ◽  
Mental Models ◽  
Mental Model ◽  
User Study ◽  
Focus Of Attention ◽  
Saliency Maps ◽  
Treatment Experiment ◽  
Rts Game ◽  
The Impact

We present a user study to investigate the impact of explanations on non-experts? understanding of reinforcement learning (RL) agents. We investigate both a common RL visualization, saliency maps (the focus of attention), and a more recent explanation type, reward-decomposition bars (predictions of future types of rewards). We designed a 124 participant, four-treatment experiment to compare participants? mental models of an RL agent in a simple Real-Time Strategy (RTS) game. Our results show that the combination of both saliency and reward bars were needed to achieve a statistically significant improvement in mental model score over the control. In addition, our qualitative analysis of the data reveals a number of effects for further study.

scholarly journals Playing Card-Based RTS Games with Deep Reinforcement Learning

2019 ◽  
Author(s):  
Tianyu Liu ◽  
Zijie Zheng ◽  
Hongchang Li ◽  
Kaigui Bian ◽  
Lingyang Song
Keyword(s):  
Reinforcement Learning ◽  
Decision Tree ◽  
State Space ◽  
Imperfect Information ◽  
Rule Based ◽  
Playing Card ◽  
Board Games ◽  
Deep Model ◽  
Game Ai ◽  
Rts Game

Game AI is of great importance as games are simulations of reality. Recent research on game AI has shown much progress in various kinds of games, such as console games, board games and MOBA games. However, the exploration in RTS games remains a challenge for their huge state space, imperfect information, sparse rewards and various strategies. Besides, the typical card-based RTS games have complex card features and are still lacking solutions. We present a deep model SEAT (selection-attention) to play card-based RTS games. The SEAT model includes two parts, a selection part for card choice and an attention part for card usage, and it learns from scratch via deep reinforcement learning. Comprehensive experiments are performed on Clash Royale, a popular mobile card-based RTS game. Empirical results show that the SEAT model agent makes it to reach a high winning rate against rule-based agents and decision-tree-based agent.

Improving RTS Game AI by Supervised Policy Learning, Tactical Search, and Deep Reinforcement Learning

2019 ◽  
Vol 14 (3) ◽  
pp. 8-18 ◽  
Author(s):  
Nicolas A. Barriga ◽  
Marius Stanescu ◽  
Felipe Besoain ◽  
Michael Buro
Keyword(s):  
Reinforcement Learning ◽  
Policy Learning ◽  
Game Ai ◽  
Rts Game

Model-based deep reinforcement learning with heuristic search for satellite attitude control

2019 ◽  
Vol 46 (3) ◽  
pp. 415-420
Author(s):  
Ke Xu ◽  
Fengge Wu ◽  
Junsuo Zhao
Keyword(s):  
Reinforcement Learning ◽  
Heuristic Search ◽  
Attitude Control ◽  
Local Optima ◽  
Content Type ◽  
Model Based ◽  
Model Free ◽  
Satellite Attitude ◽  
Satellite Attitude Control ◽  
Classical Control

Purpose Recently, deep reinforcement learning is developing rapidly and shows its power to solve difficult problems such as robotics and game of GO. Meanwhile, satellite attitude control systems are still using classical control technics such as proportional – integral – derivative and slide mode control as major solutions, facing problems with adaptability and automation. Design/methodology/approach In this paper, an approach based on deep reinforcement learning is proposed to increase adaptability and autonomy of satellite control system. It is a model-based algorithm which could find solutions with fewer episodes of learning than model-free algorithms. Findings Simulation experiment shows that when classical control crashed, this approach could find solution and reach the target with hundreds times of explorations and learning. Originality/value This approach is a non-gradient method using heuristic search to optimize policy to avoid local optima. Compared with classical control technics, this approach does not need prior knowledge of satellite or its orbit, has the ability to adapt different kinds of situations with data learning and has the ability to adapt different kinds of satellite and different tasks through transfer learning.

Q-Table compression for reinforcement learning

2018 ◽  
Vol 33 ◽  
Author(s):  
Leonardo Amado ◽  
Felipe Meneguzzi
Keyword(s):  
Reinforcement Learning ◽  
State Space ◽  
Real Time ◽  
Prior Knowledge ◽  
Q Value ◽  
State Spaces ◽  
Factor Problem ◽  
Branching Factor ◽  
Rts Game

AbstractReinforcement learning (RL) algorithms are often used to compute agents capable of acting in environments without prior knowledge of the environment dynamics. However, these algorithms struggle to converge in environments with large branching factors and their large resulting state-spaces. In this work, we develop an approach to compress the number of entries in a Q-value table using a deep auto-encoder. We develop a set of techniques to mitigate the large branching factor problem. We present the application of such techniques in the scenario of a real-time strategy (RTS) game, where both state space and branching factor are a problem. We empirically evaluate an implementation of the technique to control agents in an RTS game scenario where classical RL fails and provide a number of possible avenues of further work on this problem.

scholarly journals Heuristic Search Exploiting Non-additive and Unit Properties for RTS-game Unit Micromanagement

2015 ◽  
Vol 23 (1) ◽  
pp. 2-8 ◽  
Author(s):  
Tung Duc Nguyen ◽  
Kien Quang Nguyen ◽  
Ruck Thawonmas
Keyword(s):  
Heuristic Search ◽  
Rts Game

A HYBRID ALGORITHM FOR FORMING THE SHORTEST TRAJECTORY BASED ON THE APPLICATION OF MULTI-AGENT LEARNING WITH REINFORCEMENT, THE SEARCH ALGORITHM A* AND EXCHANGE OF EXPERIENCE

2021 ◽  
pp. 13-26
Author(s):  
Yu. V. Dubenko ◽  
E. E. Dyshkant ◽  
N. N. Timchenko ◽  
N. A. Rudeshko
Keyword(s):  
Reinforcement Learning ◽  
Heuristic Search ◽  
Intelligent Agents ◽  
Hybrid Algorithm ◽  
Search Algorithm ◽  
Heuristic Search Algorithm ◽  
Agent Learning ◽  
Multi Agent ◽  
Multi Level ◽  
Automatic Formation

The article presents a hybrid algorithm for the formation of the shortest trajectory for intelligent agents of a multi-agent system, based on the synthesis of methods of the reinforcement learning paradigm, the heuristic search algorithm A*, which has the functions of exchange of experience, as well as the automatic formation of subgroups of agents based on their visibility areas. The experimental evaluation of the developed algorithm was carried out by simulating the task of finding the target state in the maze in the Microsoft Unity environment. The results of the experiment showed that the use of the developed hybrid algorithm made it possible to reduce the time for solving the problem by an average of 12.7 % in comparison with analogs. The differences between the proposed new “hybrid algorithm for the formation of the shortest trajectory based on the use of multi-agent reinforcement learning, search algorithm A* and exchange of experience” from analogs are as follows: – application of the algorithm for the formation of subgroups of subordinate agents based on the “scope” of the leader agent for the implementation of a multi-level hierarchical system for managing a group of agents; – combining the principles of reinforcement learning and the search algorithm A*.

Sign in / Sign up

Export Citation Format

Share Document