Royale Heroes: A Unique RTS Game Using Deep Reinforcement Learning-based Autonomous Movement

We present a user study to investigate the impact of explanations on non-experts? understanding of reinforcement learning (RL) agents. We investigate both a common RL visualization, saliency maps (the focus of attention), and a more recent explanation type, reward-decomposition bars (predictions of future types of rewards). We designed a 124 participant, four-treatment experiment to compare participants? mental models of an RL agent in a simple Real-Time Strategy (RTS) game. Our results show that the combination of both saliency and reward bars were needed to achieve a statistically significant improvement in mental model score over the control. In addition, our qualitative analysis of the data reveals a number of effects for further study.

Download Full-text

Playing Card-Based RTS Games with Deep Reinforcement Learning

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2019/631 ◽

2019 ◽

Author(s):

Tianyu Liu ◽

Zijie Zheng ◽

Hongchang Li ◽

Kaigui Bian ◽

Lingyang Song

Keyword(s):

Reinforcement Learning ◽

Decision Tree ◽

State Space ◽

Imperfect Information ◽

Rule Based ◽

Playing Card ◽

Board Games ◽

Deep Model ◽

Game Ai ◽

Rts Game

Game AI is of great importance as games are simulations of reality. Recent research on game AI has shown much progress in various kinds of games, such as console games, board games and MOBA games. However, the exploration in RTS games remains a challenge for their huge state space, imperfect information, sparse rewards and various strategies. Besides, the typical card-based RTS games have complex card features and are still lacking solutions. We present a deep model SEAT (selection-attention) to play card-based RTS games. The SEAT model includes two parts, a selection part for card choice and an attention part for card usage, and it learns from scratch via deep reinforcement learning. Comprehensive experiments are performed on Clash Royale, a popular mobile card-based RTS game. Empirical results show that the SEAT model agent makes it to reach a high winning rate against rule-based agents and decision-tree-based agent.

Download Full-text

Improving RTS Game AI by Supervised Policy Learning, Tactical Search, and Deep Reinforcement Learning

IEEE Computational Intelligence Magazine ◽

10.1109/mci.2019.2919363 ◽

2019 ◽

Vol 14 (3) ◽

pp. 8-18 ◽

Cited By ~ 4

Author(s):

Nicolas A. Barriga ◽

Marius Stanescu ◽

Felipe Besoain ◽

Michael Buro

Keyword(s):

Reinforcement Learning ◽

Policy Learning ◽

Game Ai ◽

Rts Game

Download Full-text

Q-Table compression for reinforcement learning

The Knowledge Engineering Review ◽

10.1017/s0269888918000280 ◽

2018 ◽

Vol 33 ◽

Cited By ~ 1

Author(s):

Leonardo Amado ◽

Felipe Meneguzzi

Keyword(s):

Reinforcement Learning ◽

State Space ◽

Real Time ◽

Prior Knowledge ◽

Q Value ◽

State Spaces ◽

Factor Problem ◽

Branching Factor ◽

Rts Game

AbstractReinforcement learning (RL) algorithms are often used to compute agents capable of acting in environments without prior knowledge of the environment dynamics. However, these algorithms struggle to converge in environments with large branching factors and their large resulting state-spaces. In this work, we develop an approach to compress the number of entries in a Q-value table using a deep auto-encoder. We develop a set of techniques to mitigate the large branching factor problem. We present the application of such techniques in the scenario of a real-time strategy (RTS) game, where both state space and branching factor are a problem. We empirically evaluate an implementation of the technique to control agents in an RTS game scenario where classical RL fails and provide a number of possible avenues of further work on this problem.

Download Full-text