Hierarchical Architecture with Modular Network SOM and Modular Reinforcement Learning

2009 ◽  
pp. 546-556
Author(s):  
Masumi Ishikawa ◽  
Kosuke Ueno
Keyword(s):  
Reinforcement Learning ◽  
Hierarchical Architecture ◽  
Modular Network

scholarly journals On Reinforcement Learning for Full-Length Game of StarCraft

2019 ◽  
Vol 33 ◽  
pp. 4691-4698 ◽  
Author(s):  
Zhen-Jia Pang ◽  
Ruo-Ze Liu ◽  
Zhou-Yu Meng ◽  
Yi Zhang ◽  
Yang Yu ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Transfer Learning ◽  
Large Scale ◽  
Learning Algorithm ◽  
Full Length ◽  
Action Space ◽  
Hierarchical Architecture ◽  
Difficulty Level ◽  
Future Research ◽  
Grand Challenge

StarCraft II poses a grand challenge for reinforcement learning. The main difficulties include huge state space, varying action space, long horizon, etc. In this paper, we investigate a set of techniques of reinforcement learning for the full-length game of StarCraft II. We investigate a hierarchical approach, where the hierarchy involves two levels of abstraction. One is the macro-actions extracted from expert’s demonstration trajectories, which can reduce the action space in an order of magnitude yet remain effective. The other is a two-layer hierarchical architecture, which is modular and easy to scale. We also investigate a curriculum transfer learning approach that trains the agent from the simplest opponent to harder ones. On a 64×64 map and using restrictive units, we train the agent on a single machine with 4 GPUs and 48 CPU threads. We achieve a winning rate of more than 99% against the difficulty level-1 built-in AI. Through the curriculum transfer learning algorithm and a mixture of combat model, we can achieve over 93% winning rate against the most difficult noncheating built-in AI (level-7) within days. We hope this study could shed some light on the future research of large-scale reinforcement learning.

scholarly journals Habitual control of goal selection in humans

2015 ◽  
Vol 112 (45) ◽  
pp. 13817-13822 ◽  
Author(s):  
Fiery Cushman ◽  
Adam Morris
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Computational Efficiency ◽  
Behavioral Control ◽  
Hierarchical Architecture ◽  
Sequential Decision Making ◽  
Costs And Benefits ◽  
Sequential Decision ◽  
Goal Selection ◽  
Computationally Expensive

Humans choose actions based on both habit and planning. Habitual control is computationally frugal but adapts slowly to novel circumstances, whereas planning is computationally expensive but can adapt swiftly. Current research emphasizes the competition between habits and plans for behavioral control, yet many complex tasks instead favor their integration. We consider a hierarchical architecture that exploits the computational efficiency of habitual control to select goals while preserving the flexibility of planning to achieve those goals. We formalize this mechanism in a reinforcement learning setting, illustrate its costs and benefits, and experimentally demonstrate its spontaneous application in a sequential decision-making task.

Bayes factors for reinforcement-learning models of the Iowa gambling task.

Decision ◽  
2016 ◽  
Vol 3 (2) ◽  
pp. 115-131 ◽  
Author(s):  
Helen Steingroever ◽  
Ruud Wetzels ◽  
Eric-Jan Wagenmakers
Keyword(s):  
Reinforcement Learning ◽  
Iowa Gambling Task ◽  
Bayes Factors ◽  
Gambling Task ◽  
Learning Models ◽  
Reinforcement Learning Models
Sign in / Sign up

Export Citation Format

Share Document