Anticipations Control Behavior: Animal Behavior in an Anticipatory Learning Classifier System

2002 ◽  
Vol 10 (2) ◽  
pp. 75-96 ◽  
Author(s):  
Martin V Butz ◽  
Joachim Hoffmann
Keyword(s):  
Reinforcement Learning ◽  
Adaptive Learning ◽  
Behavioral Control ◽  
Learning Classifier System ◽  
Classifier System ◽  
Model Free ◽  
Learning Classifier ◽  
Controlling Behavior ◽  
On Line ◽  
Adaptive Learning Systems

The concept of anticipations controlling behavior is introduced. Background is provided about the importance of anticipations from a psychological perspective. Based on the psychological background wrapped in a framework of anticipatory behavioral control, the anticipatory learning classifier system ACS2 is explained. ACS2 learns and generalizes on-line a predictive environmental model (a model that allows the prediction of future environmental states). The model is a subjective model, that is, no global state information is available to the agent. It is shown that ACS2 can simulate anticipatory learning processes and anticipatory controlled behavior by means of the model. The simulations of various rat experiments, previously conducted by Colwill and Rescorla, show that the incorporation of anticipations is indeed crucial for simulating the behavior observed in rats. Despite the simplicity of the tasks, we show that the observed behavior reaches beyond the capabilities of model-free reinforcement learning as well as model-based reinforcement learning without on-line generalization. Possible future impacts of anticipations in adaptive learning systems are outlined.

SELF-ADAPTIVE LEARNING CLASSIFIER SYSTEM

2010 ◽  
Vol 19 (01) ◽  
pp. 275-296 ◽  
Author(s):  
OLGIERD UNOLD
Keyword(s):  
Mutation Rate ◽  
Adaptive Learning ◽  
Adaptive Mutation ◽  
Learning Classifier System ◽  
Classifier System ◽  
Learning Classifier ◽  
Adaptive Parameters ◽  
Proposed Model ◽  
Fixed Parameter ◽  
Self Adaptive

This article introduces a new kind of self-adaptation in discovery mechanism of learning classifier system XCS. Unlike the previous approaches, which incorporate self-adaptive parameters in the representation of an individual, proposed model evolves competitive population of the reduced XCSs, which are able to adapt both classifiers and genetic parameters. The experimental comparisons of self-adaptive mutation rate XCS and standard XCS interacting with 11-bit, 20-bit, and 37-bit multiplexer environment were provided. It has been shown that adapting the mutation rate can give an equivalent or better performance to known good fixed parameter settings, especially for computationally complex tasks. Moreover, the self-adaptive XCS is able to solve the problem of inappropriate for a standard XCS parameters.

Analyzing Strength-Based Classifier System from Reinforcement Learning Perspective

2009 ◽  
Vol 13 (6) ◽  
pp. 631-639
Author(s):  
Atsushi Wada ◽  
◽  
Keiki Takadama ◽  
◽  
Keyword(s):  
Reinforcement Learning ◽  
Adaptive Systems ◽  
Classifier Systems ◽  
Q Learning ◽  
State Action ◽  
Classifier System ◽  
Learning Classifier ◽  
Value Estimation ◽  
On Line ◽  
On Line Learning

Learning Classifier Systems (LCSs) are rule-based adaptive systems that have both Reinforcement Learning (RL) and rule-discovery mechanisms for effective and practical on-line learning. With the aim of establishing a common theoretical basis between LCSs and RL algorithms to share each field's findings, a detailed analysis was performed to compare the learning processes of these two approaches. Based on our previous work on deriving an equivalence between the Zeroth-level Classifier System (ZCS) and Q-learning with Function Approximation (FA), this paper extends the analysis to the influence of actually applying the conditions for this equivalence. Comparative experiments have revealed interesting implications: (1) ZCS's original parameter, the deduction rate, plays a role in stabilizing the action selection, but (2) from the Reinforcement Learning perspective, such a process inhibits the ability to accurately estimate values for the entire state-action space, thus limiting the performance of ZCS in problems requiring accurate value estimation.

Robot Reinforcement Learning Based on LCS-GDM

2013 ◽  
Vol 347-350 ◽  
pp. 416-420
Author(s):  
Yong Bin Ma
Keyword(s):  
Reinforcement Learning ◽  
Learning System ◽  
Learning Ability ◽  
Learning Classifier System ◽  
Learning Space ◽  
Rule Based ◽  
Classifier System ◽  
System A ◽  
Learning Classifier ◽  
Learning Rules

This paper proposed a robot reinforcement learning method based on learning classifier system. A learning Classifier System is a rule-based machine learning system that combines reinforcement learning and genetic algorithms. The reinforcement learning component is responsible for adjusting the strength of rules in the system according to some reward obtained from the environment. The genetic algorithm acts as an innovation discovery component which is responsible for discovering new better learning rules. The advantages of this approach are its rule-based representation, which can be easily reduce learning space, online learning ability, robustness .

Sign in / Sign up

Export Citation Format

Share Document