Anticipatory Classifier System with Average Reward Criterion in Discretized Multi-Step Environments

Initially, Anticipatory Classifier Systems (ACS) were designed to address both single and multistep decision problems. In the latter case, the objective was to maximize the total discounted rewards, usually based on Q-learning algorithms. Studies on other Learning Classifier Systems (LCS) revealed many real-world sequential decision problems where the preferred objective is the maximization of the average of successive rewards. This paper proposes a relevant modification toward the learning component, allowing us to address such problems. The modified system is called AACS2 (Averaged ACS2) and is tested on three multistep benchmark problems.

On Finding Large Sets of Rewards in Two-Player ETP–ESP Games

Games with endogenous transition probabilities and endogenous stage payoffs (or ETP–ESP games for short) are stochastic games in which both the transition probabilities and the payoffs at any stage are continuous functions of the relative frequencies of all past action combinations chosen. We present methods to compute large sets of jointly-convergent pure-strategy rewards in two-player ETP–ESP games with communicating states under the limiting average reward criterion. Such sets are useful in determining feasible rewards in a game, and instrumental in obtaining the set of (Nash) equilibrium rewards.