The policy iteration algorithm for average reward Markov decision processes with general state space

1997 ◽  
Vol 42 (12) ◽  
pp. 1663-1680 ◽  
Author(s):  
S.P. Meyn
Keyword(s):  
State Space ◽  
Markov Decision Processes ◽  
Policy Iteration ◽  
Decision Processes ◽  
Iteration Algorithm ◽  
General State ◽  
Average Reward ◽  
Markov Decision ◽  
Policy Iteration Algorithm ◽  
General State Space

scholarly journals Policy Iteration for Continuous-Time Average Reward Markov Decision Processes in Polish Spaces

2009 ◽  
Vol 2009 ◽  
pp. 1-17 ◽  
Author(s):  
Quanxin Zhu ◽  
Xinsong Yang ◽  
Chuangxia Huang
Keyword(s):  
Markov Decision Processes ◽  
Continuous Time ◽  
Policy Iteration ◽  
Decision Processes ◽  
Iteration Algorithm ◽  
Average Reward ◽  
Stationary Policy ◽  
Optimality Equation ◽  
Markov Decision ◽  
Average Reward Optimality

We study thepolicy iteration algorithm(PIA) for continuous-time jump Markov decision processes in general state and action spaces. The corresponding transition rates are allowed to beunbounded, and the reward rates may haveneither upper nor lower bounds. The criterion that we are concerned with isexpected average reward. We propose a set of conditions under which we first establish the average reward optimality equation and present the PIA. Then under twoslightlydifferent sets of conditions we show that the PIA yields the optimal (maximum) reward, an average optimal stationary policy, and a solution to the average reward optimality equation.

VECTOR-VALUED MARKOV DECISION PROCESSES WITH AVERAGE REWARD CRITERION: THE MULTICHAIN CASE

2000 ◽  
Vol 14 (4) ◽  
pp. 533-548
Author(s):  
Kazuyoshi Wakuta
Keyword(s):  
Decision Process ◽  
Decision Processes ◽  
Iteration Algorithm ◽  
Average Reward ◽  
Markov Decision ◽  
Policy Iteration Algorithm ◽  
Average Reward Criterion ◽  
Systems Of Linear Inequalities ◽  
Vector Valued ◽  
Reward Criterion

We study the multichain case of a vector-valued Markov decision process with average reward criterion. We characterize optimal deterministic stationary policies via systems of linear inequalities and discuss a policy iteration algorithm for finding all optimal deterministic stationary policies.

scholarly journals CONVERGENCE OF SIMULATION-BASED POLICY ITERATION

2003 ◽  
Vol 17 (2) ◽  
pp. 213-234 ◽  
Author(s):  
William L. Cooper ◽  
Shane G. Henderson ◽  
Mark E. Lewis
Keyword(s):  
Markov Decision Processes ◽  
Decision Rules ◽  
Almost Sure Convergence ◽  
Policy Iteration ◽  
Decision Processes ◽  
Optimal Decision ◽  
Iteration Algorithm ◽  
Simulation Based ◽  
Markov Decision ◽  
Run Lengths

Simulation-based policy iteration (SBPI) is a modification of the policy iteration algorithm for computing optimal policies for Markov decision processes. At each iteration, rather than solving the average evaluation equations, SBPI employs simulation to estimate a solution to these equations. For recurrent average-reward Markov decision processes with finite state and action spaces, we provide easily verifiable conditions that ensure that simulation-based policy iteration almost-surely eventually never leaves the set of optimal decision rules. We analyze three simulation estimators for solutions to the average evaluation equations. Using our general results, we derive simple conditions on the simulation run lengths that guarantee the almost-sure convergence of the algorithm.

Sign in / Sign up

Export Citation Format

Share Document