Optimality equations in undiscounted Markov decision processes

2003 ◽  
Author(s):  
M.L. Puterman
Keyword(s):  
Markov Decision Processes ◽  
Decision Processes ◽  
Markov Decision ◽  
Optimality Equations

scholarly journals Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces

2008 ◽  
Vol 45 (02) ◽  
pp. 417-429 ◽  
Author(s):  
Quanxin Zhu ◽  
Tomás Prieto-Rumeau
Keyword(s):  
Markov Decision Processes ◽  
Continuous Time ◽  
Optimality Criteria ◽  
Decision Processes ◽  
Polish Spaces ◽  
Overtaking Optimality ◽  
Markov Decision ◽  
Bias Optimality ◽  
Optimality Equations ◽  
Action Spaces

In this paper we study the bias and the overtaking optimality criteria for continuous-time jump Markov decision processes in general state and action spaces. The corresponding transition rates are allowed to be unbounded, and the reward rates may have neither upper nor lower bounds. Under appropriate hypotheses, we prove the existence of solutions to the bias optimality equations, the existence of bias optimal policies, and an equivalence relation between bias and overtaking optimality.

scholarly journals Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces

2008 ◽  
Vol 45 (2) ◽  
pp. 417-429 ◽  
Author(s):  
Quanxin Zhu ◽  
Tomás Prieto-Rumeau
Keyword(s):  
Markov Decision Processes ◽  
Continuous Time ◽  
Optimality Criteria ◽  
Decision Processes ◽  
Polish Spaces ◽  
Overtaking Optimality ◽  
Markov Decision ◽  
Bias Optimality ◽  
Optimality Equations ◽  
Action Spaces

In this paper we study the bias and the overtaking optimality criteria for continuous-time jump Markov decision processes in general state and action spaces. The corresponding transition rates are allowed to be unbounded, and the reward rates may have neither upper nor lower bounds. Under appropriate hypotheses, we prove the existence of solutions to the bias optimality equations, the existence of bias optimal policies, and an equivalence relation between bias and overtaking optimality.

Extreme-point solutions in Markov decision processes

1983 ◽  
Vol 20 (04) ◽  
pp. 835-842
Author(s):  
David Assaf
Keyword(s):  
Convex Function ◽  
Extreme Point ◽  
Markov Decision Processes ◽  
Convex Functions ◽  
Sufficient Conditions ◽  
Decision Processes ◽  
Markov Decision ◽  
Full Solution

The paper presents sufficient conditions for certain functions to be convex. Functions of this type often appear in Markov decision processes, where their maximum is the solution of the problem. Since a convex function takes its maximum at an extreme point, the conditions may greatly simplify a problem. In some cases a full solution may be obtained after the reduction is made. Some illustrative examples are discussed.

Sign in / Sign up

Export Citation Format

Share Document