Continuous-Time Markov Decision Processes with Discounted Rewards: The Case of Polish Spaces

2007 ◽  
Vol 32 (1) ◽  
pp. 73-87 ◽  
Author(s):  
Xianping Guo
Keyword(s):  
Markov Decision Processes ◽  
Continuous Time ◽  
Decision Processes ◽  
Polish Spaces ◽  
Discounted Rewards ◽  
Markov Decision

scholarly journals Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces

2008 ◽  
Vol 45 (02) ◽  
pp. 417-429 ◽  
Author(s):  
Quanxin Zhu ◽  
Tomás Prieto-Rumeau
Keyword(s):  
Markov Decision Processes ◽  
Continuous Time ◽  
Optimality Criteria ◽  
Decision Processes ◽  
Polish Spaces ◽  
Overtaking Optimality ◽  
Markov Decision ◽  
Bias Optimality ◽  
Optimality Equations ◽  
Action Spaces

In this paper we study the bias and the overtaking optimality criteria for continuous-time jump Markov decision processes in general state and action spaces. The corresponding transition rates are allowed to be unbounded, and the reward rates may have neither upper nor lower bounds. Under appropriate hypotheses, we prove the existence of solutions to the bias optimality equations, the existence of bias optimal policies, and an equivalence relation between bias and overtaking optimality.

scholarly journals Bias and Overtaking Optimality for Continuous-Time Jump Markov Decision Processes in Polish Spaces

2008 ◽  
Vol 45 (2) ◽  
pp. 417-429 ◽  
Author(s):  
Quanxin Zhu ◽  
Tomás Prieto-Rumeau
Keyword(s):  
Markov Decision Processes ◽  
Continuous Time ◽  
Optimality Criteria ◽  
Decision Processes ◽  
Polish Spaces ◽  
Overtaking Optimality ◽  
Markov Decision ◽  
Bias Optimality ◽  
Optimality Equations ◽  
Action Spaces

In this paper we study the bias and the overtaking optimality criteria for continuous-time jump Markov decision processes in general state and action spaces. The corresponding transition rates are allowed to be unbounded, and the reward rates may have neither upper nor lower bounds. Under appropriate hypotheses, we prove the existence of solutions to the bias optimality equations, the existence of bias optimal policies, and an equivalence relation between bias and overtaking optimality.

scholarly journals Denumerable state continuous time Markov decision processes with unbounded cost and transition rates under average criterion

2002 ◽  
Vol 43 (4) ◽  
pp. 541-557 ◽  
Author(s):  
Xianping Guo ◽  
Weiping Zhu
Keyword(s):  
Markov Decision Processes ◽  
Continuous Time ◽  
Decision Processes ◽  
Transition Rates ◽  
Birth And Death Processes ◽  
Optimality Equation ◽  
Average Criterion ◽  
Markov Decision ◽  
Unbounded Cost ◽  
Queue Model

AbstractIn this paper, we consider denumerable state continuous time Markov decision processes with (possibly unbounded) transition and cost rates under average criterion. We present a set of conditions and prove the existence of both average cost optimal stationary policies and a solution of the average optimality equation under the conditions. The results in this paper are applied to an admission control queue model and controlled birth and death processes.

Sign in / Sign up

Export Citation Format

Share Document