Finite horizon continuous-time Markov decision processes with mean and variance criteria

2018 ◽  
Vol 28 (4) ◽  
pp. 539-564 ◽  
Author(s):  
Yonghui Huang
Keyword(s):  
Markov Decision Processes ◽  
Continuous Time ◽  
Decision Processes ◽  
Finite Horizon ◽  
Markov Decision ◽  
Mean And Variance

Finite-horizon optimality for continuous-time Markov decision processes with unbounded transition rates

2015 ◽  
Vol 47 (4) ◽  
pp. 1064-1087 ◽  
Author(s):  
Xianping Guo ◽  
Xiangxiang Huang ◽  
Yonghui Huang
Keyword(s):  
Markov Decision Processes ◽  
Continuous Time ◽  
Value Function ◽  
Decision Processes ◽  
Finite Horizon ◽  
Transition Rates ◽  
Optimality Equation ◽  
Unbounded Transition Rates ◽  
Markov Decision ◽  
The Value Function

In this paper we focus on the finite-horizon optimality for denumerable continuous-time Markov decision processes, in which the transition and reward/cost rates are allowed to be unbounded, and the optimality is over the class of all randomized history-dependent policies. Under mild reasonable conditions, we first establish the existence of a solution to the finite-horizon optimality equation by designing a technique of approximations from the bounded transition rates to unbounded ones. Then we prove the existence of ε (≥ 0)-optimal Markov policies and verify that the value function is the unique solution to the optimality equation by establishing the analog of the Itô-Dynkin formula. Finally, we provide an example in which the transition rates and the value function are all unbounded and, thus, obtain solutions to some of the unsolved problems by Yushkevich (1978).

Finite-horizon optimality for continuous-time Markov decision processes with unbounded transition rates

2015 ◽  
Vol 47 (04) ◽  
pp. 1064-1087 ◽  
Author(s):  
Xianping Guo ◽  
Xiangxiang Huang ◽  
Yonghui Huang
Keyword(s):  
Markov Decision Processes ◽  
Continuous Time ◽  
Value Function ◽  
Decision Processes ◽  
Finite Horizon ◽  
Transition Rates ◽  
Optimality Equation ◽  
Unbounded Transition Rates ◽  
Markov Decision ◽  
The Value Function

In this paper we focus on the finite-horizon optimality for denumerable continuous-time Markov decision processes, in which the transition and reward/cost rates are allowed to be unbounded, and the optimality is over the class of all randomized history-dependent policies. Under mild reasonable conditions, we first establish the existence of a solution to the finite-horizon optimality equation by designing a technique of approximations from the bounded transition rates to unbounded ones. Then we prove the existence of ε (≥ 0)-optimal Markov policies and verify that the value function is the unique solution to the optimality equation by establishing the analog of the Itô-Dynkin formula. Finally, we provide an example in which the transition rates and the value function are all unbounded and, thus, obtain solutions to some of the unsolved problems by Yushkevich (1978).

Sign in / Sign up

Export Citation Format

Share Document