scholarly journals Finite horizon risk-sensitive continuous-time Markov decision processes with unbounded transition and cost rates

4OR ◽  
2019 ◽  
Vol 17 (4) ◽  
pp. 427-442 ◽  
Author(s):  
Xin Guo ◽  
Qiuli Liu ◽  
Yi Zhang
Keyword(s):  
Markov Decision Processes ◽  
Continuous Time ◽  
Decision Processes ◽  
Finite Horizon ◽  
Risk Sensitive ◽  
Markov Decision

Risk-sensitive average continuous-time Markov decision processes with unbounded transition and cost rates

2021 ◽  
Vol 58 (2) ◽  
pp. 523-550
Author(s):  
Xin Guo ◽  
Yonghui Huang
Keyword(s):  
Markov Decision Processes ◽  
Continuous Time ◽  
Decision Processes ◽  
Finite Horizon ◽  
Dynamic Programming Principle ◽  
Iteration Algorithm ◽  
Stationary Policy ◽  
Risk Sensitive ◽  
Finite State ◽  
Markov Decision

AbstractThis paper considers risk-sensitive average optimization for denumerable continuous-time Markov decision processes (CTMDPs), in which the transition and cost rates are allowed to be unbounded, and the policies can be randomized history dependent. We first derive the multiplicative dynamic programming principle and some new facts for risk-sensitive finite-horizon CTMDPs. Then, we establish the existence and uniqueness of a solution to the risk-sensitive average optimality equation (RS-AOE) through the results for risk-sensitive finite-horizon CTMDPs developed here, and also prove the existence of an optimal stationary policy via the RS-AOE. Furthermore, for the case of finite actions available at each state, we construct a sequence of models of finite-state CTMDPs with optimal stationary policies which can be obtained by a policy iteration algorithm in a finite number of iterations, and prove that an average optimal policy for the case of infinitely countable states can be approximated by those of the finite-state models. Finally, we illustrate the conditions and the iteration algorithm with an example.

Sign in / Sign up

Export Citation Format

Share Document