Bias Optimality for Continuous-Time Controlled Markov Chains

2006 ◽  
Vol 45 (1) ◽  
pp. 51-73 ◽  
Author(s):  
Tomás Prieto-Rumeau ◽  
Onésimo Hernández-Lerma
Keyword(s):  
Markov Chains ◽  
Continuous Time ◽  
Controlled Markov Chains ◽  
Bias Optimality

scholarly journals Continuous-time controlled Markov chains

2003 ◽  
Vol 13 (1) ◽  
pp. 363-388 ◽  
Author(s):  
Xianping Guo ◽  
Onésimo Hernández-Lerma
Keyword(s):  
Markov Chains ◽  
Continuous Time ◽  
Controlled Markov Chains

Discounted Continuous-Time Controlled Markov Chains: Convergence of Control Models

2012 ◽  
Vol 49 (04) ◽  
pp. 1072-1090
Author(s):  
Tomás Prieto-Rumeau ◽  
Onésimo Hernández-Lerma
Keyword(s):  
Markov Chains ◽  
Continuous Time ◽  
Convergence Rates ◽  
Control Model ◽  
Population System ◽  
Controlled Markov Chains ◽  
Control Models ◽  
Finite State ◽  
Definition Of ◽  
Optimal Reward

We are interested in continuous-time, denumerable state controlled Markov chains (CMCs), with compact Borel action sets, and possibly unbounded transition and reward rates, under the discounted reward optimality criterion. For such CMCs, we propose a definition of a sequence of control models {ℳ n } converging to a given control model ℳ, which ensures that the discount optimal reward and policies of ℳ n converge to those of ℳ. As an application, we propose a finite-state and finite-action truncation technique of the original control model ℳ, which is illustrated by approximating numerically the optimal reward and policy of a controlled population system with catastrophes. We study the corresponding convergence rates.

Sign in / Sign up

Export Citation Format

Share Document