Impulsive Control for Continuous-Time Markov Decision Processes: A Linear Programming Approach

2015 ◽  
Vol 74 (1) ◽  
pp. 129-161 ◽  
Author(s):  
F. Dufour ◽  
A. B. Piunovskiy
Keyword(s):  
Linear Programming ◽  
Markov Decision Processes ◽  
Continuous Time ◽  
Impulsive Control ◽  
Decision Processes ◽  
Programming Approach ◽  
Linear Programming Approach ◽  
Markov Decision

scholarly journals On Linear Programming for Constrained and Unconstrained Average-Cost Markov Decision Processes with Countable Action Spaces and Strictly Unbounded Costs

2021 ◽  
Author(s):  
Huizhen Yu
Keyword(s):  
Linear Programming ◽  
Markov Decision Processes ◽  
Average Cost ◽  
Decision Processes ◽  
The State ◽  
Programming Approach ◽  
One Stage ◽  
Markov Decision ◽  
Action Spaces ◽  
Borel Measurable

We consider the linear programming approach for constrained and unconstrained Markov decision processes (MDPs) under the long-run average-cost criterion, where the class of MDPs in our study have Borel state spaces and discrete countable action spaces. Under a strict unboundedness condition on the one-stage costs and a recently introduced majorization condition on the state transition stochastic kernel, we study infinite-dimensional linear programs for the average-cost MDPs and prove the absence of a duality gap and other optimality results. Our results do not require a lower-semicontinuous MDP model. Thus, they can be applied to countable action space MDPs where the dynamics and one-stage costs are discontinuous in the state variable. Our proofs make use of the continuity property of Borel measurable functions asserted by Lusin’s theorem.

scholarly journals Denumerable state continuous time Markov decision processes with unbounded cost and transition rates under average criterion

2002 ◽  
Vol 43 (4) ◽  
pp. 541-557 ◽  
Author(s):  
Xianping Guo ◽  
Weiping Zhu
Keyword(s):  
Markov Decision Processes ◽  
Continuous Time ◽  
Decision Processes ◽  
Transition Rates ◽  
Birth And Death Processes ◽  
Optimality Equation ◽  
Average Criterion ◽  
Markov Decision ◽  
Unbounded Cost ◽  
Queue Model

AbstractIn this paper, we consider denumerable state continuous time Markov decision processes with (possibly unbounded) transition and cost rates under average criterion. We present a set of conditions and prove the existence of both average cost optimal stationary policies and a solution of the average optimality equation under the conditions. The results in this paper are applied to an admission control queue model and controlled birth and death processes.

Sign in / Sign up

Export Citation Format

Share Document