scholarly journals ON THE EXISTENCE OF OPTIMAL CONTROL IN CONTINUOUS TIME MARKOV DECISION PROCESSES

10.5109/13058 ◽  
1972 ◽  
Vol 15 (1/2) ◽  
pp. 7-17 ◽  
Author(s):  
Masami Yasuda
Keyword(s):  
Optimal Control ◽  
Markov Decision Processes ◽  
Continuous Time ◽  
Decision Processes ◽  
Markov Decision ◽  
Existence Of Optimal Control

scholarly journals Impulsive Control for Continuous-Time Markov Decision Processes

2015 ◽  
Vol 47 (1) ◽  
pp. 106-127 ◽  
Author(s):  
François Dufour ◽  
Alexei B. Piunovskiy
Keyword(s):  
Optimal Control ◽  
Control Problem ◽  
Markov Decision Processes ◽  
Control Strategy ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Decision Processes ◽  
Optimal Control Strategy ◽  
Optimality Equation ◽  
Markov Decision

In this paper our objective is to study continuous-time Markov decision processes on a general Borel state space with both impulsive and continuous controls for the infinite time horizon discounted cost. The continuous-time controlled process is shown to be nonexplosive under appropriate hypotheses. The so-called Bellman equation associated to this control problem is studied. Sufficient conditions ensuring the existence and the uniqueness of a bounded measurable solution to this optimality equation are provided. Moreover, it is shown that the value function of the optimization problem under consideration satisfies this optimality equation. Sufficient conditions are also presented to ensure on the one hand the existence of an optimal control strategy, and on the other hand the existence of a ε-optimal control strategy. The decomposition of the state space into two disjoint subsets is exhibited where, roughly speaking, one should apply a gradual action or an impulsive action correspondingly to obtain an optimal or ε-optimal strategy. An interesting consequence of our previous results is as follows: the set of strategies that allow interventions at time t = 0 and only immediately after natural jumps is a sufficient set for the control problem under consideration.

scholarly journals Impulsive Control for Continuous-Time Markov Decision Processes

2015 ◽  
Vol 47 (01) ◽  
pp. 106-127 ◽  
Author(s):  
François Dufour ◽  
Alexei B. Piunovskiy
Keyword(s):  
Optimal Control ◽  
Control Problem ◽  
Markov Decision Processes ◽  
Control Strategy ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Decision Processes ◽  
Optimal Control Strategy ◽  
Optimality Equation ◽  
Markov Decision

In this paper our objective is to study continuous-time Markov decision processes on a general Borel state space with both impulsive and continuous controls for the infinite time horizon discounted cost. The continuous-time controlled process is shown to be nonexplosive under appropriate hypotheses. The so-called Bellman equation associated to this control problem is studied. Sufficient conditions ensuring the existence and the uniqueness of a bounded measurable solution to this optimality equation are provided. Moreover, it is shown that the value function of the optimization problem under consideration satisfies this optimality equation. Sufficient conditions are also presented to ensure on the one hand the existence of an optimal control strategy, and on the other hand the existence of a ε-optimal control strategy. The decomposition of the state space into two disjoint subsets is exhibited where, roughly speaking, one should apply a gradual action or an impulsive action correspondingly to obtain an optimal or ε-optimal strategy. An interesting consequence of our previous results is as follows: the set of strategies that allow interventions at time t = 0 and only immediately after natural jumps is a sufficient set for the control problem under consideration.

scholarly journals Denumerable state continuous time Markov decision processes with unbounded cost and transition rates under average criterion

2002 ◽  
Vol 43 (4) ◽  
pp. 541-557 ◽  
Author(s):  
Xianping Guo ◽  
Weiping Zhu
Keyword(s):  
Markov Decision Processes ◽  
Continuous Time ◽  
Decision Processes ◽  
Transition Rates ◽  
Birth And Death Processes ◽  
Optimality Equation ◽  
Average Criterion ◽  
Markov Decision ◽  
Unbounded Cost ◽  
Queue Model

AbstractIn this paper, we consider denumerable state continuous time Markov decision processes with (possibly unbounded) transition and cost rates under average criterion. We present a set of conditions and prove the existence of both average cost optimal stationary policies and a solution of the average optimality equation under the conditions. The results in this paper are applied to an admission control queue model and controlled birth and death processes.

Continuous-Time Markov Decision Processes

2020 ◽  
Author(s):  
Alexey Piunovskiy ◽  
Yi Zhang
Keyword(s):  
Markov Decision Processes ◽  
Continuous Time ◽  
Decision Processes ◽  
Markov Decision

Preferred Rules in Continuous Time Markov Decision Processes

1974 ◽  
Vol 21 (3) ◽  
pp. 348-357 ◽  
Author(s):  
Mark R. Lembersky
Keyword(s):  
Markov Decision Processes ◽  
Continuous Time ◽  
Decision Processes ◽  
Markov Decision
Sign in / Sign up

Export Citation Format

Share Document