Mixed risk-neutral/minimax control of discrete-time, finite-state Markov decision processes

2000 ◽  
Vol 45 (3) ◽  
pp. 528-532 ◽  
Author(s):  
S.P. Coraluppi ◽  
S.I. Marcus
Keyword(s):  
Markov Decision Processes ◽  
Discrete Time ◽  
Decision Processes ◽  
Minimax Control ◽  
Risk Neutral ◽  
Finite State ◽  
Markov Decision

scholarly journals Average optimality for Markov decision processes in borel spaces: a new condition and approach

2006 ◽  
Vol 43 (02) ◽  
pp. 318-334
Author(s):  
Xianping Guo ◽  
Quanxin Zhu
Keyword(s):  
Markov Decision Processes ◽  
Discrete Time ◽  
Existence Of Solutions ◽  
Sufficient Conditions ◽  
Decision Processes ◽  
Stationary Policy ◽  
Markov Decision ◽  
Optimal Stationary Policy ◽  
Optimality Inequality ◽  
Action Spaces

In this paper we study discrete-time Markov decision processes with Borel state and action spaces. The criterion is to minimize average expected costs, and the costs may have neither upper nor lower bounds. We first provide two average optimality inequalities of opposing directions and give conditions for the existence of solutions to them. Then, using the two inequalities, we ensure the existence of an average optimal (deterministic) stationary policy under additional continuity-compactness assumptions. Our conditions are slightly weaker than those in the previous literature. Also, some new sufficient conditions for the existence of an average optimal stationary policy are imposed on the primitive data of the model. Moreover, our approach is slightly different from the well-known ‘optimality inequality approach’ widely used in Markov decision processes. Finally, we illustrate our results in two examples.

scholarly journals An analysis of transient Markov decision processes

2006 ◽  
Vol 43 (3) ◽  
pp. 603-621 ◽  
Author(s):  
Huw W. James ◽  
E. J. Collins
Keyword(s):  
Markov Decision Processes ◽  
Value Function ◽  
Decision Processes ◽  
Optimal Value Function ◽  
Natural Form ◽  
Convergence Results ◽  
Optimal Value ◽  
Finite State ◽  
Markov Decision ◽  
Bounded Below

This paper is concerned with the analysis of Markov decision processes in which a natural form of termination ensures that the expected future costs are bounded, at least under some policies. Whereas most previous analyses have restricted attention to the case where the set of states is finite, this paper analyses the case where the set of states is not necessarily finite or even countable. It is shown that all the existence, uniqueness, and convergence results of the finite-state case hold when the set of states is a general Borel space, provided we make the additional assumption that the optimal value function is bounded below. We give a sufficient condition for the optimal value function to be bounded below which holds, in particular, if the set of states is countable.

Sign in / Sign up

Export Citation Format

Share Document