Optimality equations and inequalities in a class of risk-sensitive average cost Markov decision chains

2009 ◽  
Vol 71 (1) ◽  
pp. 47-84 ◽  
Author(s):  
Rolando Cavazos-Cadena
Keyword(s):  
Average Cost ◽  
Risk Sensitive ◽  
Markov Decision ◽  
Optimality Equations

Vanishing discount approximations in controlled Markov chains with risk-sensitive average criterion

2018 ◽  
Vol 50 (01) ◽  
pp. 204-230 ◽  
Author(s):  
Rolando Cavazos-Cadena ◽  
Daniel Hernández-Hernández
Keyword(s):  
Average Cost ◽  
Control Policy ◽  
Sensitivity Coefficient ◽  
Value Functions ◽  
Risk Sensitive ◽  
Finite State ◽  
Markov Decision ◽  
Functional Part ◽  
Optimal Average ◽  
Average Cost Optimality Equation

Abstract This work concerns Markov decision chains on a finite state space. The decision-maker has a constant and nonnull risk sensitivity coefficient, and the performance of a control policy is measured by two different indices, namely, the discounted and average criteria. Motivated by well-known results for the risk-neutral case, the problem of approximating the optimal risk-sensitive average cost in terms of the optimal risk-sensitive discounted value functions is addressed. Under suitable communication assumptions, it is shown that, as the discount factor increases to 1, appropriate normalizations of the optimal discounted value functions converge to the optimal average cost, and to the functional part of the solution of the risk-sensitive average cost optimality equation.

scholarly journals Explicit Solution of the Average-Cost Optimality Equation for a Pest-Control Problem

2011 ◽  
Vol 2011 ◽  
pp. 1-11
Author(s):  
Epaminondas G. Kyriakidis
Keyword(s):  
Optimal Control ◽  
Markov Decision Process ◽  
Pest Control ◽  
Continuous Time ◽  
Decision Process ◽  
Average Cost ◽  
Optimality Equation ◽  
Markov Decision ◽  
Average Cost Optimality Equation ◽  
Cost Optimality

We introduce a Markov decision process in continuous time for the optimal control of a simple symmetrical immigration-emigration process by the introduction of total catastrophes. It is proved that a particular control-limit policy is average cost optimal within the class of all stationary policies by verifying that the relative values of this policy are the solution of the corresponding optimality equation.

Optimality conditions for a Markov decision chain with unbounded costs

1980 ◽  
Vol 17 (04) ◽  
pp. 996-1003
Author(s):  
D. R. Robinson
Keyword(s):  
Dynamic Programming ◽  
Optimality Conditions ◽  
Average Cost ◽  
Optimality Equation ◽  
Dynamic Programming Equation ◽  
Potential Condition ◽  
Markov Decision ◽  
General Conditions

It is known that when costs are unbounded satisfaction of the appropriate dynamic programming ‘optimality' equation by a policy is not sufficient to guarantee its average optimality. A ‘lowest-order potential' condition is introduced which, along with the dynamic programming equation, is sufficient to establish the optimality of the policy. Also, it is shown that under fairly general conditions, if the lowest-order potential condition is not satisfied there exists a non-memoryless policy with smaller average cost than the policy satisfying the dynamic programming equation.

Sign in / Sign up

Export Citation Format

Share Document