scholarly journals On the policy improvement algorithm for ergodic risk-sensitive control

2020 ◽  
pp. 1-26
Author(s):  
Ari Arapostathis ◽  
Anup Biswas ◽  
Somnath Pradhan
Keyword(s):  
Control Problem ◽  
General Result ◽  
Region Of Attraction ◽  
Policy Improvement ◽  
Improvement Algorithm ◽  
Risk Sensitive ◽  
Risk Sensitive Control ◽  
Whole Space ◽  
Controlled Diffusions ◽  
Running Cost

In this article we consider the ergodic risk-sensitive control problem for a large class of multidimensional controlled diffusions on the whole space. We study the minimization and maximization problems under either a blanket stability hypothesis, or a near-monotone assumption on the running cost. We establish the convergence of the policy improvement algorithm for these models. We also present a more general result concerning the region of attraction of the equilibrium of the algorithm.

Analysis of a risk-sensitive control problem for hidden Markov chains

1999 ◽  
Vol 44 (5) ◽  
pp. 1093-1100 ◽  
Author(s):  
D. Hernandez-Hernandez ◽  
S.I. Marcus ◽  
P.J. Fard
Keyword(s):  
Markov Chains ◽  
Control Problem ◽  
Hidden Markov ◽  
Hidden Markov Chains ◽  
Risk Sensitive ◽  
Risk Sensitive Control

scholarly journals Risk-sensitive control of stochastic hybrid systems on infinite time horizon

2000 ◽  
Vol 5 (6) ◽  
pp. 459-478 ◽  
Author(s):  
Thordur Runolfsson
Keyword(s):  
Control Problem ◽  
Hybrid Systems ◽  
Risk Sensitivity ◽  
Optimal Controls ◽  
Stochastic Hybrid Systems ◽  
Risk Sensitive ◽  
Risk Sensitive Control ◽  
Equivalence Relationship ◽  
Noise Input ◽  
The Cost

A risk-sensitive optimal control problem is considered for a hybrid system that consists of continuous time diffusion process that depends on a discrete valued mode variable that is modeled as a Markov chain. Optimality conditions are presented and conditions for the existence of optimal controls are derived. It is shown that the optimal risk-sensitive control problem is equivalent to the upper value of an associated stochastic differential game, and insight into the contributions of the noise input and mode variable to the risk sensitivity of the cost functional is given. Furthermore, it is shown that due to the mode variable risk sensitivity, the equivalence relationship that has been observed between risk-sensitive andH∞control in the nonhybrid case does not hold for stochastic hybrid systems.

A Class of Decision Processes Showing Policy-Improvement/Newton–Raphson Equivalence

1989 ◽  
Vol 3 (3) ◽  
pp. 397-403 ◽  
Author(s):  
P. Whittle
Keyword(s):  
Optimality Condition ◽  
Regularity Condition ◽  
Decision Processes ◽  
Policy Improvement ◽  
Improvement Algorithm ◽  
Newton Raphson ◽  
Raphson Algorithm

A condition expressed in Eq. (7) is given which, with one simplifying regularity condition, ensures that the policy-improvement algorithm is equivalent to application of the Newton–Raphson algorithm to an optimality condition. It is shown that this condition covers the two known cases of such equivalence, and another example is noted. The condition is believed to be necessary to within transformations of the problem, but this has not been proved.

Sign in / Sign up

Export Citation Format

Share Document