Determining the Best Trade-Off Between Expected Economic Return and the Risk of Undesirable Events When Managing a Randomly Varying Population

1979 ◽  
Vol 36 (8) ◽  
pp. 939-947 ◽  
Author(s):  
Roy Mendelssohn
Keyword(s):  
Markov Decision Processes ◽  
Population Based ◽  
Decision Processes ◽  
Expected Return ◽  
Trade Off ◽  
Long Run ◽  
Optimal Policies ◽  
Markov Decision ◽  
Long Run Risk ◽  
Varying Population

Conditions are given that imply there exist policies that "minimize risk" of undesirable events for stochastic harvesting models. It is shown that for many problems, either such a policy will not exist, or else it is an "extreme" policy that is equally undesirable. Techniques are given to systematically trade-off decreases in the long-run expected return with decreases in the long-run risk. Several numerical examples are given for models of salmon runs, when both population-based risks and harvest-based risks are considered. Key words: Markov decision processes, risk, salmon management, Pareto optimal policies, trade-off curves, linear programing

scholarly journals Markov Decision Processes with Multiple Long-run Average Objectives

2014 ◽  
Vol 10 (1) ◽  
Author(s):  
Tomáš Brázdil ◽  
Václav Brožek ◽  
Krishnendu Chatterjee ◽  
Vojtěch Forejt ◽  
Antonín Kučera
Keyword(s):  
Markov Decision Processes ◽  
Decision Processes ◽  
Long Run ◽  
Markov Decision

scholarly journals Markov decision processes with quasi-hyperbolic discounting

2020 ◽  
Author(s):  
Anna Jaśkiewicz ◽  
Andrzej S. Nowak
Keyword(s):  
Markov Decision Processes ◽  
Dynamic Game ◽  
Hyperbolic Discounting ◽  
Decision Processes ◽  
State Spaces ◽  
Long Run ◽  
Markov Perfect ◽  
Markov Decision ◽  
Time Inconsistent ◽  
Perfect Equilibria

AbstractWe study Markov decision processes with Borel state spaces under quasi-hyperbolic discounting. This type of discounting nicely models human behaviour, which is time-inconsistent in the long run. The decision maker has preferences changing in time. Therefore, the standard approach based on the Bellman optimality principle fails. Within a dynamic game-theoretic framework, we prove the existence of randomised stationary Markov perfect equilibria for a large class of Markov decision processes with transitions having a density function. We also show that randomisation can be restricted to two actions in every state of the process. Moreover, we prove that under some conditions, this equilibrium can be replaced by a deterministic one. For models with countable state spaces, we establish the existence of deterministic Markov perfect equilibria. Many examples are given to illustrate our results, including a portfolio selection model with quasi-hyperbolic discounting.

Average optimal policies in Markov decision drift processes with applications to a queueing and a replacement model

1983 ◽  
Vol 15 (2) ◽  
pp. 274-303 ◽  
Author(s):  
Arie Hordijk ◽  
Frank A. Van Der Duyn Schouten
Keyword(s):  
Markov Decision Processes ◽  
Optimal Policy ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Decision Processes ◽  
Time Parameter ◽  
Queueing Model ◽  
Replacement Model ◽  
Optimal Policies ◽  
Markov Decision

Recently the authors introduced the concept of Markov decision drift processes. A Markov decision drift process can be seen as a straightforward generalization of a Markov decision process with continuous time parameter. In this paper we investigate the existence of stationary average optimal policies for Markov decision drift processes. Using a well-known Abelian theorem we derive sufficient conditions, which guarantee that a ‘limit point' of a sequence of discounted optimal policies with the discounting factor approaching 1 is an average optimal policy. An alternative set of sufficient conditions is obtained for the case in which the discounted optimal policies generate regenerative stochastic processes. The latter set of conditions is easier to verify in several applications. The results of this paper are also applicable to Markov decision processes with discrete or continuous time parameter and to semi-Markov decision processes. In this sense they generalize some well-known results for Markov decision processes with finite or compact action space. Applications to an M/M/1 queueing model and a maintenance replacement model are given. It is shown that under certain conditions on the model parameters the average optimal policy for the M/M/1 queueing model is monotone non-decreasing (as a function of the number of waiting customers) with respect to the service intensity and monotone non-increasing with respect to the arrival intensity. For the maintenance replacement model we prove the average optimality of a bang-bang type policy. Special attention is paid to the computation of the optimal control parameters.

Conditions for the uniqueness of optimal policies of discounted Markov decision processes

2004 ◽  
Vol 60 (3) ◽  
pp. 415-436 ◽  
Author(s):  
Daniel Cruz-Su�rez ◽  
Ra�l Montes-de-Oca ◽  
Francisco Salem-Silva
Keyword(s):  
Markov Decision Processes ◽  
Decision Processes ◽  
Optimal Policies ◽  
Markov Decision

Cost rate heuristics for semi-Markov decision processes

1992 ◽  
Vol 29 (03) ◽  
pp. 633-644
Author(s):  
K. D. Glazebrook ◽  
Michael P. Bailey ◽  
Lyn R. Whitaker
Keyword(s):  
Dynamic Programming ◽  
Markov Decision Processes ◽  
Preventive Maintenance ◽  
Decision Processes ◽  
Cost Rate ◽  
Backwards Induction ◽  
Optimal Policies ◽  
Markov Decision ◽  
Speed Of Evolution

In response to the computational complexity of the dynamic programming/backwards induction approach to the development of optimal policies for semi-Markov decision processes, we propose a class of heuristics resulting from an inductive process which proceeds forwards in time. These heuristics always choose actions in such a way as to minimize some measure of the current cost rate. We describe a procedure for calculating such cost rate heuristics. The quality of the performance of such policies is related to the speed of evolution (in a cost sense) of the process. A simple model of preventive maintenance is described in detail. Cost rate heuristics for this problem are calculated and assessed computationally.

Sign in / Sign up

Export Citation Format

Share Document