Resolvent operators of Markov processes and their applications in the control of a finite dam

1989 ◽  
Vol 26 (02) ◽  
pp. 314-324 ◽  
Author(s):  
F. A. Attia
Keyword(s):  
Markov Chain ◽  
Markov Process ◽  
Wiener Process ◽  
Markov Processes ◽  
Resolvent Operators ◽  
Input Process ◽  
Long Run ◽  
Negative Drift ◽  
Finite Dam ◽  
Irreducible Markov Chain

The resolvent operators of the following two processes are obtained: (a) the bivariate Markov process W = (X, Y), where Y(t) is an irreducible Markov chain and X(t) is its integral, and (b) the geometric Wiener process G(t) = exp{B(t} where B(t) is a Wiener process with non-negative drift μ and variance parameter σ2. These results are then used via a limiting procedure to determine the long-run average cost per unit time of operating a finite dam where the input process is either X(t) or G(t). The system is controlled by a policy (Attia [1], Lam [6]).

Resolvent operators of Markov processes and their applications in the control of a finite dam

1989 ◽  
Vol 26 (2) ◽  
pp. 314-324 ◽  
Author(s):  
F. A. Attia
Keyword(s):  
Markov Chain ◽  
Markov Process ◽  
Wiener Process ◽  
Markov Processes ◽  
Resolvent Operators ◽  
Input Process ◽  
Long Run ◽  
Negative Drift ◽  
Finite Dam ◽  
Irreducible Markov Chain

The resolvent operators of the following two processes are obtained: (a) the bivariate Markov process W = (X, Y), where Y(t) is an irreducible Markov chain and X(t) is its integral, and (b) the geometric Wiener process G(t) = exp{B(t} where B(t) is a Wiener process with non-negative drift μ and variance parameter σ2. These results are then used via a limiting procedure to determine the long-run average cost per unit time of operating a finite dam where the input process is either X(t) or G(t). The system is controlled by a policy (Attia [1], Lam [6]).

The control of a finite dam

1982 ◽  
Vol 19 (4) ◽  
pp. 815-825 ◽  
Author(s):  
F. A. Attia ◽  
P. J. Brockwell
Keyword(s):  
Markov Chain ◽  
Wiener Process ◽  
Average Cost ◽  
Type B ◽  
Input Process ◽  
Long Run ◽  
The Cost ◽  
Finite Dam

The long-run average cost per unit time of operating a finite dam controlled by a PlM policy (Faddy (1974), Zuckerman (1977)) is determined when the cumulative input process is (a) a Wiener process with drift and (b) the integral of a Markov chain. It is shown how the cost for (a) can be obtained as the limit of the costs associated with a sequence of input processes of the type (b).

The control of a finite dam

1982 ◽  
Vol 19 (04) ◽  
pp. 815-825 ◽  
Author(s):  
F. A. Attia ◽  
P. J. Brockwell
Keyword(s):  
Markov Chain ◽  
Wiener Process ◽  
Average Cost ◽  
Type B ◽  
Input Process ◽  
Long Run ◽  
The Cost ◽  
Finite Dam

The long-run average cost per unit time of operating a finite dam controlled by a Pl M policy (Faddy (1974), Zuckerman (1977)) is determined when the cumulative input process is (a) a Wiener process with drift and (b) the integral of a Markov chain. It is shown how the cost for (a) can be obtained as the limit of the costs associated with a sequence of input processes of the type (b).

Optimal control of a finite dam: average-cost case

1985 ◽  
Vol 22 (02) ◽  
pp. 480-484 ◽  
Author(s):  
Lam Yeh
Keyword(s):  
Optimal Control ◽  
Wiener Process ◽  
Release Rate ◽  
Optimal Policy ◽  
Average Cost ◽  
Output Rate ◽  
Penalty Cost ◽  
Long Run ◽  
Negative Drift ◽  
Finite Dam

We consider the problem of minimizing the long-run average cost per unit time of operating a finite dam in the class of the policies of the following type. Assume that the dam is initially empty, the release rate is kept at 0 until the dam storage increases to λ, and as soon as this occurs, water is released at rate M, then the output rate is kept at M as long as the dam storage is more than τ and it must be decreased to 0 if the dam storage becomes τ. We assume that the input of water into the finite dam is a Wiener process with non-negative drift μ and variance parameter σ 2. There is a cost in increasing the output rate from 0 to M as well as in decreasing the rate from M to 0 and whenever the dam storage is below level a, there is a penalty cost per unit time depending on the level. A reward is given for each unit of water released. In this paper, the long-run average cost per unit time is determined, and therefore the optimal policy can be found numerically.

The control of a finite dam with penalty cost function: Markov input rate

1987 ◽  
Vol 24 (2) ◽  
pp. 457-465 ◽  
Author(s):  
F. A. Attia
Keyword(s):  
Markov Chain ◽  
Cost Function ◽  
Measurable Function ◽  
Average Cost ◽  
Input Rate ◽  
Input Process ◽  
Penalty Cost ◽  
Bounded Measurable Function ◽  
Long Run ◽  
Finite Dam

The long-run average cost per unit time of operating a finite dam controlled by a policy (Lam Yeh (1985)) is determined when the cumulative input process is the integral of a Markov chain. A penalty cost which accrues continuously at a rate g(X(t)), where g is a bounded measurable function of the content, is also introduced. An example where the input rate is a two-state Markov chain is considered in detail to illustrate the computations.

Optimal control of a finite dam using P M λΤ policies and penalty cost: total discounted and long run average cases

1990 ◽  
Vol 27 (04) ◽  
pp. 888-898
Author(s):  
M. Abdel-hameed ◽  
Y. Nakhi
Keyword(s):  
Optimal Control ◽  
Wiener Process ◽  
Average Cost ◽  
Input Process ◽  
Average Case ◽  
Discounted Cost ◽  
Penalty Cost ◽  
Drift Term ◽  
Long Run ◽  
Finite Dam

Zuckermann [10] considers the problem of optimal control of a finite dam using policies, assuming that the input process is Wiener with drift term μ ≧ 0. Lam Yeh and Lou Jiann Hua [7] treat the case where the input is a Wiener process with a reflecting boundary at zero, with drift term μ ≧ 0, using the long-run average cost and total discounted cost criteria. Attia [1] obtains results similar to those of Lam Yeh and Lou Jiann Hua for the long-run average case and extends them to include μ < 0. In this paper we look further into the results of Zuckerman [10], simplify some of the work of Attia [1], [2], offering corrections to some of his formulae and extend the results of Lam Yeh and Lou Jiann Hua [7].

Optimal control of a finite dam: average-cost case

1985 ◽  
Vol 22 (2) ◽  
pp. 480-484 ◽  
Author(s):  
Lam Yeh
Keyword(s):  
Optimal Control ◽  
Wiener Process ◽  
Release Rate ◽  
Optimal Policy ◽  
Average Cost ◽  
Output Rate ◽  
Penalty Cost ◽  
Long Run ◽  
Negative Drift ◽  
Finite Dam

We consider the problem of minimizing the long-run average cost per unit time of operating a finite dam in the class of the policies of the following type. Assume that the dam is initially empty, the release rate is kept at 0 until the dam storage increases to λ, and as soon as this occurs, water is released at rate M, then the output rate is kept at M as long as the dam storage is more than τ and it must be decreased to 0 if the dam storage becomes τ. We assume that the input of water into the finite dam is a Wiener process with non-negative drift μ and variance parameter σ2. There is a cost in increasing the output rate from 0 to M as well as in decreasing the rate from M to 0 and whenever the dam storage is below level a, there is a penalty cost per unit time depending on the level. A reward is given for each unit of water released. In this paper, the long-run average cost per unit time is determined, and therefore the optimal policy can be found numerically.

Optimal control of a finite dam using PMλΤ policies and penalty cost: total discounted and long run average cases

1990 ◽  
Vol 27 (4) ◽  
pp. 888-898 ◽  
Author(s):  
M. Abdel-hameed ◽  
Y. Nakhi
Keyword(s):  
Optimal Control ◽  
Wiener Process ◽  
Average Cost ◽  
Input Process ◽  
Average Case ◽  
Discounted Cost ◽  
Penalty Cost ◽  
Drift Term ◽  
Long Run ◽  
Finite Dam

Zuckermann [10] considers the problem of optimal control of a finite dam using policies, assuming that the input process is Wiener with drift term μ ≧ 0. Lam Yeh and Lou Jiann Hua [7] treat the case where the input is a Wiener process with a reflecting boundary at zero, with drift term μ ≧ 0, using the long-run average cost and total discounted cost criteria. Attia [1] obtains results similar to those of Lam Yeh and Lou Jiann Hua for the long-run average case and extends them to include μ < 0. In this paper we look further into the results of Zuckerman [10], simplify some of the work of Attia [1], [2], offering corrections to some of his formulae and extend the results of Lam Yeh and Lou Jiann Hua [7].

The control of a finite dam with penalty cost function: Markov input rate

1987 ◽  
Vol 24 (02) ◽  
pp. 457-465 ◽  
Author(s):  
F. A. Attia
Keyword(s):  
Markov Chain ◽  
Cost Function ◽  
Measurable Function ◽  
Average Cost ◽  
Input Rate ◽  
Input Process ◽  
Penalty Cost ◽  
Bounded Measurable Function ◽  
Long Run ◽  
Finite Dam

The long-run average cost per unit time of operating a finite dam controlled by a policy (Lam Yeh (1985)) is determined when the cumulative input process is the integral of a Markov chain. A penalty cost which accrues continuously at a rate g(X(t)), where g is a bounded measurable function of the content, is also introduced. An example where the input rate is a two-state Markov chain is considered in detail to illustrate the computations.

scholarly journals Conditioning an additive functional of a Markov chain to stay nonnegative. I. Survival for a long time

2005 ◽  
Vol 37 (4) ◽  
pp. 1015-1034 ◽  
Author(s):  
Saul D. Jacka ◽  
Zorana Lazic ◽  
Jon Warren
Keyword(s):  
Markov Chain ◽  
Weak Convergence ◽  
State Space ◽  
Continuous Time ◽  
Additive Functional ◽  
Finite State ◽  
Long Time ◽  
Negative Drift ◽  
Irreducible Markov Chain ◽  
Finite State Space

Let (Xt)t≥0 be a continuous-time irreducible Markov chain on a finite state space E, let v be a map v: E→ℝ\{0}, and let (φt)t≥0 be an additive functional defined by φt=∫0tv(Xs)d s. We consider the case in which the process (φt)t≥0 is oscillating and that in which (φt)t≥0 has a negative drift. In each of these cases, we condition the process (Xt,φt)t≥0 on the event that (φt)t≥0 is nonnegative until time T and prove weak convergence of the conditioned process as T→∞.

Sign in / Sign up

Export Citation Format

Share Document