General optimal stopping theorems for semi-Markov processes

1993 ◽  
Vol 25 (4) ◽  
pp. 825-846 ◽  
Author(s):  
Frans A. Boshuizen ◽  
José M. Gouweleeuw
Keyword(s):  
Markov Processes ◽  
Optimal Stopping ◽  
Stopping Time ◽  
General Setting ◽  
The State ◽  
Shock Models ◽  
Special Cases ◽  
Optimal Stopping Problems ◽  
The Times ◽  
Semi Markov Processes

In this paper, optimal stopping problems for semi-Markov processes are studied in a fairly general setting. In such a process transitions are made from state to state in accordance with a Markov chain, but the amount of time spent in each state is random. The times spent in each state follow a general renewal process. They may depend on the present state as well as on the state into which the next transition is made.Our goal is to maximize the expected net return, which is given as a function of the state at time t minus some cost function. Discounting may or may not be considered. The main theorems (Theorems 3.5 and 3.11) are expressions for the optimal stopping time in the undiscounted and discounted case. These theorems generalize results of Zuckerman [16] and Boshuizen and Gouweleeuw [3]. Applications are given in various special cases.The results developed in this paper can also be applied to semi-Markov shock models, as considered in Taylor [13], Feldman [6] and Zuckerman [15].

General optimal stopping theorems for semi-Markov processes

1993 ◽  
Vol 25 (04) ◽  
pp. 825-846 ◽  
Author(s):  
Frans A. Boshuizen ◽  
José M. Gouweleeuw
Keyword(s):  
Markov Processes ◽  
Optimal Stopping ◽  
Stopping Time ◽  
General Setting ◽  
The State ◽  
Shock Models ◽  
Special Cases ◽  
Optimal Stopping Problems ◽  
The Times ◽  
Semi Markov Processes

In this paper, optimal stopping problems for semi-Markov processes are studied in a fairly general setting. In such a process transitions are made from state to state in accordance with a Markov chain, but the amount of time spent in each state is random. The times spent in each state follow a general renewal process. They may depend on the present state as well as on the state into which the next transition is made. Our goal is to maximize the expected net return, which is given as a function of the state at time t minus some cost function. Discounting may or may not be considered. The main theorems (Theorems 3.5 and 3.11) are expressions for the optimal stopping time in the undiscounted and discounted case. These theorems generalize results of Zuckerman [16] and Boshuizen and Gouweleeuw [3]. Applications are given in various special cases. The results developed in this paper can also be applied to semi-Markov shock models, as considered in Taylor [13], Feldman [6] and Zuckerman [15].

scholarly journals A Forward Algorithm for Solving Optimal Stopping Problems

2006 ◽  
Vol 43 (01) ◽  
pp. 102-113
Author(s):  
Albrecht Irle
Keyword(s):  
State Space ◽  
Optimal Stopping ◽  
Stopping Time ◽  
The State ◽  
Optimal Stopping Problem ◽  
Forward Algorithm ◽  
Optimal Stopping Time ◽  
Entrance Time ◽  
Markov Sequence ◽  
Optimal Stopping Problems

We consider the optimal stopping problem for g(Z n ), where Z n , n = 1, 2, …, is a homogeneous Markov sequence. An algorithm, called forward improvement iteration, is presented by which an optimal stopping time can be computed. Using an iterative step, this algorithm computes a sequence B 0 ⊇ B 1 ⊇ B 2 ⊇ · · · of subsets of the state space such that the first entrance time into the intersection F of these sets is an optimal stopping time. Various applications are given.

scholarly journals A Forward Algorithm for Solving Optimal Stopping Problems

2006 ◽  
Vol 43 (1) ◽  
pp. 102-113 ◽  
Author(s):  
Albrecht Irle
Keyword(s):  
State Space ◽  
Optimal Stopping ◽  
Stopping Time ◽  
The State ◽  
Optimal Stopping Problem ◽  
Forward Algorithm ◽  
Optimal Stopping Time ◽  
Entrance Time ◽  
Markov Sequence ◽  
Optimal Stopping Problems

We consider the optimal stopping problem for g(Zn), where Zn, n = 1, 2, …, is a homogeneous Markov sequence. An algorithm, called forward improvement iteration, is presented by which an optimal stopping time can be computed. Using an iterative step, this algorithm computes a sequence B0 ⊇ B1 ⊇ B2 ⊇ · · · of subsets of the state space such that the first entrance time into the intersection F of these sets is an optimal stopping time. Various applications are given.

On wald-type optimal stopping for Brownian motion

1997 ◽  
Vol 34 (1) ◽  
pp. 66-73 ◽  
Author(s):  
S. E. Graversen ◽  
G. Peškir
Keyword(s):  
Brownian Motion ◽  
Optimal Stopping ◽  
Stopping Time ◽  
Maximal Inequality ◽  
Stopping Times ◽  
Standard Brownian Motion ◽  
Square Root ◽  
Real Analysis ◽  
Optimal Stopping Problems ◽  
Wald's Identity

The solution is presented to all optimal stopping problems of the form supτE(G(|Β τ |) – cτ), where is standard Brownian motion and the supremum is taken over all stopping times τ for B with finite expectation, while the map G : ℝ+ → ℝ satisfies for some being given and fixed. The optimal stopping time is shown to be the hitting time by the reflecting Brownian motion of the set of all (approximate) maximum points of the map . The method of proof relies upon Wald's identity for Brownian motion and simple real analysis arguments. A simple proof of the Dubins–Jacka–Schwarz–Shepp–Shiryaev (square root of two) maximal inequality for randomly stopped Brownian motion is given as an application.

Optimal stopping of a risk process: model with interest rates

2002 ◽  
Vol 39 (2) ◽  
pp. 261-270 ◽  
Author(s):  
Bogdan Krzysztof Muciek
Keyword(s):  
Interest Rates ◽  
Optimal Stopping ◽  
Process Model ◽  
Stopping Time ◽  
Risk Process ◽  
Risk Theory ◽  
Dynamic Programming Method ◽  
Insurance Company ◽  
Initial Capital ◽  
The Times

The following problem in risk theory is considered. An insurance company, endowed with an initial capital a ≥ 0, receives premiums and pays out claims that occur according to a renewal process {N(t), t ≥ 0}. The times between consecutive claims are i.i.d. The sequence of successive claims is a sequence of i.i.d. random variables. The capital of the company is invested at interest rate α ∊ [0,1], claims increase at rate β ∊ [0,1]. The aim is to find the stopping time that maximizes the capital of the company. A dynamic programming method is used to find the optimal stopping time and to specify the expected capital at that time.

First passage to a general threshold for a process corresponding to sampling at Poisson times

1971 ◽  
Vol 8 (03) ◽  
pp. 573-588 ◽  
Author(s):  
Barry Belkin
Keyword(s):  
Markov Processes ◽  
Classical Case ◽  
First Passage ◽  
Explicit Result ◽  
Threshold Functions ◽  
Linear Threshold Functions ◽  
Linear Threshold ◽  
Special Cases ◽  
Weiner Process ◽  
Semi Markov Processes

The problem of computing the distribution of the time of first passage to a constant threshold for special classes of stochastic processes has been the subject of considerable study. For example, Baxter and Donsker (1957) have considered the problem for processes with stationary, independent increments, Darling and Siegert (1953) for continuous Markov processes, Mehr and McFadden (1965) for Gauss-Markov processes, and Stone (1969) for semi-Markov processes. The results, however, generally express the first passage distribution in terms of transforms which can be inverted only in a relatively few special cases, such as in the classical case of the Weiner process and for certain stable and compound Poisson processes. For linear threshold functions and processes with non-negative interchangeable increments the first passage problem has been studied by Takács (1957) (an explicit result was obtained by Pyke (1959) in the special case of a Poisson process). Again in the case of a linear threshold, an explicit form for the first passage distribution was found by Slepian (1961) for the Weiner process. For the Ornstein-Uhlenbeck process and certain U-shaped thresholds the problem has recently been studied by Daniels (1969).

Optimal stopping of a risk process: model with interest rates

2002 ◽  
Vol 39 (02) ◽  
pp. 261-270 ◽  
Author(s):  
Bogdan Krzysztof Muciek
Keyword(s):  
Interest Rates ◽  
Optimal Stopping ◽  
Process Model ◽  
Stopping Time ◽  
Risk Process ◽  
Risk Theory ◽  
Dynamic Programming Method ◽  
Insurance Company ◽  
Initial Capital ◽  
The Times

The following problem in risk theory is considered. An insurance company, endowed with an initial capital a ≥ 0, receives premiums and pays out claims that occur according to a renewal process {N(t), t ≥ 0}. The times between consecutive claims are i.i.d. The sequence of successive claims is a sequence of i.i.d. random variables. The capital of the company is invested at interest rate α ∊ [0,1], claims increase at rate β ∊ [0,1]. The aim is to find the stopping time that maximizes the capital of the company. A dynamic programming method is used to find the optimal stopping time and to specify the expected capital at that time.

scholarly journals Possibility of Application of the Theory of Semi-Markov Processes to Determine Reliability of Diagnosing Systems / MOŻLIWOŚĆ ZASTOSOWANIA TEORII PROCESÓW SEMI-MARKOWA DO OKREŚLENIA NIEZAWODNOŚCI SYSTEMÓW DIAGNOZUJĄCYCH

2012 ◽  
Vol 24 (1) ◽  
pp. 49-58 ◽  
Author(s):  
Jerzy Girtler
Keyword(s):  
Markov Process ◽  
Markov Processes ◽  
Continuous Time ◽  
Diagnostic Tests ◽  
The State ◽  
Actual State ◽  
Reliability Model ◽  
Discrete State ◽  
Semi Markov Process ◽  
Semi Markov Processes

Abstract The paper provides justification for the necessity to define reliability of diagnosing systems (SDG) in order to develop a diagnosis on state of any technical mechanism being a diagnosed system (SDN). It has been shown that the knowledge of SDG reliability enables defining diagnosis reliability. It has been assumed that the diagnosis reliability can be defined as a diagnosis property which specifies the degree of recognizing by a diagnosing system (SDG) the actual state of the diagnosed system (SDN) which may be any mechanism, and the conditional probability p(S*/K*) of occurrence (existence) of state S* of the mechanism (SDN) as a diagnosis measure provided that at a specified reliability of SDG, the vector K* of values of diagnostic parameters implied by the state, is observed. The probability that SDG is in the state of ability during diagnostic tests and the following diagnostic inferences leading to development of a diagnosis about the SDN state, has been accepted as a measure of SDG reliability. The theory of semi-Markov processes has been used for defining the SDG reliability, that enabled to develop a SDG reliability model in the form of a seven-state (continuous-time discrete-state) semi-Markov process of changes of SDG states.

Sign in / Sign up

Export Citation Format

Share Document