Continuous Strong Markov Processes in Dimension One

1998 ◽  
Author(s):  
Sigurd Assing ◽  
Wolfgang M. Schmidt
Keyword(s):  
Markov Processes ◽  
Dimension One ◽  
Strong Markov

Continuous-time allocation indices and their discrete-time approximation

1986 ◽  
Vol 18 (03) ◽  
pp. 724-746
Author(s):  
W. J. R. Eplett
Keyword(s):  
Markov Processes ◽  
Discrete Time ◽  
Time Allocation ◽  
Optimal Policy ◽  
Continuous Time ◽  
Bandit Problems ◽  
Time Approximation ◽  
Discrete Time Approximation ◽  
Strong Markov

The theory of allocation indices for defining the optimal policy in multi-armed bandit problems developed by Gittins is presented in the continuous-time case where the projects (or ‘arms’) are strong Markov processes. Complications peculiar to the continuous-time case are discussed. This motivates investigation of whether approximation of the continuous-time problems by discrete-time versions provides a valid technique with convergent allocation indices and optimal expected rewards. Conditions are presented under which the convergence holds.

scholarly journals Impulse control and expected suprema

2017 ◽  
Vol 49 (1) ◽  
pp. 238-257 ◽  
Author(s):  
Sören Christensen ◽  
Paavo Salminen
Keyword(s):  
Nonlinear Equation ◽  
Markov Processes ◽  
Real Line ◽  
Impulse Control ◽  
Control Problems ◽  
Main Ingredient ◽  
Excessive Functions ◽  
Representation Result ◽  
Optimal Impulse ◽  
Strong Markov

Abstract We consider a class of impulse control problems for general underlying strong Markov processes on the real line, which allows for an explicit solution. The optimal impulse times are shown to be of a threshold type and the optimal threshold is characterised as a solution of a (typically nonlinear) equation. The main ingredient we use is a representation result for excessive functions in terms of expected suprema.

On Strong Markov Processes

1957 ◽  
Vol 2 (2) ◽  
pp. 181-205 ◽  
Author(s):  
A. A. Yushkevich
Keyword(s):  
Markov Processes ◽  
Strong Markov

Strong Markov Processes

1956 ◽  
Vol 1 (1) ◽  
pp. 134-139 ◽  
Author(s):  
E. B. Dynkin ◽  
A. A. Yushkevich
Keyword(s):  
Markov Processes ◽  
Strong Markov
Sign in / Sign up

Export Citation Format

Share Document