Asymptotic properties of a finite state continuous time Morkov decision process

1982 ◽  
Author(s):  
Gerd Rode
Keyword(s):  
Continuous Time ◽  
Decision Process ◽  
Asymptotic Properties ◽  
Finite State

On quasi-stationary distributions in absorbing continuous-time finite Markov chains

1967 ◽  
Vol 4 (1) ◽  
pp. 192-196 ◽  
Author(s):  
J. N. Darroch ◽  
E. Seneta
Keyword(s):  
Markov Chains ◽  
Discrete Time ◽  
Continuous Time ◽  
Present Note ◽  
Time Parameter ◽  
Stationary Distributions ◽  
Finite State ◽  
Absorbing Markov Chains ◽  
Finite Markov Chains ◽  
Finite State Space

In a recent paper, the authors have discussed the concept of quasi-stationary distributions for absorbing Markov chains having a finite state space, with the further restriction of discrete time. The purpose of the present note is to summarize the analogous results when the time parameter is continuous.

scholarly journals The Continuous-Time Ehrenfest Process in Term Structure Modelling

2010 ◽  
Vol 47 (3) ◽  
pp. 693-712
Author(s):  
A. Kaplun
Keyword(s):  
Term Structure ◽  
Continuous Time ◽  
Numerical Results ◽  
Vasicek Model ◽  
Finite State

In this paper, a finite-state mean-reverting model for the short rate, based on the continuous-time Ehrenfest process, will be examined. Two explicit pricing formulae for zero-coupon bonds will be derived in the general and special symmetric cases. Its limiting relationship to the Vasicek model will be examined with some numerical results.

scholarly journals Explicit Solution of the Average-Cost Optimality Equation for a Pest-Control Problem

2011 ◽  
Vol 2011 ◽  
pp. 1-11
Author(s):  
Epaminondas G. Kyriakidis
Keyword(s):  
Optimal Control ◽  
Markov Decision Process ◽  
Pest Control ◽  
Continuous Time ◽  
Decision Process ◽  
Average Cost ◽  
Optimality Equation ◽  
Markov Decision ◽  
Average Cost Optimality Equation ◽  
Cost Optimality

We introduce a Markov decision process in continuous time for the optimal control of a simple symmetrical immigration-emigration process by the introduction of total catastrophes. It is proved that a particular control-limit policy is average cost optimal within the class of all stationary policies by verifying that the relative values of this policy are the solution of the corresponding optimality equation.

scholarly journals Conditioning an additive functional of a Markov chain to stay nonnegative. I. Survival for a long time

2005 ◽  
Vol 37 (4) ◽  
pp. 1015-1034 ◽  
Author(s):  
Saul D. Jacka ◽  
Zorana Lazic ◽  
Jon Warren
Keyword(s):  
Markov Chain ◽  
Weak Convergence ◽  
State Space ◽  
Continuous Time ◽  
Additive Functional ◽  
Finite State ◽  
Long Time ◽  
Negative Drift ◽  
Irreducible Markov Chain ◽  
Finite State Space

Let (Xt)t≥0 be a continuous-time irreducible Markov chain on a finite state space E, let v be a map v: E→ℝ\{0}, and let (φt)t≥0 be an additive functional defined by φt=∫0tv(Xs)d s. We consider the case in which the process (φt)t≥0 is oscillating and that in which (φt)t≥0 has a negative drift. In each of these cases, we condition the process (Xt,φt)t≥0 on the event that (φt)t≥0 is nonnegative until time T and prove weak convergence of the conditioned process as T→∞.

A Fluid EOQ Model with Markovian Environment

2015 ◽  
Vol 52 (02) ◽  
pp. 473-489
Author(s):  
Yonit Barron
Keyword(s):  
Continuous Time ◽  
Inventory Level ◽  
Continuous Time Markov Chain ◽  
Holding Costs ◽  
Eoq Model ◽  
Long Run ◽  
Average Criterion ◽  
Production Inventory ◽  
Finite State ◽  
Cost Functionals

We consider a production-inventory model operating in a stochastic environment that is modulated by a finite state continuous-time Markov chain. When the inventory level reaches zero, an order is placed from an external supplier. The costs (purchasing and holding costs) are modulated by the state at the order epoch time. Applying a matrix analytic approach, fluid flow techniques, and martingales, we develop methods to obtain explicit equations for these cost functionals in the discounted case and under the long-run average criterion. Finally, we extend the model to allow backlogging.

HEDGING OPTIONS IN A DOUBLY MARKOV-MODULATED FINANCIAL MARKET VIA STOCHASTIC FLOWS

2019 ◽  
Vol 22 (08) ◽  
pp. 1950047 ◽  
Author(s):  
TAK KUEN SIU ◽  
ROBERT J. ELLIOTT
Keyword(s):  
Markov Chain ◽  
Financial Market ◽  
Continuous Time ◽  
Regime Switching ◽  
Stochastic Flows ◽  
Finite State ◽  
Markov Modulated ◽  
Appreciation Rate ◽  
The Interest Rate ◽  
Over Time

The hedging of a European-style contingent claim is studied in a continuous-time doubly Markov-modulated financial market, where the interest rate of a bond is modulated by an observable, continuous-time, finite-state, Markov chain and the appreciation rate of a risky share is modulated by a continuous-time, finite-state, hidden Markov chain. The first chain describes the evolution of credit ratings of the bond over time while the second chain models the evolution of the hidden state of an underlying economy over time. Stochastic flows of diffeomorphisms are used to derive some hedge quantities, or Greeks, for the claim. A mixed filter-based and regime-switching Black–Scholes partial differential equation is obtained governing the price of the claim. It will be shown that the delta hedge ratio process obtained from stochastic flows is a risk-minimizing, admissible mean-self-financing portfolio process. Both the first-order and second-order Greeks will be considered.

scholarly journals The Continuous-Time Ehrenfest Process in Term Structure Modelling

2010 ◽  
Vol 47 (03) ◽  
pp. 693-712
Author(s):  
A. Kaplun
Keyword(s):  
Term Structure ◽  
Continuous Time ◽  
Numerical Results ◽  
Vasicek Model ◽  
Finite State

In this paper, a finite-state mean-reverting model for the short rate, based on the continuous-time Ehrenfest process, will be examined. Two explicit pricing formulae for zero-coupon bonds will be derived in the general and special symmetric cases. Its limiting relationship to the Vasicek model will be examined with some numerical results.

scholarly journals Estimation for Nonnegative Lévy-Driven Ornstein-Uhlenbeck Processes

2007 ◽  
Vol 44 (04) ◽  
pp. 977-989 ◽  
Author(s):  
Peter J. Brockwell ◽  
Richard A. Davis ◽  
Yu Yang
Keyword(s):  
Continuous Time ◽  
Lévy Process ◽  
Moving Average ◽  
Asymptotic Properties ◽  
Estimation Procedure ◽  
Financial Econometrics ◽  
Levy Process ◽  
Efficient Estimation ◽  
Autoregressive Moving Average ◽  
Continuous Time Processes

Continuous-time autoregressive moving average (CARMA) processes with a nonnegative kernel and driven by a nondecreasing Lévy process constitute a very general class of stationary, nonnegative continuous-time processes. In financial econometrics a stationary Ornstein-Uhlenbeck (or CAR(1)) process, driven by a nondecreasing Lévy process, was introduced by Barndorff-Nielsen and Shephard (2001) as a model for stochastic volatility to allow for a wide variety of possible marginal distributions and the possibility of jumps. For such processes, we take advantage of the nonnegativity of the increments of the driving Lévy process to study the properties of a highly efficient estimation procedure for the parameters when observations are available of the CAR(1) process at uniformly spaced times 0,h,…,Nh. We also show how to reconstruct the background driving Lévy process from a continuously observed realization of the process and use this result to estimate the increments of the Lévy process itself when h is small. Asymptotic properties of the coefficient estimator are derived and the results illustrated using a simulated gamma-driven Ornstein-Uhlenbeck process.

Sign in / Sign up

Export Citation Format

Share Document