scholarly journals Facilitating Numerical Solutions of Inhomogeneous Continuous Time Markov Chains Using Ergodicity Bounds Obtained with Logarithmic Norm Method

Mathematics ◽  
2020 ◽  
Vol 9 (1) ◽  
pp. 42
Author(s):  
Alexander Zeifman ◽  
Yacov Satin ◽  
Ivan Kovalev ◽  
Rostislav Razumchik ◽  
Victor Korolev
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Numerical Solutions ◽  
Time Interval ◽  
Single Server ◽  
Time Varying ◽  
Logarithmic Norm ◽  
Discrete State ◽  
Continuous Time Markov Chains

The problem considered is the computation of the (limiting) time-dependent performance characteristics of one-dimensional continuous-time Markov chains with discrete state space and time varying intensities. Numerical solution techniques can benefit from methods providing ergodicity bounds because the latter can indicate how to choose the position and the length of the “distant time interval” (in the periodic case) on which the solution has to be computed. They can also be helpful whenever the state space truncation is required. In this paper one such analytic method—the logarithmic norm method—is being reviewed. Its applicability is shown within the queueing theory context with three examples: the classical time-varying M/M/2 queue; the time-varying single-server Markovian system with bulk arrivals, queue skipping policy and catastrophes; and the time-varying Markovian bulk-arrival and bulk-service system with state-dependent control. In each case it is shown whether and how the bounds on the rate of convergence can be obtained. Numerical examples are provided.

scholarly journals Two Approaches to the Construction of Perturbation Bounds for Continuous-Time Markov Chains

Mathematics ◽  
2020 ◽  
Vol 8 (2) ◽  
pp. 253 ◽  
Author(s):  
Alexander Zeifman ◽  
Victor Korolev ◽  
Yacov Satin
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Countable State Space ◽  
Perturbation Bounds ◽  
Queuing Models ◽  
Continuous Time Markov Chains ◽  
Quantitative Estimates ◽  
Countable State

This paper is largely a review. It considers two main methods used to study stability and to obtain appropriate quantitative estimates of perturbations of (inhomogeneous) Markov chains with continuous time and a finite or countable state space. An approach is described to the construction of perturbation estimates for the main five classes of such chains associated with queuing models. Several specific models are considered for which the limit characteristics and perturbation bounds for admissible “perturbed” processes are calculated.

Quasi-ergodicity for non-homogeneous continuous-time Markov chains

1989 ◽  
Vol 26 (3) ◽  
pp. 643-648 ◽  
Author(s):  
A. I. Zeifman
Keyword(s):  
Differential Equation ◽  
Markov Chain ◽  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Continuous Time Markov Chain ◽  
Countable State Space ◽  
Continuous Time Markov Chains ◽  
Countable State

We consider a non-homogeneous continuous-time Markov chain X(t) with countable state space. Definitions of uniform and strong quasi-ergodicity are introduced. The forward Kolmogorov system for X(t) is considered as a differential equation in the space of sequences l1. Sufficient conditions for uniform quasi-ergodicity are deduced from this equation. We consider conditions of uniform and strong ergodicity in the case of proportional intensities.

scholarly journals Property-Driven State-Space Coarsening for Continuous Time Markov Chains

2016 ◽  
pp. 3-18 ◽  
Author(s):  
Michalis Michaelides ◽  
Dimitrios Milios ◽  
Jane Hillston ◽  
Guido Sanguinetti
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Continuous Time Markov Chains

State Space Minimization Preserving Embeddings for Continuous-Time Markov Chains

2021 ◽  
pp. 44-61
Author(s):  
Susmoy Das ◽  
Arpit Sharma
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Continuous Time Markov Chains

Markovian couplings staying in arbitrary subsets of the state space

2002 ◽  
Vol 39 (01) ◽  
pp. 197-212 ◽  
Author(s):  
F. Javier López ◽  
Gerardo Sanz
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Continuous Time ◽  
Sufficient Conditions ◽  
Transition Rates ◽  
Arbitrary Subset ◽  
State Spaces ◽  
Continuous Time Markov Chains ◽  
Countable State ◽  
Necessary And Sufficient

Let (X t ) and (Y t ) be continuous-time Markov chains with countable state spaces E and F and let K be an arbitrary subset of E x F. We give necessary and sufficient conditions on the transition rates of (X t ) and (Y t ) for the existence of a coupling which stays in K. We also show that when such a coupling exists, it can be chosen to be Markovian and give a way to construct it. In the case E=F and K ⊆ E x E, we see how the problem of construction of the coupling can be simplified. We give some examples of use and application of our results, including a new concept of lumpability in Markov chains.

American Option Valuation under Continuous-Time Markov Chains

2015 ◽  
Vol 47 (2) ◽  
pp. 378-401 ◽  
Author(s):  
B. Eriksson ◽  
M. R. Pistorius
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Optimal Stopping ◽  
Continuous Time ◽  
Markov Models ◽  
American Options ◽  
American Option ◽  
Discrete State ◽  
Continuous Time Markov Chains ◽  
Exponential Lévy Models

This paper is concerned with the solution of the optimal stopping problem associated to the value of American options driven by continuous-time Markov chains. The value-function of an American option in this setting is characterised as the unique solution (in a distributional sense) of a system of variational inequalities. Furthermore, with continuous and smooth fit principles not applicable in this discrete state-space setting, a novel explicit characterisation is provided of the optimal stopping boundary in terms of the generator of the underlying Markov chain. Subsequently, an algorithm is presented for the valuation of American options under Markov chain models. By application to a suitably chosen sequence of Markov chains, the algorithm provides an approximate valuation of an American option under a class of Markov models that includes diffusion models, exponential Lévy models, and stochastic differential equations driven by Lévy processes. Numerical experiments for a range of different models suggest that the approximation algorithm is flexible and accurate. A proof of convergence is also provided.

American Option Valuation under Continuous-Time Markov Chains

2015 ◽  
Vol 47 (02) ◽  
pp. 378-401
Author(s):  
B. Eriksson ◽  
M. R. Pistorius
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Optimal Stopping ◽  
Continuous Time ◽  
Markov Models ◽  
American Options ◽  
American Option ◽  
Discrete State ◽  
Continuous Time Markov Chains ◽  
Exponential Lévy Models

This paper is concerned with the solution of the optimal stopping problem associated to the value of American options driven by continuous-time Markov chains. The value-function of an American option in this setting is characterised as the unique solution (in a distributional sense) of a system of variational inequalities. Furthermore, with continuous and smooth fit principles not applicable in this discrete state-space setting, a novel explicit characterisation is provided of the optimal stopping boundary in terms of the generator of the underlying Markov chain. Subsequently, an algorithm is presented for the valuation of American options under Markov chain models. By application to a suitably chosen sequence of Markov chains, the algorithm provides an approximate valuation of an American option under a class of Markov models that includes diffusion models, exponential Lévy models, and stochastic differential equations driven by Lévy processes. Numerical experiments for a range of different models suggest that the approximation algorithm is flexible and accurate. A proof of convergence is also provided.

scholarly journals Path integrals for continuous-time Markov chains

2002 ◽  
Vol 39 (4) ◽  
pp. 901-904 ◽  
Author(s):  
P. K. Pollett ◽  
V. T. Stefanov
Keyword(s):  
Markov Chains ◽  
State Space ◽  
Path Integral ◽  
Continuous Time ◽  
Path Integrals ◽  
Countable State Space ◽  
Continuous Time Markov Chains ◽  
Countable State

This note presents a method of evaluating the distribution of a path integral for Markov chains on a countable state space.

Sign in / Sign up

Export Citation Format

Share Document