The analysis of non-Markovian stochastic processes by the inclusion of supplementary variables

1955 ◽  
Vol 51 (3) ◽  
pp. 433-441 ◽  
Author(s):  
D. R. Cox
Keyword(s):  
Differential Equations ◽  
Stochastic Processes ◽  
Markov Processes ◽  
Continuous Time ◽  
Transition Probabilities ◽  
Laplace Transforms ◽  
Supplementary Variables ◽  
Discrete States

ABSTRACTCertain stochastic processes with discrete states in continuous time can be converted into Markov processes by the well-known method of including supplementary variables. It is shown that the resulting integro-differential equations simplify considerably when some distributions associated with the process have rational Laplace transforms. The results justify the formal use of complex transition probabilities. Conditions under which it is likely to be possible to obtain a solution for arbitrary distributions are examined, and the results are related briefly to other methods of investigating these processes.

A use of complex probabilities in the theory of stochastic processes

1955 ◽  
Vol 51 (2) ◽  
pp. 313-319 ◽  
Author(s):  
D. R. Cox
Keyword(s):  
Stochastic Processes ◽  
Exponential Distribution ◽  
Continuous Time ◽  
Transition Probabilities ◽  
Laplace Transforms ◽  
Exponential Distributions ◽  
Discrete States

ABSTRACTThe exponential distribution is very important in the theory of stochastic processes with discrete states in continuous time. A. K. Erlang suggested a method of extending to other distributions methods that apply in the first instance only to exponential distributions. His idea is generalized to cover all distributions with rational Laplace transforms; this involves the formal use of complex transition probabilities. Properties of the method are considered.

A description of the long-term behaviour of absorbing continuous-time Markov chains using a centre manifold

1990 ◽  
Vol 22 (1) ◽  
pp. 111-128 ◽  
Author(s):  
P. K. Pollett ◽  
A. J. Roberts
Keyword(s):  
Differential Equations ◽  
Markov Chains ◽  
Invariant Manifold ◽  
Markov Processes ◽  
Continuous Time ◽  
Transition Probabilities ◽  
Centre Manifold ◽  
Continuous Time Markov Chains ◽  
Linear Differential

We use the notion of an invariant manifold to describe the long-term behaviour of absorbing continuous-time Markov processes with a denumerable infinity of states. We show that there exists an invariant manifold for the forward differential equations and we are able to describe the evolution of the state probabilities on this manifold. Our approach gives rise to a new method for calculating conditional limiting distributions, one which is also appropriate for dealing with processes whose transition probabilities satisfy a system of non-linear differential equations.

A description of the long-term behaviour of absorbing continuous-time Markov chains using a centre manifold

1990 ◽  
Vol 22 (01) ◽  
pp. 111-128 ◽  
Author(s):  
P. K. Pollett ◽  
A. J. Roberts
Keyword(s):  
Differential Equations ◽  
Markov Chains ◽  
Invariant Manifold ◽  
Markov Processes ◽  
Continuous Time ◽  
Transition Probabilities ◽  
Centre Manifold ◽  
Continuous Time Markov Chains ◽  
Linear Differential

We use the notion of an invariant manifold to describe the long-term behaviour of absorbing continuous-time Markov processes with a denumerable infinity of states. We show that there exists an invariant manifold for the forward differential equations and we are able to describe the evolution of the state probabilities on this manifold. Our approach gives rise to a new method for calculating conditional limiting distributions, one which is also appropriate for dealing with processes whose transition probabilities satisfy a system of non-linear differential equations.

Kolmogorov's differential equations for non-stationary, countable state Markov processes with uniformly continuous transition probabilities

1973 ◽  
Vol 73 (1) ◽  
pp. 119-138 ◽  
Author(s):  
Gerald S. Goodman ◽  
S. Johansen
Keyword(s):  
Markov Chain ◽  
Differential Equations ◽  
State Space ◽  
Markov Processes ◽  
Transition Probabilities ◽  
The State ◽  
Continuous Transition ◽  
Countable State Space ◽  
Countable State ◽  
Uniformly Continuous

1. SummaryWe shall consider a non-stationary Markov chain on a countable state space E. The transition probabilities {P(s, t), 0 ≤ s ≤ t <t0 ≤ ∞} are assumed to be continuous in (s, t) uniformly in the state i ε E.

On an approximation made when analysing stochastic processes

1976 ◽  
Vol 13 (4) ◽  
pp. 672-683 ◽  
Author(s):  
Byron J. T. Morgan ◽  
John P. Hinde
Keyword(s):  
Markov Process ◽  
Stochastic Processes ◽  
Continuous Time ◽  
Discrete States

We investigate the effect of a particular mode of approximation by means of four examples of its use; in each case the model approximated is a Markov process with discrete states in continuous time.

scholarly journals A Model of Indel Evolution by Finite-State, Continuous-Time Machines

Genetics ◽  
2020 ◽  
Vol 216 (4) ◽  
pp. 1187-1204
Author(s):  
Ian Holmes
Keyword(s):  
Differential Equations ◽  
Continuous Time ◽  
Markov Models ◽  
Transition Probabilities ◽  
Simulated Data ◽  
Coarse Graining ◽  
Automata Theory ◽  
State Spaces ◽  
Finite State ◽  
Related Approach

We introduce a systematic method of approximating finite-time transition probabilities for continuous-time insertion-deletion models on sequences. The method uses automata theory to describe the action of an infinitesimal evolutionary generator on a probability distribution over alignments, where both the generator and the alignment distribution can be represented by pair hidden Markov models (HMMs). In general, combining HMMs in this way induces a multiplication of their state spaces; to control this, we introduce a coarse-graining operation to keep the state space at a constant size. This leads naturally to ordinary differential equations for the evolution of the transition probabilities of the approximating pair HMM. The TKF91 model emerges as an exact solution to these equations for the special case of single-residue indels. For the more general case of multiple-residue indels, the equations can be solved by numerical integration. Using simulated data, we show that the resulting distribution over alignments, when compared to previous approximations, is a better fit over a broader range of parameters. We also propose a related approach to develop differential equations for sufficient statistics to estimate the underlying instantaneous indel rates by expectation maximization. Our code and data are available at https://github.com/ihh/trajectory-likelihood.

scholarly journals The relationship between intensity and stochastic matrices for continuous-time discrete value stochastic non-homogeneous processes with Markov property

2017 ◽  
Vol 13 (3) ◽  
pp. 7244-7256
Author(s):  
Mi los lawa Sokol
Keyword(s):  
Markov Process ◽  
Stochastic Processes ◽  
Markov Processes ◽  
Continuous Time ◽  
Markov Property ◽  
Time Dependent ◽  
Stochastic Matrices ◽  
Homogeneous Markov Process ◽  
The Relationship ◽  
Time Form

The matrices of non-homogeneous Markov processes consist of time-dependent functions whose values at time form typical intensity matrices. For solvingsome problems they must be changed into stochastic matrices. A stochas-tic matrix for non-homogeneous Markov process consists of time-dependent functions, whose values are probabilities and it depend on assumed time pe- riod. In this paper formulas for these functions are derived. Although the formula is not simple, it allows proving some theorems for Markov stochastic processes, well known for homogeneous processes, but for non-homogeneous ones the proofs of them turned out shorter.

Continuous-time Markov processes, orthogonal polynomials and Lancaster probabilities

2020 ◽  
Vol 24 ◽  
pp. 100-112
Author(s):  
Ramsés H. Mena ◽  
Freddy Palma
Keyword(s):  
Time Dependence ◽  
Orthogonal Polynomials ◽  
Conditional Probability ◽  
Markov Processes ◽  
Continuous Time ◽  
Transition Probabilities ◽  
Spectral Expansion ◽  
Orthogonal Representation ◽  
Probability Structure

This work links the conditional probability structure of Lancaster probabilities to a construction of reversible continuous-time Markov processes. Such a task is achieved by using the spectral expansion of the corresponding transition probabilities in order to introduce a continuous time dependence in the orthogonal representation inherent to Lancaster probabilities. This relationship provides a novel methodology to build continuous-time Markov processes via Lancaster probabilities. Particular cases of well-known models are seen to fall within this approach. As a byproduct, it also unveils new identities associated to well known orthogonal polynomials.

Sign in / Sign up

Export Citation Format

Share Document