scholarly journals Analysis of Markov Jump Processes under Terminal Constraints

2021 ◽  
pp. 210-229
Author(s):  
Michael Backenköhler ◽  
Luca Bortolussi ◽  
Gerrit Großmann ◽  
Verena Wolf
Keyword(s):  
State Space ◽  
Rare Event ◽  
Jump Processes ◽  
Markov Jump ◽  
Stochastic Filtering ◽  
Markov Jump Processes ◽  
Inference Problems ◽  
Terminal Constraints ◽  
Finite State ◽  
Endpoint Constraints

AbstractMany probabilistic inference problems such as stochastic filtering or the computation of rare event probabilities require model analysis under initial and terminal constraints. We propose a solution to this bridging problem for the widely used class of population-structured Markov jump processes. The method is based on a state-space lumping scheme that aggregates states in a grid structure. The resulting approximate bridging distribution is used to iteratively refine relevant and truncate irrelevant parts of the state-space. This way, the algorithm learns a well-justified finite-state projection yielding guaranteed lower bounds for the system behavior under endpoint constraints. We demonstrate the method’s applicability to a wide range of problems such as Bayesian inference and the analysis of rare events.

scholarly journals Automated State-Dependent Importance Sampling for Markov Jump Processes via Sampling from the Zero-Variance Distribution

2014 ◽  
Vol 51 (3) ◽  
pp. 741-755
Author(s):  
Adam W. Grace ◽  
Dirk P. Kroese ◽  
Werner Sandmann
Keyword(s):  
Importance Sampling ◽  
Rare Event ◽  
Jump Process ◽  
Small Sample ◽  
Telecommunication Networks ◽  
Jump Processes ◽  
Markov Jump ◽  
Markov Jump Processes ◽  
State Dependent ◽  
Wide Range

Many complex systems can be modeled via Markov jump processes. Applications include chemical reactions, population dynamics, and telecommunication networks. Rare-event estimation for such models can be difficult and is often computationally expensive, because typically many (or very long) paths of the Markov jump process need to be simulated in order to observe the rare event. We present a state-dependent importance sampling approach to this problem that is adaptive and uses Markov chain Monte Carlo to sample from the zero-variance importance sampling distribution. The method is applicable to a wide range of Markov jump processes and achieves high accuracy, while requiring only a small sample to obtain the importance parameters. We demonstrate its efficiency through benchmark examples in queueing theory and stochastic chemical kinetics.

Markov jump processes with a singularity

2000 ◽  
Vol 32 (03) ◽  
pp. 779-799 ◽  
Author(s):  
Ole E. Barndorff-Nielsen ◽  
Fred Espen Benth ◽  
Jens Ledet Jensen
Keyword(s):  
State Space ◽  
Laser Cooling ◽  
Quantum Physics ◽  
Jump Processes ◽  
Main Question ◽  
Markov Jump ◽  
Markov Jump Processes ◽  
Equation Problem ◽  
Continuous State Space ◽  
Continuous State

Certain types of Markov jump processes x(t) with continuous state space and one or more absorbing states are studied. Cases where the transition rate in state x is of the form λ(x) = |x|δ in a neighbourhood of the origin in ℝ d are considered, in particular. This type of problem arises from quantum physics in the study of laser cooling of atoms, and the present paper connects to recent work in the physics literature. The main question addressed is that of the asymptotic behaviour of x(t) near the origin for large t. The study involves solution of a renewal equation problem in continuous state space.

IFRA Properties of Some Markov Jump Processes with General State Space

1987 ◽  
Vol 12 (3) ◽  
pp. 562-568 ◽  
Author(s):  
Moshe Shaked ◽  
J. George Shanthikumar
Keyword(s):  
State Space ◽  
Jump Processes ◽  
General State ◽  
Markov Jump ◽  
Markov Jump Processes ◽  
General State Space

Markov jump processes with a singularity

2000 ◽  
Vol 32 (3) ◽  
pp. 779-799 ◽  
Author(s):  
Ole E. Barndorff-Nielsen ◽  
Fred Espen Benth ◽  
Jens Ledet Jensen
Keyword(s):  
State Space ◽  
Laser Cooling ◽  
Quantum Physics ◽  
Jump Processes ◽  
Main Question ◽  
Markov Jump ◽  
Markov Jump Processes ◽  
Equation Problem ◽  
Continuous State Space ◽  
Continuous State

Certain types of Markov jump processes x(t) with continuous state space and one or more absorbing states are studied. Cases where the transition rate in state x is of the form λ(x) = |x|δ in a neighbourhood of the origin in ℝd are considered, in particular. This type of problem arises from quantum physics in the study of laser cooling of atoms, and the present paper connects to recent work in the physics literature. The main question addressed is that of the asymptotic behaviour of x(t) near the origin for large t. The study involves solution of a renewal equation problem in continuous state space.

Automated State-Dependent Importance Sampling for Markov Jump Processes via Sampling from the Zero-Variance Distribution

2014 ◽  
Vol 51 (03) ◽  
pp. 741-755
Author(s):  
Adam W. Grace ◽  
Dirk P. Kroese ◽  
Werner Sandmann
Keyword(s):  
Importance Sampling ◽  
Rare Event ◽  
Jump Process ◽  
Small Sample ◽  
Telecommunication Networks ◽  
Jump Processes ◽  
Markov Jump ◽  
Markov Jump Processes ◽  
State Dependent ◽  
Wide Range

Many complex systems can be modeled via Markov jump processes. Applications include chemical reactions, population dynamics, and telecommunication networks. Rare-event estimation for such models can be difficult and is often computationally expensive, because typically many (or very long) paths of the Markov jump process need to be simulated in order to observe the rare event. We present a state-dependent importance sampling approach to this problem that is adaptive and uses Markov chain Monte Carlo to sample from the zero-variance importance sampling distribution. The method is applicable to a wide range of Markov jump processes and achieves high accuracy, while requiring only a small sample to obtain the importance parameters. We demonstrate its efficiency through benchmark examples in queueing theory and stochastic chemical kinetics.

Asymptotics of the Exit Distribution for Markov Jump Processes; Application to Atm

1994 ◽  
Vol 46 (06) ◽  
pp. 1238-1262 ◽  
Author(s):  
I. Iscoe ◽  
D. Mcdonald ◽  
K. Qian
Keyword(s):  
Busy Period ◽  
Rare Event ◽  
Jump Process ◽  
Jump Processes ◽  
Markov Jump ◽  
Markov Jump Process ◽  
Markov Jump Processes ◽  
Exit Distribution ◽  
Simulation Time

Abstract We approximate the exit distribution of a Markov jump process into a set of forbidden states and we apply these general results to an ATM multiplexor. In this case the forbidden states represent an overloaded multiplexor. Statistics for this overload or busy period are difficult to obtain since this is such a rare event. Starting from the approximate exit distribution, one may simulate the busy period without wasting simulation time waiting for the overload to occur.

scholarly journals Jump processes as generalized gradient flows

2022 ◽  
Vol 61 (1) ◽  
Author(s):  
Mark A. Peletier ◽  
Riccarda Rossi ◽  
Giuseppe Savaré ◽  
Oliver Tse
Keyword(s):  
State Space ◽  
Uniqueness Result ◽  
Jump Processes ◽  
Gradient Flows ◽  
Kolmogorov Equations ◽  
Generalized Gradient ◽  
Markov Jump ◽  
Polish Spaces ◽  
Markov Jump Processes ◽  
Definition Of

AbstractWe have created a functional framework for a class of non-metric gradient systems. The state space is a space of nonnegative measures, and the class of systems includes the Forward Kolmogorov equations for the laws of Markov jump processes on Polish spaces. This framework comprises a definition of a notion of solutions, a method to prove existence, and an archetype uniqueness result. We do this by using only the structure that is provided directly by the dissipation functional, which need not be homogeneous, and we do not appeal to any metric structure.

Sign in / Sign up

Export Citation Format

Share Document