scholarly journals Master Equation for the Finite State Space Planning Problem

Author(s):  
Charles Bertucci ◽  
Jean-Michel Lasry ◽  
Pierre-Louis Lions
1996 ◽  
Vol 33 (2) ◽  
pp. 523-535 ◽  
Author(s):  
Søren Asmussen ◽  
Offer Kella

We consider a dam in which the release rate depends both on the state and some modulating process. Conditions for the existence of a limiting distribution are established in terms of an associated risk process. The case where the release rate is a product of the state and the modulating process is given special attention, and in particular explicit formulas are obtained for a finite state space Markov modulation.


Author(s):  
Tabea Waizmann ◽  
Luca Bortolussi ◽  
Andrea Vandin ◽  
Mirco Tribastone

Stochastic reaction networks are a fundamental model to describe interactions between species where random fluctuations are relevant. The master equation provides the evolution of the probability distribution across the discrete state space consisting of vectors of population counts for each species. However, since its exact solution is often elusive, several analytical approximations have been proposed. The deterministic rate equation (DRE) gives a macroscopic approximation as a compact system of differential equations that estimate the average populations for each species, but it may be inaccurate in the case of nonlinear interaction dynamics. Here we propose finite-state expansion (FSE), an analytical method mediating between the microscopic and the macroscopic interpretations of a stochastic reaction network by coupling the master equation dynamics of a chosen subset of the discrete state space with the mean population dynamics of the DRE. An algorithm translates a network into an expanded one where each discrete state is represented as a further distinct species. This translation exactly preserves the stochastic dynamics, but the DRE of the expanded network can be interpreted as a correction to the original one. The effectiveness of FSE is demonstrated in models that challenge state-of-the-art techniques due to intrinsic noise, multi-scale populations and multi-stability.


2005 ◽  
Vol 37 (4) ◽  
pp. 1015-1034 ◽  
Author(s):  
Saul D. Jacka ◽  
Zorana Lazic ◽  
Jon Warren

Let (Xt)t≥0 be a continuous-time irreducible Markov chain on a finite state space E, let v be a map v: E→ℝ\{0}, and let (φt)t≥0 be an additive functional defined by φt=∫0tv(Xs)d s. We consider the case in which the process (φt)t≥0 is oscillating and that in which (φt)t≥0 has a negative drift. In each of these cases, we condition the process (Xt,φt)t≥0 on the event that (φt)t≥0 is nonnegative until time T and prove weak convergence of the conditioned process as T→∞.


Sign in / Sign up

Export Citation Format

Share Document