Transition Probabilities for Markov Process on a Line Segment Located within a Quarter-Plane

Author(s):  
A.V. Kalinkin

The paper considers a quadratic birth-death Markov process. The points on a line segment located within a quarter-plane represent the states of the random process. We designate the set of vectors that have integer non-negative coordinates as our quarter plane. The process is defined by infinitesimal characteristics, or transition probability densities. These characteristics are determined by a quadratic function of the coordinates at the segment points with integer coordinates. The boundary points of the segment are absorbing; at these points, the random process stops. We investigated a critical case when process jumps are equally probable at the moment of exiting a point. We derived expressions describing transition probabilities of the Markov process as a spectral series. We used a two-dimensional exponential generating function of transition probabilities and a two-dimensional generating function of transition probabilities. The first and second systems of ordinary differential Kolmogorov equations for Markov process transition probabilities are reduced to second-order mixed type partial differential equations for a double generating function. We solve the resulting system of linear equations using separation of variables. The spectrum obtained is discrete. The eigen-functions are expressed in terms of hypergeometric functions. The particular solution constructed is a Fourier series, whose coefficients are derived by means of expo-nential expansion. We employed sums of functional series known in the theory of special functions to construct the exponential expansion required

2015 ◽  
Vol 52 (01) ◽  
pp. 247-257 ◽  
Author(s):  
A. V. Kalinkin ◽  
A. V. Mastikhin

For a Markov two-dimensional death-process of a special class we consider the use of Fourier methods to obtain an exact solution of the Kolmogorov equations for the exponential (double) generating function of the transition probabilities. Using special functions, we obtain an integral representation for the generating function of the transition probabilities. We state the expression of the expectation and variance of the stochastic process and establish a limit theorem.


2015 ◽  
Vol 52 (1) ◽  
pp. 247-257 ◽  
Author(s):  
A. V. Kalinkin ◽  
A. V. Mastikhin

For a Markov two-dimensional death-process of a special class we consider the use of Fourier methods to obtain an exact solution of the Kolmogorov equations for the exponential (double) generating function of the transition probabilities. Using special functions, we obtain an integral representation for the generating function of the transition probabilities. We state the expression of the expectation and variance of the stochastic process and establish a limit theorem.


1969 ◽  
Vol 6 (3) ◽  
pp. 704-707 ◽  
Author(s):  
Thomas L. Vlach ◽  
Ralph L. Disney

The departure process from the GI/G/1 queue is shown to be a semi-Markov process imbedded at departure points with a two-dimensional state space. Transition probabilities for this process are defined and derived from the distributions of the arrival and service processes. The one step transition probabilities and a stationary distribution are obtained for the imbedded two-dimensional Markov chain.


2013 ◽  
Vol 2013 ◽  
pp. 1-11 ◽  
Author(s):  
Yanling Wei ◽  
Mao Wang ◽  
Hamid Reza Karimi ◽  
Jianbin Qiu

This paper investigates the problem ofℋ∞state-feedback control for a class of two-dimensional (2D) discrete-time Markovian jump linear time-delay systems with defective mode information. The mathematical model of the 2D system is established based on the well-known Fornasini-Marchesini local state-space model, and the defective mode information simultaneously consists of the exactly known, partially unknown, and uncertain transition probabilities. By carefully analyzing the features of the transition probability matrices, together with the convexification of uncertain domains, a newℋ∞performance analysis criterion for the underlying system is firstly derived, and then theℋ∞state-feedback controller synthesis is developed via a linearisation technique. It is shown that the controller gains can be constructed by solving a set of linear matrix inequalities. Finally, an illustrative example is provided to verify the effectiveness of the proposed design method.


Author(s):  
Zaiton Haron ◽  
David Oldham

Kertas kerja ini menguji kaedah novel, iaitu Markov untuk tujuan simulasi pengorakan bunyi di jalan raya. Kaedah ini menganggap deretan bangunan di tepi jalan menyerap dan memantulkan bunyi secara berserak. Proses simulasi menganggap proses pengorakan bunyi sebagai proses Markov jujukan pertama bercirikan matrix kebarangkalian perpindahan pancaran bunyi di antara permukaan–permukaan. Keputusan simulasi menggunakan kaedah Markov dibandingkan dengan keputusan diperolehi dari model kommersial RAYNOISE dengan menggunakan pilihan pantulan berserak. Hasil keputusan menunjukkan paras tekanan bunyi di jalan raya yang diramal oleh kaedah Markov mempunyai kesepadanan yang baik dengan ramalan diperolehi dari model RAYNOISE. Ini menunjukkan kaedah Markov mempunyai potensi untuk meramal pantulan berganda bagi keadaan sempadan berserak. Kesan agihan serapan permukaan bangunan juga dikaji, dan dengan skop dan anggapan kajian didapati jalan raya yang mempunyai deretan bangunan berpermukaan menyerap bunyi berupaya menghasilkan pengurangan bunyi kurang dari 1 dB. Kata kunci: Pantulan berserak; proses Markov; kebarangkalian perpindahan; pengorakan bunyi; kawalan bunyi bising This paper examined the capability of the novel approach called Markov in the simulation of sound propagation in streets. The approach assumes that the facades lining the streets absorb and reflect sound diffusely. The simulation process treated the sound propagation process as first order Markov process characterised by a matrix of transition probabilities relating to sound radiation between surfaces. The results of simulation using Markov model were compared with the results obtained from a commercial model, RAYNOISE using the diffuse reflection option. The results showed that sound pressure level in a street predicted by the Markov model was in good agreement with predictions obtained using RAYNOISE model. This suggest that the Markov model has the potential to predict multiple reflections for diffuse boundary conditions. The effects of distribution absorption of building facades were also investigated and within the scope and assumptions in this study; it is shown streets with absorbent building facade result in sound reductions typically less than 1 dB. Key words: Diffuse reflection; Markov process; transition probability; sound propagation; noise control


1969 ◽  
Vol 6 (03) ◽  
pp. 704-707 ◽  
Author(s):  
Thomas L. Vlach ◽  
Ralph L. Disney

The departure process from the GI/G/1 queue is shown to be a semi-Markov process imbedded at departure points with a two-dimensional state space. Transition probabilities for this process are defined and derived from the distributions of the arrival and service processes. The one step transition probabilities and a stationary distribution are obtained for the imbedded two-dimensional Markov chain.


2013 ◽  
Vol 2013 ◽  
pp. 1-13 ◽  
Author(s):  
A. Valor ◽  
F. Caleyo ◽  
L. Alfonso ◽  
J. C. Velázquez ◽  
J. M. Hallen

The stochastic nature of pitting corrosion of metallic structures has been widely recognized. It is assumed that this kind of deterioration retains no memory of the past, so only the current state of the damage influences its future development. This characteristic allows pitting corrosion to be categorized as a Markov process. In this paper, two different models of pitting corrosion, developed using Markov chains, are presented. Firstly, a continuous-time, nonhomogeneous linear growth (pure birth) Markov process is used to model external pitting corrosion in underground pipelines. A closed-form solution of the system of Kolmogorov's forward equations is used to describe the transition probability function in a discrete pit depth space. The transition probability function is identified by correlating the stochastic pit depth mean with the empirical deterministic mean. In the second model, the distribution of maximum pit depths in a pitting experiment is successfully modeled after the combination of two stochastic processes: pit initiation and pit growth. Pit generation is modeled as a nonhomogeneous Poisson process, in which induction time is simulated as the realization of a Weibull process. Pit growth is simulated using a nonhomogeneous Markov process. An analytical solution of Kolmogorov's system of equations is also found for the transition probabilities from the first Markov state. Extreme value statistics is employed to find the distribution of maximum pit depths.


Genetics ◽  
1974 ◽  
Vol 76 (2) ◽  
pp. 367-377
Author(s):  
Takeo Maruyama

ABSTRACT A Markov process (chain) of gene frequency change is derived for a geographically-structured model of a population. The population consists of colonies which are connected by migration. Selection operates in each colony independently. It is shown that there exists a stochastic clock that transforms the originally complicated process of gene frequency change to a random walk which is independent of the geographical structure of the population. The time parameter is a local random time that is dependent on the sample path. In fact, if the alleles are selectively neutral, the time parameter is exactly equal to the sum of the average local genetic variation appearing in the population, and otherwise they are approximately equal. The Kolmogorov forward and backward equations of the process are obtained. As a limit of large population size, a diffusion process is derived. The transition probabilities of the Markov chain and of the diffusion process are obtained explicitly. Certain quantities of biological interest are shown to be independent of the population structure. The quantities are the fixation probability of a mutant, the sum of the average local genetic variation and the variation summed over the generations in which the gene frequency in the whole population assumes a specified value.


Author(s):  
Juan Xiong ◽  
Qiyu Fang ◽  
Jialing Chen ◽  
Yingxin Li ◽  
Huiyi Li ◽  
...  

Background: Postpartum depression (PPD) has been recognized as a severe public health problem worldwide due to its high incidence and the detrimental consequences not only for the mother but for the infant and the family. However, the pattern of natural transition trajectories of PPD has rarely been explored. Methods: In this research, a quantitative longitudinal study was conducted to explore the PPD progression process, providing information on the transition probability, hazard ratio, and the mean sojourn time in the three postnatal mental states, namely normal state, mild PPD, and severe PPD. The multi-state Markov model was built based on 912 depression status assessments in 304 Chinese primiparous women over multiple time points of six weeks postpartum, three months postpartum, and six months postpartum. Results: Among the 608 PPD status transitions from one visit to the next visit, 6.2% (38/608) showed deterioration of mental status from the level at the previous visit; while 40.0% (243/608) showed improvement at the next visit. A subject in normal state who does transition then has a probability of 49.8% of worsening to mild PPD, and 50.2% to severe PPD. A subject with mild PPD who does transition has a 20.0% chance of worsening to severe PPD. A subject with severe PPD is more likely to improve to mild PPD than developing to the normal state. On average, the sojourn time in the normal state, mild PPD, and severe PPD was 64.12, 6.29, and 9.37 weeks, respectively. Women in normal state had 6.0%, 8.5%, 8.7%, and 8.8% chances of progress to severe PPD within three months, nine months, one year, and three years, respectively. Increased all kinds of supports were associated with decreased risk of deterioration from normal state to severe PPD (hazard ratio, HR: 0.42–0.65); and increased informational supports, evaluation of support, and maternal age were associated with alleviation from severe PPD to normal state (HR: 1.46–2.27). Conclusions: The PPD state transition probabilities caused more attention and awareness about the regular PPD screening for postnatal women and the timely intervention for women with mild or severe PPD. The preventive actions on PPD should be conducted at the early stages, and three yearly; at least one yearly screening is strongly recommended. Emotional support, material support, informational support, and evaluation of support had significant positive associations with the prevention of PPD progression transitions. The derived transition probabilities and sojourn time can serve as an importance reference for health professionals to make proactive plans and target interventions for PPD.


Mathematics ◽  
2021 ◽  
Vol 9 (8) ◽  
pp. 868
Author(s):  
Khrystyna Prysyazhnyk ◽  
Iryna Bazylevych ◽  
Ludmila Mitkova ◽  
Iryna Ivanochko

The homogeneous branching process with migration and continuous time is considered. We investigated the distribution of the period-life τ, i.e., the length of the time interval between the moment when the process is initiated by a positive number of particles and the moment when there are no individuals in the population for the first time. The probability generating function of the random process, which describes the behavior of the process within the period-life, was obtained. The boundary theorem for the period-life of the subcritical or critical branching process with migration was found.


Sign in / Sign up

Export Citation Format

Share Document