Matrix product-form solutions for Markov chains with a tree structure

1994 ◽  
Vol 26 (04) ◽  
pp. 965-987 ◽  
Author(s):  
Raymond W. Yeung ◽  
Bhaskar Sengupta
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Closed Form Solution ◽  
Steady State Solution ◽  
Form Solution ◽  
Product Form ◽  
Matrix Product ◽  
Preemptive Resume ◽  
Finite Set ◽  
Markov Modulated

We have two aims in this paper. First, we generalize the well-known theory of matrix-geometric methods of Neuts to more complicated Markov chains. Second, we use the theory to solve a last-come-first-served queue with a generalized preemptive resume (LCFS-GPR) discipline. The structure of the Markov chain considered in this paper is one in which one of the variables can take values in a countable set, which is arranged in the form of a tree. The other variable takes values from a finite set. Each node of the tree can branch out into d other nodes. The steady-state solution of this Markov chain has a matrix product-form, which can be expressed as a function of d matrices Rl,· ··, Rd. We then use this theory to solve a multiclass LCFS-GPR queue, in which the service times have PH-distributions and arrivals are according to the Markov modulated Poisson process. In this discipline, when a customer's service is preempted in phase j (due to a new arrival), the resumption of service at a later time could take place in a phase which depends on j. We also obtain a closed form solution for the stationary distribution of an LCFS-GPR queue when the arrivals are Poisson. This result generalizes the known result on a LCFS preemptive resume queue, which can be obtained from Kelly's symmetric queue.

Matrix product-form solutions for Markov chains with a tree structure

1994 ◽  
Vol 26 (4) ◽  
pp. 965-987 ◽  
Author(s):  
Raymond W. Yeung ◽  
Bhaskar Sengupta
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Closed Form Solution ◽  
Steady State Solution ◽  
Form Solution ◽  
Product Form ◽  
Matrix Product ◽  
Preemptive Resume ◽  
Finite Set ◽  
Markov Modulated

We have two aims in this paper. First, we generalize the well-known theory of matrix-geometric methods of Neuts to more complicated Markov chains. Second, we use the theory to solve a last-come-first-served queue with a generalized preemptive resume (LCFS-GPR) discipline. The structure of the Markov chain considered in this paper is one in which one of the variables can take values in a countable set, which is arranged in the form of a tree. The other variable takes values from a finite set. Each node of the tree can branch out into d other nodes. The steady-state solution of this Markov chain has a matrix product-form, which can be expressed as a function of d matrices Rl,· ··, Rd. We then use this theory to solve a multiclass LCFS-GPR queue, in which the service times have PH-distributions and arrivals are according to the Markov modulated Poisson process. In this discipline, when a customer's service is preempted in phase j (due to a new arrival), the resumption of service at a later time could take place in a phase which depends on j. We also obtain a closed form solution for the stationary distribution of an LCFS-GPR queue when the arrivals are Poisson. This result generalizes the known result on a LCFS preemptive resume queue, which can be obtained from Kelly's symmetric queue.

On identifiability and order of continuous-time aggregated Markov chains, Markov-modulated Poisson processes, and phase-type distributions

1996 ◽  
Vol 33 (3) ◽  
pp. 640-653 ◽  
Author(s):  
Tobias Rydén
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Continuous Time ◽  
Poisson Processes ◽  
Hitting Times ◽  
Phase Type ◽  
Phase Type Distributions ◽  
Minimum Number ◽  
Markov Modulated ◽  
Two Parameters

An aggregated Markov chain is a Markov chain for which some states cannot be distinguished from each other by the observer. In this paper we consider the identifiability problem for such processes in continuous time, i.e. the problem of determining whether two parameters induce identical laws for the observable process or not. We also study the order of a continuous-time aggregated Markov chain, which is the minimum number of states needed to represent it. In particular, we give a lower bound on the order. As a by-product, we obtain results of this kind also for Markov-modulated Poisson processes, i.e. doubly stochastic Poisson processes whose intensities are directed by continuous-time Markov chains, and phase-type distributions, which are hitting times in finite-state Markov chains.

scholarly journals Hitting Time and Inverse Problems for Markov Chains

2008 ◽  
Vol 45 (03) ◽  
pp. 640-649
Author(s):  
Victor de la Peña ◽  
Henryk Gzyl ◽  
Patrick McDonald
Keyword(s):  
Markov Chain ◽  
Inverse Problems ◽  
Markov Chains ◽  
Transition Probabilities ◽  
Hitting Time ◽  
First Hitting Time ◽  
Joint Distributions ◽  
Finite Set

Let W n be a simple Markov chain on the integers. Suppose that X n is a simple Markov chain on the integers whose transition probabilities coincide with those of W n off a finite set. We prove that there is an M > 0 such that the Markov chain W n and the joint distributions of the first hitting time and first hitting place of X n started at the origin for the sets {-M, M} and {-(M + 1), (M + 1)} algorithmically determine the transition probabilities of X n .

scholarly journals Wind energy forecasts in calculation of expected energy not served

2021 ◽  
Author(s):  
Richard Sun
Keyword(s):  
Markov Chain ◽  
Wind Energy ◽  
Optimal Power Flow ◽  
Markov Chain Model ◽  
Moving Average ◽  
Closed Form Solution ◽  
Arma Model ◽  
Accurate Method ◽  
Form Solution ◽  
Chain Model

The stochastic nature of wind energy generation introduces uncertainties and risk in generation schedules computed using optimal power flow (OPF). This risk is quantified as expected energy not served (EENS) and computed via an error distribution found for each hourly forecast. This thesis produces an accurate method of estimating EENS that is also suitable for real-time OPF calculation. This thesis examines two statistical predictive models used to forecast hourly production of wind energy generators (WEGs), Markov chain model, and auto-regressive moving-average (ARMA) model, and their effects on EENS. Persistence model is used as a benchmark for comparison. For persistence and ARMA models, both Gaussian and Cauchy error distributions are used to compute EENS via a closed-form solution that reduces computational complexity. Markov chain and ARMA both provide accurate forecasts of WEG power generation though Markov Chain model performs significantly better. The Markov chain model also produces the most accurate EENS estimate of the three models.

scholarly journals An exact and explicit formula for pricing lookback options with regime switching

2021 ◽  
Vol 0 (0) ◽  
pp. 0
Author(s):  
Leunglung Chan ◽  
Song-Ping Zhu
Keyword(s):  
Regime Switching ◽  
Closed Form Solution ◽  
Form Solution ◽  
Risky Asset ◽  
Chain Process ◽  
Switching Model ◽  
Lookback Options ◽  
State Regime ◽  
Markov Modulated ◽  
Appreciation Rate

<p style='text-indent:20px;'>This paper investigates the pricing of European-style lookback options when the price dynamics of the underlying risky asset are assumed to follow a Markov-modulated Geometric Brownian motion; that is, the appreciation rate and the volatility of the underlying risky asset depend on states of the economy described by a continuous-time Markov chain process. We derive an exact, explicit and closed-form solution for European-style lookback options in a two-state regime switching model.</p>

scholarly journals Wind energy forecasts in calculation of expected energy not served

2021 ◽  
Author(s):  
Richard Sun
Keyword(s):  
Markov Chain ◽  
Wind Energy ◽  
Optimal Power Flow ◽  
Markov Chain Model ◽  
Moving Average ◽  
Closed Form Solution ◽  
Arma Model ◽  
Accurate Method ◽  
Form Solution ◽  
Chain Model

The stochastic nature of wind energy generation introduces uncertainties and risk in generation schedules computed using optimal power flow (OPF). This risk is quantified as expected energy not served (EENS) and computed via an error distribution found for each hourly forecast. This thesis produces an accurate method of estimating EENS that is also suitable for real-time OPF calculation. This thesis examines two statistical predictive models used to forecast hourly production of wind energy generators (WEGs), Markov chain model, and auto-regressive moving-average (ARMA) model, and their effects on EENS. Persistence model is used as a benchmark for comparison. For persistence and ARMA models, both Gaussian and Cauchy error distributions are used to compute EENS via a closed-form solution that reduces computational complexity. Markov chain and ARMA both provide accurate forecasts of WEG power generation though Markov Chain model performs significantly better. The Markov chain model also produces the most accurate EENS estimate of the three models.

scholarly journals Hitting Time and Inverse Problems for Markov Chains

2008 ◽  
Vol 45 (3) ◽  
pp. 640-649
Author(s):  
Victor de la Peña ◽  
Henryk Gzyl ◽  
Patrick McDonald
Keyword(s):  
Markov Chain ◽  
Inverse Problems ◽  
Markov Chains ◽  
Transition Probabilities ◽  
Hitting Time ◽  
First Hitting Time ◽  
Joint Distributions ◽  
Finite Set

Let Wn be a simple Markov chain on the integers. Suppose that Xn is a simple Markov chain on the integers whose transition probabilities coincide with those of Wn off a finite set. We prove that there is an M > 0 such that the Markov chain Wn and the joint distributions of the first hitting time and first hitting place of Xn started at the origin for the sets {-M, M} and {-(M + 1), (M + 1)} algorithmically determine the transition probabilities of Xn.

Sign in / Sign up

Export Citation Format

Share Document