Filtering of Markov renewal queues, III: semi-Markov processes embedded in feedback queues

1984 ◽  
Vol 16 (2) ◽  
pp. 422-436 ◽  
Author(s):  
Jeffrey J. Hunter
Keyword(s):  
Markov Processes ◽  
Queue Length ◽  
Markov Renewal Process ◽  
Bernoulli Feedback ◽  
State Dependent ◽  
Special Cases ◽  
Transition Points ◽  
Semi Markov Processes ◽  
Feedback Queues ◽  
Birth Death

In Part I (Hunter) a study of feedback queueing models was initiated. For such models the queue-length process embedded at all transition points was formulated as a Markov renewal process (MRP). This led to the observation that the queue-length processes embedded at any of the ‘arrival', ‘departure', ‘feedback', ‘input', ‘output' or ‘external' transition epochs are also MRP. Part I concentrated on the properties of the embedded discrete-time Markov chains. In this part we examine the semi-Markov processes associated with each of these embedded MRP and derive expressions for the stationary distributions associated with their irreducible subspaces. The special cases of birth-death queues with instantaneous state-dependent feedback, M/M/1/N and M/M/1 queues with instantaneous Bernoulli feedback are considered in detail. The results obtained complement those derived in Part II (Hunter) for birth-death queues without feedback.

Filtering of Markov renewal queues, III: semi-Markov processes embedded in feedback queues

1984 ◽  
Vol 16 (02) ◽  
pp. 422-436 ◽  
Author(s):  
Jeffrey J. Hunter
Keyword(s):  
Markov Processes ◽  
Queue Length ◽  
Markov Renewal Process ◽  
Bernoulli Feedback ◽  
State Dependent ◽  
Special Cases ◽  
Transition Points ◽  
Semi Markov Processes ◽  
Feedback Queues ◽  
Birth Death

In Part I (Hunter) a study of feedback queueing models was initiated. For such models the queue-length process embedded at all transition points was formulated as a Markov renewal process (MRP). This led to the observation that the queue-length processes embedded at any of the ‘arrival', ‘departure', ‘feedback', ‘input', ‘output' or ‘external' transition epochs are also MRP. Part I concentrated on the properties of the embedded discrete-time Markov chains. In this part we examine the semi-Markov processes associated with each of these embedded MRP and derive expressions for the stationary distributions associated with their irreducible subspaces. The special cases of birth-death queues with instantaneous state-dependent feedback, M/M/1/N and M/M/1 queues with instantaneous Bernoulli feedback are considered in detail. The results obtained complement those derived in Part II (Hunter) for birth-death queues without feedback.

Filtering of Markov renewal queues, I: Feedback queues

1983 ◽  
Vol 15 (02) ◽  
pp. 349-375 ◽  
Author(s):  
Jeffrey J. Hunter
Keyword(s):  
Queue Length ◽  
Queueing Systems ◽  
Renewal Processes ◽  
Focus Attention ◽  
Bernoulli Feedback ◽  
State Dependent ◽  
Markov Renewal Processes ◽  
Special Cases ◽  
Filtering Procedure ◽  
Feedback Queues

Queueing systems which can be formulated as Markov renewal processes with basic transitions of three types, ‘arrivals', ‘departures' and ‘feedbacks' are examined. The filtering procedure developed for Markov renewal processes by Çinlar (1969) is applied to such queueing models to show that the queue-length processes embedded at any of the ‘arrival', ‘departure', ‘feedback', ‘input', ‘output' or ‘external' transition epochs are also Markov renewal. In this part we focus attention on the derivation of stationary and limiting distributions (when they exist) for each of the embedded discrete-time processes, the embedded Markov chains. These results are applied to birth–death queues with instantaneous state-dependent feedback including the special cases of M/M/1/N and M/M/1 queues with instantaneous Bernoulli feedback.

Filtering of Markov renewal queues, I: Feedback queues

1983 ◽  
Vol 15 (2) ◽  
pp. 349-375 ◽  
Author(s):  
Jeffrey J. Hunter
Keyword(s):  
Queue Length ◽  
Queueing Systems ◽  
Renewal Processes ◽  
Focus Attention ◽  
Bernoulli Feedback ◽  
State Dependent ◽  
Markov Renewal Processes ◽  
Special Cases ◽  
Filtering Procedure ◽  
Feedback Queues

Queueing systems which can be formulated as Markov renewal processes with basic transitions of three types, ‘arrivals', ‘departures' and ‘feedbacks' are examined. The filtering procedure developed for Markov renewal processes by Çinlar (1969) is applied to such queueing models to show that the queue-length processes embedded at any of the ‘arrival', ‘departure', ‘feedback', ‘input', ‘output' or ‘external' transition epochs are also Markov renewal. In this part we focus attention on the derivation of stationary and limiting distributions (when they exist) for each of the embedded discrete-time processes, the embedded Markov chains. These results are applied to birth–death queues with instantaneous state-dependent feedback including the special cases of M/M/1/N and M/M/1 queues with instantaneous Bernoulli feedback.

Filtering of Markov renewal queues, IV: Flow processes in feedback queues

1985 ◽  
Vol 17 (2) ◽  
pp. 386-407 ◽  
Author(s):  
Jeffrey J. Hunter
Keyword(s):  
Queue Length ◽  
Queueing Systems ◽  
Arrival Process ◽  
Markov Renewal Process ◽  
Renewal Processes ◽  
Flow Process ◽  
Special Cases ◽  
Flow Processes ◽  
Transition Points ◽  
The Times

This paper is a continuation of the study of a class of queueing systems where the queue-length process embedded at basic transition points, which consist of ‘arrivals’, ‘departures’ and ‘feedbacks’, is a Markov renewal process (MRP). The filtering procedure of Çinlar (1969) was used in [12] to show that the queue length process embedded separately at ‘arrivals’, ‘departures’, ‘feedbacks’, ‘inputs’ (arrivals and feedbacks), ‘outputs’ (departures and feedbacks) and ‘external’ transitions (arrivals and departures) are also MRP. In this paper expressions for the elements of each Markov renewal kernel are derived, and thence expressions for the distribution of the times between transitions, under stationary conditions, are found for each of the above flow processes. In particular, it is shown that the inter-event distributions for the arrival process and the departure process are the same, with an equivalent result holding for inputs and outputs. Further, expressions for the stationary joint distributions of successive intervals between events in each flow process are derived and interconnections, using the concept of reversed Markov renewal processes, are explored. Conditions under which any of the flow processes are renewal processes or, more particularly, Poisson processes are also investigated. Special cases including, in particular, the M/M/1/N and M/M/1 model with instantaneous Bernoulli feedback, are examined.

Filtering of Markov renewal queues, IV: Flow processes in feedback queues

1985 ◽  
Vol 17 (02) ◽  
pp. 386-407
Author(s):  
Jeffrey J. Hunter
Keyword(s):  
Queue Length ◽  
Queueing Systems ◽  
Arrival Process ◽  
Markov Renewal Process ◽  
Renewal Processes ◽  
Flow Process ◽  
Special Cases ◽  
Flow Processes ◽  
Transition Points ◽  
The Times

This paper is a continuation of the study of a class of queueing systems where the queue-length process embedded at basic transition points, which consist of ‘arrivals’, ‘departures’ and ‘feedbacks’, is a Markov renewal process (MRP). The filtering procedure of Çinlar (1969) was used in [12] to show that the queue length process embedded separately at ‘arrivals’, ‘departures’, ‘feedbacks’, ‘inputs’ (arrivals and feedbacks), ‘outputs’ (departures and feedbacks) and ‘external’ transitions (arrivals and departures) are also MRP. In this paper expressions for the elements of each Markov renewal kernel are derived, and thence expressions for the distribution of the times between transitions, under stationary conditions, are found for each of the above flow processes. In particular, it is shown that the inter-event distributions for the arrival process and the departure process are the same, with an equivalent result holding for inputs and outputs. Further, expressions for the stationary joint distributions of successive intervals between events in each flow process are derived and interconnections, using the concept of reversed Markov renewal processes, are explored. Conditions under which any of the flow processes are renewal processes or, more particularly, Poisson processes are also investigated. Special cases including, in particular, the M/M/1/N and M/M/1 model with instantaneous Bernoulli feedback, are examined.

First passage to a general threshold for a process corresponding to sampling at Poisson times

1971 ◽  
Vol 8 (03) ◽  
pp. 573-588 ◽  
Author(s):  
Barry Belkin
Keyword(s):  
Markov Processes ◽  
Classical Case ◽  
First Passage ◽  
Explicit Result ◽  
Threshold Functions ◽  
Linear Threshold Functions ◽  
Linear Threshold ◽  
Special Cases ◽  
Weiner Process ◽  
Semi Markov Processes

The problem of computing the distribution of the time of first passage to a constant threshold for special classes of stochastic processes has been the subject of considerable study. For example, Baxter and Donsker (1957) have considered the problem for processes with stationary, independent increments, Darling and Siegert (1953) for continuous Markov processes, Mehr and McFadden (1965) for Gauss-Markov processes, and Stone (1969) for semi-Markov processes. The results, however, generally express the first passage distribution in terms of transforms which can be inverted only in a relatively few special cases, such as in the classical case of the Weiner process and for certain stable and compound Poisson processes. For linear threshold functions and processes with non-negative interchangeable increments the first passage problem has been studied by Takács (1957) (an explicit result was obtained by Pyke (1959) in the special case of a Poisson process). Again in the case of a linear threshold, an explicit form for the first passage distribution was found by Slepian (1961) for the Weiner process. For the Ornstein-Uhlenbeck process and certain U-shaped thresholds the problem has recently been studied by Daniels (1969).

An N-policy discrete-time Geo/G/1 queue with modified multiple server vacations and Bernoulli feedback*

2019 ◽  
Vol 53 (2) ◽  
pp. 367-387
Author(s):  
Shaojun Lan ◽  
Yinghui Tang
Keyword(s):  
Discrete Time ◽  
Queue Length ◽  
Length Distribution ◽  
Probability Generating Function ◽  
Renewal Theory ◽  
Function Technique ◽  
Single Server ◽  
Bernoulli Feedback ◽  
Queue Length Distribution ◽  
Special Cases

This paper deals with a single-server discrete-time Geo/G/1 queueing model with Bernoulli feedback and N-policy where the server leaves for modified multiple vacations once the system becomes empty. Applying the law of probability decomposition, the renewal theory and the probability generating function technique, we explicitly derive the transient queue length distribution as well as the recursive expressions of the steady-state queue length distribution. Especially, some corresponding results under special cases are directly obtained. Furthermore, some numerical results are provided for illustrative purposes. Finally, a cost optimization problem is numerically analyzed under a given cost structure.

General optimal stopping theorems for semi-Markov processes

1993 ◽  
Vol 25 (4) ◽  
pp. 825-846 ◽  
Author(s):  
Frans A. Boshuizen ◽  
José M. Gouweleeuw
Keyword(s):  
Markov Processes ◽  
Optimal Stopping ◽  
Stopping Time ◽  
General Setting ◽  
The State ◽  
Shock Models ◽  
Special Cases ◽  
Optimal Stopping Problems ◽  
The Times ◽  
Semi Markov Processes

In this paper, optimal stopping problems for semi-Markov processes are studied in a fairly general setting. In such a process transitions are made from state to state in accordance with a Markov chain, but the amount of time spent in each state is random. The times spent in each state follow a general renewal process. They may depend on the present state as well as on the state into which the next transition is made.Our goal is to maximize the expected net return, which is given as a function of the state at time t minus some cost function. Discounting may or may not be considered. The main theorems (Theorems 3.5 and 3.11) are expressions for the optimal stopping time in the undiscounted and discounted case. These theorems generalize results of Zuckerman [16] and Boshuizen and Gouweleeuw [3]. Applications are given in various special cases.The results developed in this paper can also be applied to semi-Markov shock models, as considered in Taylor [13], Feldman [6] and Zuckerman [15].

Filtering of Markov renewal queues, II: Birth-death queues

1983 ◽  
Vol 15 (2) ◽  
pp. 376-391 ◽  
Author(s):  
Jeffrey J. Hunter
Keyword(s):  
Discrete Time ◽  
Continuous Time ◽  
Queue Length ◽  
Queueing Systems ◽  
Initial Study ◽  
Focused Attention ◽  
Renewal Processes ◽  
Markov Renewal Processes ◽  
Semi Markov Processes ◽  
Birth Death

In this part we extend and particularise results developed by the author in Part I (pp. 349–375) for a class of queueing systems which can be formulated as Markov renewal processes. We examine those models where the basic transition consists of only two types: ‘arrivals' and ‘departures'. The ‘arrival lobby' and ‘departure lobby' queue-length processes are shown, using the results of Part I to be Markov renewal. Whereas the initial study focused attention on the behaviour of the embedded discrete-time Markov chains, in this paper we examine, in detail, the embedded continuous-time semi-Markov processes. The limiting distributions of the queue-length processes in both continuous and discrete time are derived and interrelationships between them are examined in the case of continuous-time birth–death queues including the M/M/1/M and M/M/1 variants. Results for discrete-time birth–death queues are also derived.

First passage to a general threshold for a process corresponding to sampling at Poisson times

1971 ◽  
Vol 8 (3) ◽  
pp. 573-588 ◽  
Author(s):  
Barry Belkin
Keyword(s):  
Markov Processes ◽  
Classical Case ◽  
First Passage ◽  
Explicit Result ◽  
Threshold Functions ◽  
Linear Threshold Functions ◽  
Linear Threshold ◽  
Special Cases ◽  
Weiner Process ◽  
Semi Markov Processes

The problem of computing the distribution of the time of first passage to a constant threshold for special classes of stochastic processes has been the subject of considerable study. For example, Baxter and Donsker (1957) have considered the problem for processes with stationary, independent increments, Darling and Siegert (1953) for continuous Markov processes, Mehr and McFadden (1965) for Gauss-Markov processes, and Stone (1969) for semi-Markov processes. The results, however, generally express the first passage distribution in terms of transforms which can be inverted only in a relatively few special cases, such as in the classical case of the Weiner process and for certain stable and compound Poisson processes. For linear threshold functions and processes with non-negative interchangeable increments the first passage problem has been studied by Takács (1957) (an explicit result was obtained by Pyke (1959) in the special case of a Poisson process). Again in the case of a linear threshold, an explicit form for the first passage distribution was found by Slepian (1961) for the Weiner process. For the Ornstein-Uhlenbeck process and certain U-shaped thresholds the problem has recently been studied by Daniels (1969).

Sign in / Sign up

Export Citation Format

Share Document