A note on bias optimality in controlled queueing systems

The use ofbias optimalityto distinguish among gain optimal policies was recently studied by Haviv and Puterman [1] and extended in Lewiset al.[2]. In [1], upon arrival to anM/M/1 queue, customers offer the gatekeeper a rewardR. If accepted, the gatekeeper immediately receives the reward, but is charged a holding cost,c(s), depending on the number of customers in the system. The gatekeeper, whose objective is to ‘maximize’ rewards, must decide whether to admit the customer. If the customer is accepted, the customer joins the queue and awaits service. Haviv and Puterman [1] showed there can be only two Markovian, stationary, deterministic gain optimal policies and that only the policy which uses thelargercontrol limit is bias optimal. This showed the usefulness of bias optimality to distinguish between gain optimal policies. In the same paper, they conjectured that if the gatekeeper receives the reward uponcompletionof a job instead of upon entry, the bias optimal policy will be the lower control limit. This note confirms that conjecture.

Download Full-text

Bias Optimality in Controlled Queueing Systems

Journal of Applied Probability ◽

10.1239/jap/1032192558 ◽

1998 ◽

Vol 35 (1) ◽

pp. 136-150 ◽

Cited By ~ 37

Author(s):

Moshe Haviv ◽

Martin L. Puterman

Keyword(s):

Queueing System ◽

Queueing Systems ◽

Control Limit ◽

Structure Of Solutions ◽

Stationary Policy ◽

Optimality Equation ◽

Markov Decision ◽

Optimal Stationary Policy ◽

Bias Optimality ◽

Control Limits

This paper studies an admission control M/M/1 queueing system. It shows that the only gain (average) optimal stationary policies with gain and bias which satisfy the optimality equation are of control limit type, that there are at most two and, if there are two, they occur consecutively. Conditions are provided which ensure the existence of two gain optimal control limit policies and are illustrated with an example. The main result is that bias optimality distinguishes these two gain optimal policies and that the larger of the two control limits is the unique bias optimal stationary policy. Consequently it is also Blackwell optimal. This result is established by appealing to the third optimality equation of the Markov decision process and some observations concerning the structure of solutions of the second optimality equation.

Download Full-text

Bias Optimality in Controlled Queueing Systems

Journal of Applied Probability ◽

10.1017/s0021900200014741 ◽

1998 ◽

Vol 35 (01) ◽

pp. 136-150 ◽

Cited By ~ 3

Author(s):

Moshe Haviv ◽

Martin L. Puterman

Keyword(s):

Queueing System ◽

Queueing Systems ◽

Control Limit ◽

Structure Of Solutions ◽

Stationary Policy ◽

Optimality Equation ◽

Markov Decision ◽

Optimal Stationary Policy ◽

Bias Optimality ◽

Control Limits

This paper studies an admission control M/M/1 queueing system. It shows that the only gain (average) optimal stationary policies with gain and bias which satisfy the optimality equation are of control limit type, that there are at most two and, if there are two, they occur consecutively. Conditions are provided which ensure the existence of two gain optimal control limit policies and are illustrated with an example. The main result is that bias optimality distinguishes these two gain optimal policies and that the larger of the two control limits is the unique bias optimal stationary policy. Consequently it is also Blackwell optimal. This result is established by appealing to the third optimality equation of the Markov decision process and some observations concerning the structure of solutions of the second optimality equation.

Download Full-text

OPTIMALITY OF CONTROL LIMIT MAINTENANCE POLICIES UNDER NONSTATIONARY DETERIORATION

Probability in the Engineering and Informational Sciences ◽

10.1017/s026996489913105x ◽

1999 ◽

Vol 13 (1) ◽

pp. 55-70 ◽

Cited By ~ 15

Author(s):

Zvi Benyamini ◽

Uri Yechiali

Keyword(s):

Optimal Policy ◽

Transition Probabilities ◽

Stationary Problem ◽

Control Limit ◽

Maintenance Costs ◽

Operation And Maintenance ◽

Deterioration Process ◽

Maintenance Policies ◽

Deteriorating Systems ◽

Limit Form

Control limit type policies are widely discussed in the literature, particularly regarding the maintenance of deteriorating systems. Previous studies deal mainly with stationary deterioration processes, where costs and transition probabilities depend only on the state of the system, regardless of its cumulative age. In this paper, we consider a nonstationary deterioration process, in which operation and maintenance costs, as well as transition probabilities “deteriorate” with both the system's state and its cumulative age. We discuss conditions under which control limit policies are optimal for such processes and compare them with those used in the analysis of stationary models.Two maintenance models are examined: in the first (as in the majority of classic studies), the only maintenance action allowed is the replacement of the system by a new one. In this case, we show that the nonstationary results are direct generalizations of their counterparts in stationary models. We propose an efficient algorithm for finding the optimal policy, utilizing its control limit form. In the second model we also allow for repairs to better states (without changing the age). In this case, the optimal policy is shown to have the form of a 3-way control limit rule. However, conditions analogous to those used in the stationary problem do not suffice, so additional, more restrictive ones are suggested and discussed.

Download Full-text

The distribution of the maximum number of customers present simultaneously during a busy period for the queueing systems M/G/1 and G/M/1

Journal of Applied Probability ◽

10.1017/s0021900200025328 ◽

1967 ◽

Vol 4 (01) ◽

pp. 162-179 ◽

Cited By ~ 7

Author(s):

J. W. Cohen

Keyword(s):

Busy Period ◽

Transition Probabilities ◽

Queueing Systems ◽

Entrance Time ◽

Distribution Of The Maximum ◽

Number Of Customers

The distribution of the maximum number of customers simultaneously present during a busy period is studied for the queueing systems M/G/1 and G/M/1. These distributions are obtained by using taboo probabilities. Some new relations for transition probabilities and entrance time distributions are derived.

Download Full-text

The Computation of Average Optimal Policies in Denumerable State Markov Decision Chains

Advances in Applied Probability ◽

10.1017/s0001867800027816 ◽

1997 ◽

Vol 29 (01) ◽

pp. 114-137

Author(s):

Linn I. Sennott

Keyword(s):

Discrete Time ◽

Average Cost ◽

Queueing Systems ◽

State Spaces ◽

Original Process ◽

Optimal Policies ◽

Finite State ◽

Markov Decision ◽

Optimal Average ◽

Infinite State

This paper studies the expected average cost control problem for discrete-time Markov decision processes with denumerably infinite state spaces. A sequence of finite state space truncations is defined such that the average costs and average optimal policies in the sequence converge to the optimal average cost and an optimal policy in the original process. The theory is illustrated with several examples from the control of discrete-time queueing systems. Numerical results are discussed.

Download Full-text

On the ergodic distribution of oscillating queueing systems

Journal of Applied Mathematics and Stochastic Analysis ◽

10.1155/s104895330300025x ◽

2003 ◽

Vol 16 (4) ◽

pp. 311-326 ◽

Cited By ~ 6

Author(s):

Mykola Bratiychuk ◽

Andrzej Chydzinski

Keyword(s):

Queueing Systems ◽

Numerical Example ◽

New Class ◽

Ergodic Distribution ◽

Number Of Customers ◽

Special Case

This paper examines a new class of queueing systems and proves a theorem on the existence of the ergodic distribution of the number of customers in such a system. An ergodic distribution is computed explicitly for the special case of a G/M−M/1 system, where the interarrival distribution does not change and both service distributions are exponential. A numerical example is also given.

Download Full-text

Concavity of the throughput of tandem queueing systems with finite buffer storage space

Advances in Applied Probability ◽

10.1017/s0001867800020024 ◽

1990 ◽

Vol 22 (03) ◽

pp. 764-767 ◽

Cited By ~ 7

Author(s):

Ludolf E. Meester ◽

J. George Shanthikumar

Keyword(s):

Queueing System ◽

Queueing Systems ◽

Sample Path ◽

Concave Function ◽

Time Interval ◽

Single Server ◽

Finite Buffer ◽

Buffer Storage ◽

Unlimited Supply ◽

Number Of Customers

We consider a tandem queueing system with m stages and finite intermediate buffer storage spaces. Each stage has a single server and the service times are independent and exponentially distributed. There is an unlimited supply of customers in front of the first stage. For this system we show that the number of customers departing from each of the m stages during the time interval [0, t] for any t ≧ 0 is strongly stochastically increasing and concave in the buffer storage capacities. Consequently the throughput of this tandem queueing system is an increasing and concave function of the buffer storage capacities. We establish this result using a sample path recursion for the departure processes from the m stages of the tandem queueing system, that may be of independent interest. The concavity of the throughput is used along with the reversibility property of tandem queues to obtain the optimal buffer space allocation that maximizes the throughput for a three-stage tandem queue.

Download Full-text

Bias optimal admission control policies for a multiclass nonstationary queueing system

Journal of Applied Probability ◽

10.1017/s0021900200021483 ◽

2002 ◽

Vol 39 (01) ◽

pp. 20-37 ◽

Cited By ~ 8

Author(s):

Mark E. Lewis ◽

Hayriye Ayhan ◽

Robert D. Foley

Keyword(s):

Optimal Policy ◽

Queueing System ◽

Sufficient Conditions ◽

System Capacity ◽

Average Reward ◽

Finite Capacity ◽

Long Run ◽

Optimal Policies ◽

Service Rates ◽

Long Run Average Reward

We consider a finite-capacity queueing system where arriving customers offer rewards which are paid upon acceptance into the system. The gatekeeper, whose objective is to ‘maximize’ rewards, decides if the reward offered is sufficient to accept or reject the arriving customer. Suppose the arrival rates, service rates, and system capacity are changing over time in a known manner. We show that all bias optimal (a refinement of long-run average reward optimal) policies are of threshold form. Furthermore, we give sufficient conditions for the bias optimal policy to be monotonic in time. We show, via a counterexample, that if these conditions are violated, the optimal policy may not be monotonic in time or of threshold form.

Download Full-text

Bounds on the rate of convergence for one class of inhomogeneous Markovian queueing models with possible batch arrivals and services

International Journal of Applied Mathematics and Computer Science ◽

10.2478/amcs-2018-0011 ◽

2018 ◽

Vol 28 (1) ◽

pp. 141-154 ◽

Cited By ~ 12

Author(s):

Alexander Zeifman ◽

Rostislav Razumchik ◽

Yacov Satin ◽

Ksenia Kiseleva ◽

Anna Korotysheva ◽

...

Keyword(s):

Rate Of Convergence ◽

Queueing Systems ◽

Approximation Error ◽

Linear Operators ◽

Upper And Lower Bounds ◽

Logarithmic Norm ◽

Batch Arrivals ◽

State Dependent ◽

The Mean ◽

Number Of Customers

AbstractIn this paper we present a method for the computation of convergence bounds for four classes of multiserver queueing systems, described by inhomogeneous Markov chains. Specifically, we consider an inhomogeneous M/M/S queueing system with possible state-dependent arrival and service intensities, and additionally possible batch arrivals and batch service. A unified approach based on a logarithmic norm of linear operators for obtaining sharp upper and lower bounds on the rate of convergence and corresponding sharp perturbation bounds is described. As a side effect, we show, by virtue of numerical examples, that the approach based on a logarithmic norm can also be used to approximate limiting characteristics (the idle probability and the mean number of customers in the system) of the systems considered with a given approximation error.

Download Full-text