Discrete Time Queueing Systems

1994 ◽  
pp. 275-325
Author(s):  
Mark W. Garrett ◽  
San-Qi Li
Keyword(s):  
Discrete Time ◽  
Queueing Systems

The Computation of Average Optimal Policies in Denumerable State Markov Decision Chains

1997 ◽  
Vol 29 (01) ◽  
pp. 114-137
Author(s):  
Linn I. Sennott
Keyword(s):  
Discrete Time ◽  
Average Cost ◽  
Queueing Systems ◽  
State Spaces ◽  
Original Process ◽  
Optimal Policies ◽  
Finite State ◽  
Markov Decision ◽  
Optimal Average ◽  
Infinite State

This paper studies the expected average cost control problem for discrete-time Markov decision processes with denumerably infinite state spaces. A sequence of finite state space truncations is defined such that the average costs and average optimal policies in the sequence converge to the optimal average cost and an optimal policy in the original process. The theory is illustrated with several examples from the control of discrete-time queueing systems. Numerical results are discussed.

AN OPTIMAL WAIT POLICY IN TWO DISCRETE TIME QUEUEING SYSTEMS

2009 ◽  
pp. 69-77
Author(s):  
JUNJI KOYANAGI ◽  
DAISUKE NANBA ◽  
HAJIME KAWAI
Keyword(s):  
Discrete Time ◽  
Queueing Systems

Performance of discrete-time queueing systems

1993 ◽  
Vol 20 (3) ◽  
pp. 303-320 ◽  
Author(s):  
Herwig Bruneel
Keyword(s):  
Discrete Time ◽  
Queueing Systems

Constrained Discounted Markov Decision Chains

1991 ◽  
Vol 5 (4) ◽  
pp. 463-475 ◽  
Author(s):  
Linn I. Sennott
Keyword(s):  
State Space ◽  
Discrete Time ◽  
Queueing Systems ◽  
Operating Cost ◽  
Countable State Space ◽  
Countable State ◽  
Markov Decision ◽  
Holding Cost

A Markov decision chain with countable state space incurs two types of costs: an operating cost and a holding cost. The objective is to minimize the expected discounted operating cost, subject to a constraint on the expected discounted holding cost. The existence of an optimal randomized simple policy is proved. This is a policy that randomizes between two stationary policies, that differ in at most one state. Several examples from the control of discrete time queueing systems are discussed.

Sign in / Sign up

Export Citation Format

Share Document