Analysis of a Customer Assignment Model with No State Information

1994 ◽  
Vol 8 (3) ◽  
pp. 419-429 ◽  
Author(s):  
A. Hordijk ◽  
G. M. Koole ◽  
J. A. Loeve
Keyword(s):  
Waiting Times ◽  
Queueing Network ◽  
State Information ◽  
Assignment Rule ◽  
Holding Costs ◽  
Parallel Queues ◽  
Assignment Model ◽  
Long Run ◽  
Markov Decision ◽  
Partial State

In this paper we analyze a queueing network consisting of parallel queues and arriving customers that have to be assigned to one of the queues. The assignment rule may not depend on the numbers of customers in the queues. Our goal is to find a policy that is optimal with respect to the long-run average cost. We will consider two cases: holding costs and waiting times. A recently developed algorithm for Markov decision chains with partial state information is applied. It turns out that the periodic policies found by this algorithm are close, if not equal, to the optimal ones.

Semi-Markov decision processes with polynomial reward

1982 ◽  
Vol 19 (2) ◽  
pp. 301-309 ◽  
Author(s):  
Zvi Rosberg
Keyword(s):  
Transition Period ◽  
Queueing Network ◽  
Decision Processes ◽  
Average Reward ◽  
Network Scheduling ◽  
Long Run ◽  
Markov Decision ◽  
Average Reward Criterion ◽  
Long Run Average Reward ◽  
Reward Criterion

A semi-Markov decision process, with a denumerable multidimensional state space, is considered. At any given state only a finite number of actions can be taken to control the process. The immediate reward earned in one transition period is merely assumed to be bounded by a polynomial and a bound is imposed on a weighted moment of the next state reached in one transition. It is shown that under an ergodicity assumption there is a stationary optimal policy for the long-run average reward criterion. A queueing network scheduling problem, for which previous criteria are inapplicable, is given as an application.

scholarly journals OPTIMAL ADMISSION AND ROUTING WITH CONGESTION-SENSITIVE CUSTOMER CLASSES

2021 ◽  
pp. 1-25
Author(s):  
Ayse Aslan
Keyword(s):  
Sample Path ◽  
Service Station ◽  
Holding Costs ◽  
Long Run ◽  
Optimal Policies ◽  
Markov Decision ◽  
Alternative Service ◽  
Routing Control ◽  
The Difference ◽  
Self Service

This paper considers optimal admission and routing control in multi-class service systems in which customers can either receive quality regular service which is subject to congestion or can receive congestion-free but less desirable service at an alternative service station, which we call the self-service station. We formulate the problem within the Markov decision process framework and focus on characterizing the structure of dynamic optimal policies which maximize the expected long-run rewards. For this, value function and sample path arguments are used. The congestion sensitivity of customers is modeled with class-independent holding costs at the regular service station. The results show how the admission rewards of customer classes affect their priorities at the regular and self-service stations. We explore that the priority for regular service may not only depend on regular service admission rewards of classes but also on the difference between regular and self-service admission rewards. We show that optimal policies have monotonicity properties, regarding the optimal decisions of individual customer classes such that they divide the state space into three connected regions per class.

Semi-Markov decision processes with polynomial reward

1982 ◽  
Vol 19 (02) ◽  
pp. 301-309
Author(s):  
Zvi Rosberg
Keyword(s):  
Transition Period ◽  
Queueing Network ◽  
Decision Processes ◽  
Average Reward ◽  
Network Scheduling ◽  
Long Run ◽  
Markov Decision ◽  
Average Reward Criterion ◽  
Long Run Average Reward ◽  
Reward Criterion

A semi-Markov decision process, with a denumerable multidimensional state space, is considered. At any given state only a finite number of actions can be taken to control the process. The immediate reward earned in one transition period is merely assumed to be bounded by a polynomial and a bound is imposed on a weighted moment of the next state reached in one transition. It is shown that under an ergodicity assumption there is a stationary optimal policy for the long-run average reward criterion. A queueing network scheduling problem, for which previous criteria are inapplicable, is given as an application.

CAPACITY ASSIGNMENT MODEL TO DEFENSE CASCADING FAILURES

2009 ◽  
Vol 20 (07) ◽  
pp. 991-999 ◽  
Author(s):  
J. J. WU ◽  
H. J. SUN ◽  
Z. Y. GAO
Keyword(s):  
Traffic Flow ◽  
Cascading Failures ◽  
Traffic Networks ◽  
Practical Application ◽  
Assignment Rule ◽  
Scale Free ◽  
Capacity Assignment ◽  
Assignment Model ◽  
Equilibrium Assignment ◽  
Theoretical Significance

How to alleviate the damages of cascading failures triggered by the overload of edges/nodes is common in complex networks. To describe the whole cascading failures process from edges overloading to nodes malfunctioning and the dynamic spanning clustering with the evolvement of traffic flow, we propose a capacity assignment model by introducing an equilibrium assignment rule of flow in artificially created scale-free traffic networks. Additionally, the capacity update rule of node is given in this paper. We show that a single failed edge may undergo the cascading failures of nodes, and a small failure has the potential to trigger a global cascade. It is suggested that enhancing the capacity of node is particularly important for the design of any complex network to defense the cascading failures. Meanwhile, it has very important theoretical significance and practical application worthiness in the development of effective methods to alleviate the damage of one or some failed edges/nodes.

scholarly journals Markov Decision Processes with Multiple Long-run Average Objectives

2014 ◽  
Vol 10 (1) ◽  
Author(s):  
Tomáš Brázdil ◽  
Václav Brožek ◽  
Krishnendu Chatterjee ◽  
Vojtěch Forejt ◽  
Antonín Kučera
Keyword(s):  
Markov Decision Processes ◽  
Decision Processes ◽  
Long Run ◽  
Markov Decision

A Fluid EOQ Model with Markovian Environment

2015 ◽  
Vol 52 (02) ◽  
pp. 473-489
Author(s):  
Yonit Barron
Keyword(s):  
Continuous Time ◽  
Inventory Level ◽  
Continuous Time Markov Chain ◽  
Holding Costs ◽  
Eoq Model ◽  
Long Run ◽  
Average Criterion ◽  
Production Inventory ◽  
Finite State ◽  
Cost Functionals

We consider a production-inventory model operating in a stochastic environment that is modulated by a finite state continuous-time Markov chain. When the inventory level reaches zero, an order is placed from an external supplier. The costs (purchasing and holding costs) are modulated by the state at the order epoch time. Applying a matrix analytic approach, fluid flow techniques, and martingales, we develop methods to obtain explicit equations for these cost functionals in the discounted case and under the long-run average criterion. Finally, we extend the model to allow backlogging.

Sign in / Sign up

Export Citation Format

Share Document