Multi-Armed Bandits, Gittins Index, and its Calculation

2014 ◽  
pp. 416-435 ◽  
Author(s):  
Jhelum Chakravorty ◽  
Aditya Mahajan
Keyword(s):  
Gittins Index

On transforming an index for generalised bandit problems

1995 ◽  
Vol 32 (1) ◽  
pp. 168-182 ◽  
Author(s):  
K. D. Glazebrook ◽  
S. Greatrix
Keyword(s):  
Dynamic Programming ◽  
Policy Evaluation ◽  
Gittins Index ◽  
Bandit Problem ◽  
Bandit Problems ◽  
Index Policies

Nash (1980) demonstrated that index policies are optimal for a class of generalised bandit problem. A transform of the index concerned has many of the attributes of the Gittins index. The transformed index is positive-valued, with maximal values yielding optimal actions. It may be characterised as the value of a restart problem and is hence computable via dynamic programming methodologies. The transformed index can also be used in procedures for policy evaluation.

scholarly journals Switching Costs and the Gittins Index

Econometrica ◽  
1994 ◽  
Vol 62 (3) ◽  
pp. 687 ◽  
Author(s):  
Jeffrey S. Banks ◽  
Rangarajan K. Sundaram
Keyword(s):  
Switching Costs ◽  
Gittins Index

scholarly journals Some indexable families of restless bandit problems

2006 ◽  
Vol 38 (3) ◽  
pp. 643-672 ◽  
Author(s):  
K. D. Glazebrook ◽  
D. Ruiz-Hernandez ◽  
C. Kirkbride
Keyword(s):  
Index Theory ◽  
Stochastic Scheduling ◽  
Gittins Index ◽  
Scheduling Problems ◽  
Bandit Problems ◽  
Index Policy ◽  
Restless Bandit ◽  
Machine Maintenance ◽  
State Evolution ◽  
Strong Performance

In 1988 Whittle introduced an important but intractable class of restless bandit problems which generalise the multiarmed bandit problems of Gittins by allowing state evolution for passive projects. Whittle's account deployed a Lagrangian relaxation of the optimisation problem to develop an index heuristic. Despite a developing body of evidence (both theoretical and empirical) which underscores the strong performance of Whittle's index policy, a continuing challenge to implementation is the need to establish that the competing projects all pass an indexability test. In this paper we employ Gittins' index theory to establish the indexability of (inter alia) general families of restless bandits which arise in problems of machine maintenance and stochastic scheduling problems with switching penalties. We also give formulae for the resulting Whittle indices. Numerical investigations testify to the outstandingly strong performance of the index heuristics concerned.

Restless bandits: activity allocation in a changing world

1988 ◽  
Vol 25 (A) ◽  
pp. 287-298 ◽  
Author(s):  
P. Whittle
Keyword(s):  
Lagrange Multiplier ◽  
Gittins Index ◽  
Constant Ratio ◽  
Expected Number ◽  
Restless Bandits ◽  
Changing World

We consider a population of n projects which in general continue to evolve whether in operation or not (although by different rules). It is desired to choose the projects in operation at each instant of time so as to maximise the expected rate of reward, under a constraint upon the expected number of projects in operation. The Lagrange multiplier associated with this constraint defines an index which reduces to the Gittins index when projects not being operated are static. If one is constrained to operate m projects exactly then arguments are advanced to support the conjecture that, for m and n large in constant ratio, the policy of operating the m projects of largest current index is nearly optimal. The index is evaluated for some particular projects.

scholarly journals A Short Proof of the Gittins Index Theorem

1994 ◽  
Vol 4 (1) ◽  
pp. 194-199 ◽  
Author(s):  
John N. Tsitsiklis
Keyword(s):  
Short Proof ◽  
Index Theorem ◽  
Gittins Index
Sign in / Sign up

Export Citation Format

Share Document