On a Two-armed Bandit Problem with both Continuous and Impulse Actions and Discounted Rewards

1993 ◽  
pp. 249-266
Author(s):  
A. A. Yushkevich
Keyword(s):  
Bandit Problem ◽  
Discounted Rewards

Some results for the two armed bandit problem

Optimization ◽  
1976 ◽  
Vol 7 (3) ◽  
pp. 471-475 ◽  
Author(s):  
P.W. Jones
Keyword(s):  
Bandit Problem

Exploration by Pigeons on an N-Armed Bandit Problem

2007 ◽  
Author(s):  
Dennis Garlick ◽  
Aaron P. Blaisdell
Keyword(s):  
Bandit Problem

scholarly journals Fast Two-Stage Computation of an Index Policy for Multi-Armed Bandits with Setup Delays

Mathematics ◽  
2020 ◽  
Vol 9 (1) ◽  
pp. 52
Author(s):  
José Niño-Mora
Keyword(s):  
Numerical Study ◽  
Arithmetic Operation ◽  
Bandit Problem ◽  
Index Policy ◽  
Two Stage ◽  
Second Stage ◽  
Whittle Index ◽  
Set Up ◽  
Computing Method ◽  
Special Case

We consider the multi-armed bandit problem with penalties for switching that include setup delays and costs, extending the former results of the author for the special case with no switching delays. A priority index for projects with setup delays that characterizes, in part, optimal policies was introduced by Asawa and Teneketzis in 1996, yet without giving a means of computing it. We present a fast two-stage index computing method, which computes the continuation index (which applies when the project has been set up) in a first stage and certain extra quantities with cubic (arithmetic-operation) complexity in the number of project states and then computes the switching index (which applies when the project is not set up), in a second stage, with quadratic complexity. The approach is based on new methodological advances on restless bandit indexation, which are introduced and deployed herein, being motivated by the limitations of previous results, exploiting the fact that the aforementioned index is the Whittle index of the project in its restless reformulation. A numerical study demonstrates substantial runtime speed-ups of the new two-stage index algorithm versus a general one-stage Whittle index algorithm. The study further gives evidence that, in a multi-project setting, the index policy is consistently nearly optimal.

On transforming an index for generalised bandit problems

1995 ◽  
Vol 32 (1) ◽  
pp. 168-182 ◽  
Author(s):  
K. D. Glazebrook ◽  
S. Greatrix
Keyword(s):  
Dynamic Programming ◽  
Policy Evaluation ◽  
Gittins Index ◽  
Bandit Problem ◽  
Bandit Problems ◽  
Index Policies

Nash (1980) demonstrated that index policies are optimal for a class of generalised bandit problem. A transform of the index concerned has many of the attributes of the Gittins index. The transformed index is positive-valued, with maximal values yielding optimal actions. It may be characterised as the value of a restart problem and is hence computable via dynamic programming methodologies. The transformed index can also be used in procedures for policy evaluation.

scholarly journals Encoding of Time-Discounted Rewards in Orbitofrontal Cortex Is Independent of Value Representation

Neuron ◽  
2006 ◽  
Vol 51 (4) ◽  
pp. 509-520 ◽  
Author(s):  
Matthew R. Roesch ◽  
Adam R. Taylor ◽  
Geoffrey Schoenbaum
Keyword(s):  
Orbitofrontal Cortex ◽  
Discounted Rewards ◽  
Value Representation

Gaussian One-Armed Bandit Problem

2021 ◽  
Author(s):  
Alexander Kolnogorov
Keyword(s):  
Bandit Problem
Sign in / Sign up

Export Citation Format

Share Document