scholarly journals An Online Minimax Optimal Algorithm for Adversarial Multiarmed Bandit Problem

2018 ◽  
Vol 29 (11) ◽  
pp. 5565-5580 ◽  
Author(s):  
Kaan Gokcesu ◽  
Suleyman Serdar Kozat
Keyword(s):  
Optimal Algorithm ◽  
Bandit Problem ◽  
Multiarmed Bandit

Scheduling Jobs That Are Subject to Deterministic Due Dates and Have Deteriorating Expected Rewards

1997 ◽  
Vol 11 (1) ◽  
pp. 65-78 ◽  
Author(s):  
Takashi Ishikida ◽  
Yat-wah Wan
Keyword(s):  
Single Server ◽  
Due Dates ◽  
Bandit Problem ◽  
Scheduling Policy ◽  
Total Reward ◽  
Multiarmed Bandit

A single server processes jobs that can yield rewards but expire on predetermined dates. Expected immediate rewards from each job are deteriorating. The instance is formulated as a multiarmed bandit problem, and an index-based scheduling policy is shown to maximize the expected total reward.

scholarly journals Index policies for discounted bandit problems with availability constraints

2008 ◽  
Vol 40 (02) ◽  
pp. 377-400 ◽  
Author(s):  
Savas Dayanik ◽  
Warren Powell ◽  
Kazutoshi Yamazaki
Keyword(s):  
Bandit Problem ◽  
Bandit Problems ◽  
Index Policy ◽  
State Action ◽  
Index Policies ◽  
Availability Constraints ◽  
Whittle Index ◽  
Multiarmed Bandit

A multiarmed bandit problem is studied when the arms are not always available. The arms are first assumed to be intermittently available with some state/action-dependent probabilities. It is proven that no index policy can attain the maximum expected total discounted reward in every instance of that problem. The Whittle index policy is derived, and its properties are studied. Then it is assumed that the arms may break down, but repair is an option at some cost, and the new Whittle index policy is derived. Both problems are indexable. The proposed index policies cannot be dominated by any other index policy over all multiarmed bandit problems considered here. Whittle indices are evaluated for Bernoulli arms with unknown success probabilities.

scholarly journals Optimal Posted-Price Mechanism in Microtask Crowdsourcing

2017 ◽  
Author(s):  
Zehong Hu ◽  
Jie Zhang
Keyword(s):  
Upper Bound ◽  
Optimal Algorithm ◽  
Price Mechanism ◽  
Bandit Problem ◽  
Price Data ◽  
Real Price ◽  
Price Range ◽  
Posted Price

Posted-price mechanisms are widely-adopted to decide the price of tasks in popular microtask crowdsourcing. In this paper, we propose a novel posted-price mechanism which not only outperforms existing mechanisms on performance but also avoids their need of a finite price range. The advantages are achieved by converting the pricing problem into a multi-armed bandit problem and designing an optimal algorithm to exploit the unique features of microtask crowdsourcing. We theoretically show the optimality of our algorithm and prove that the performance upper bound can be achieved without the need of a prior price range. We also conduct extensive experiments using real price data to verify the advantages and practicability of our mechanism.

Matching While Learning

2021 ◽  
Author(s):  
Ramesh Johari ◽  
Vijay Kamble ◽  
Yash Kanoria
Keyword(s):  
Labor Market ◽  
Shadow Price ◽  
Cold Start ◽  
Bandit Problem ◽  
Different Types ◽  
The Future ◽  
Limited Supply ◽  
Cold Starts ◽  
Cold Start Problem ◽  
Multiarmed Bandit

Platforms face a cold start problem whenever new users arrive: namely, the platform must learn attributes of new users (explore) in order to match them better in the future (exploit). How should a platform handle cold starts when there are limited quantities of the items being recommended? For instance, how should a labor market platform match workers to jobs over the lifetime of the worker, given a limited supply of jobs? In this setting, there is one multiarmed bandit problem for each worker, coupled together by the constrained supply of jobs of different types. A solution is developed to this problem. It is found that the platform should estimate a shadow price for each job type, and for each worker, adjust payoffs by these prices (i) to balance learning with payoffs early on and (ii) to myopically match them thereafter.

A Note on the Equivalence of Upper Confidence Bounds and Gittins Indices for Patient Agents

2020 ◽  
Author(s):  
Daniel Russo
Keyword(s):  
Posterior Distribution ◽  
Error Term ◽  
Discount Factor ◽  
Gittins Index ◽  
Confidence Bound ◽  
Bandit Problem ◽  
Confidence Bounds ◽  
Upper Confidence Bound ◽  
Gittins Indices ◽  
Multiarmed Bandit

This note gives a short, self-contained proof of a sharp connection between Gittins indices and Bayesian upper confidence bound algorithms. I consider a Gaussian multiarmed bandit problem with discount factor [Formula: see text]. The Gittins index of an arm is shown to equal the [Formula: see text]-quantile of the posterior distribution of the arm's mean plus an error term that vanishes as [Formula: see text]. In this sense, for sufficiently patient agents, a Gittins index measures the highest plausible mean-reward of an arm in a manner equivalent to an upper confidence bound.

scholarly journals Index policies for discounted bandit problems with availability constraints

2008 ◽  
Vol 40 (2) ◽  
pp. 377-400 ◽  
Author(s):  
Savas Dayanik ◽  
Warren Powell ◽  
Kazutoshi Yamazaki
Keyword(s):  
Bandit Problem ◽  
Bandit Problems ◽  
Index Policy ◽  
State Action ◽  
Index Policies ◽  
Availability Constraints ◽  
Whittle Index ◽  
Multiarmed Bandit

A multiarmed bandit problem is studied when the arms are not always available. The arms are first assumed to be intermittently available with some state/action-dependent probabilities. It is proven that no index policy can attain the maximum expected total discounted reward in every instance of that problem. The Whittle index policy is derived, and its properties are studied. Then it is assumed that the arms may break down, but repair is an option at some cost, and the new Whittle index policy is derived. Both problems are indexable. The proposed index policies cannot be dominated by any other index policy over all multiarmed bandit problems considered here. Whittle indices are evaluated for Bernoulli arms with unknown success probabilities.

Sign in / Sign up

Export Citation Format

Share Document