scholarly journals Bernoulli multi-armed bandit problem under delayed feedback

Author(s):  
A. S. Dzhoha

Online learning under delayed feedback has been recently gaining increasing attention. Learning with delays is more natural in most practical applications since the feedback from the environment is not immediate. For example, the response to a drug in clinical trials could take a while. In this paper, we study the multi-armed bandit problem with Bernoulli distribution in the environment with delays by evaluating the Explore-First algorithm. We obtain the upper bounds of the algorithm, the theoretical results are applied to develop the software framework for conducting numerical experiments.

Mathematics ◽  
2021 ◽  
Vol 9 (9) ◽  
pp. 994
Author(s):  
Elisa Alòs ◽  
Jorge A. León

Here, we review some results of fractional volatility models, where the volatility is driven by fractional Brownian motion (fBm). In these models, the future average volatility is not a process adapted to the underlying filtration, and fBm is not a semimartingale in general. So, we cannot use the classical Itô’s calculus to explain how the memory properties of fBm allow us to describe some empirical findings of the implied volatility surface through Hull and White type formulas. Thus, Malliavin calculus provides a natural approach to deal with the implied volatility without assuming any particular structure of the volatility. The aim of this paper is to provides the basic tools of Malliavin calculus for the study of fractional volatility models. That is, we explain how the long and short memory of fBm improves the description of the implied volatility. In particular, we consider in detail a model that combines the long and short memory properties of fBm as an example of the approach introduced in this paper. The theoretical results are tested with numerical experiments.


2022 ◽  
Vol 40 ◽  
pp. 1-11
Author(s):  
Parviz Darania ◽  
Saeed Pishbin

In this note, we study a class of multistep collocation methods for the numerical integration of nonlinear Volterra-Fredholm Integral Equations (V-FIEs). The derived method is characterized by a lower triangular or diagonal coefficient matrix of the nonlinear system for the computation of the stages which, as it is known, can beexploited to get an efficient implementation. Convergence analysis and linear stability estimates are investigated. Finally numerical experiments are given, which confirm our theoretical results.


2011 ◽  
Vol 31 (6) ◽  
pp. 785-786 ◽  
Author(s):  
David O. Meltzer ◽  
Ties Hoomans ◽  
Jeannette W. Chung ◽  
Anirban Basu

Value of information (VOI) techniques can provide estimates of the expected benefits from clinical research studies that can inform decisions about the design and priority of those studies. Most VOI studies use decision-analytic models to characterize the uncertainty of the effects of interventions on health outcomes, but the complexity of constructing such models can pose barriers to some practical applications of VOI. However, because some clinical studies can directly characterize uncertainty in health outcomes, it may sometimes be possible to perform VOI analysis with only minimal modeling. This article 1) develops a framework to define and classify minimal modeling approaches to VOI, 2) reviews existing VOI studies that apply minimal modeling approaches, and 3) illustrates and discusses the application of the minimal modeling to two new clinical applications to which the approach appears well suited because clinical trials with comprehensive outcomes provide preliminary estimates of the uncertainty in outcomes. We conclude that minimal modeling approaches to VOI can be readily applied to in some instances to estimate the expected benefits of clinical research.


2017 ◽  
Vol 7 (4) ◽  
pp. 827-836
Author(s):  
Ze-Jia Xie ◽  
Xiao-Qing Jin ◽  
Zhi Zhao

AbstractSome convergence bounds of the minimal residual (MINRES) method are studied when the method is applied for solving Hermitian indefinite linear systems. The matrices of these linear systems are supposed to have some properties so that their spectra are all clustered around ±1. New convergence bounds depending on the spectrum of the coefficient matrix are presented. Some numerical experiments are shown to demonstrate our theoretical results.


Mathematics ◽  
2019 ◽  
Vol 7 (8) ◽  
pp. 708 ◽  
Author(s):  
Suthep Suantai ◽  
Suparat Kesornprom ◽  
Prasit Cholamjiak

We investigate the split variational inclusion problem in Hilbert spaces. We propose efficient algorithms in which, in each iteration, the stepsize is chosen self-adaptive, and proves weak and strong convergence theorems. We provide numerical experiments to validate the theoretical results for solving the split variational inclusion problem as well as the comparison to algorithms defined by Byrne et al. and Chuang, respectively. It is shown that the proposed algorithms outrun other algorithms via numerical experiments. As applications, we apply our method to compressed sensing in signal recovery. The proposed methods have as a main advantage that the computation of the Lipschitz constants for the gradient of functions is dropped in generating the sequences.


Entropy ◽  
2020 ◽  
Vol 22 (7) ◽  
pp. 788
Author(s):  
Lan V. Truong ◽  
Jonathan Scarlett

In this paper, we consider techniques for establishing lower bounds on the number of arm pulls for best-arm identification in the multi-armed bandit problem. While a recent divergence-based approach was shown to provide improvements over an older gap-based approach, we show that the latter can be refined to match the former (up to constant factors) in many cases of interest under Bernoulli rewards, including the case that the rewards are bounded away from zero and one. Together with existing upper bounds, this indicates that the divergence-based and gap-based approaches are both effective for establishing sample complexity lower bounds for best-arm identification.


Complexity ◽  
2020 ◽  
Vol 2020 ◽  
pp. 1-15
Author(s):  
Yue Zhang ◽  
Yuxuan Li

In this paper, a stochastic SEIR (Susceptible-Exposed-Infected-Removed) epidemic dynamic model with migration and human awareness in complex networks is constructed. The awareness is described by an exponential function. The existence of global positive solutions for the stochastic system in complex networks is obtained. The sufficient conditions are presented for the extinction and persistence of the disease. Under the conditions of disease persistence, the distance between the stochastic solution and the local disease equilibrium of the corresponding deterministic system is estimated in the time sense. Some numerical experiments are also presented to illustrate the theoretical results. Although the awareness introduced in the model cannot affect the extinction of the disease, the scale of the disease will eventually decrease as human awareness increases.


2015 ◽  
Vol 4 (1) ◽  
pp. 57
Author(s):  
Pramod M. Lad ◽  
Rebecca Dahl

The Data and Safety Monitoring Board (DSMB) monitors the progress of clinical trials for safety and implements stopping rules as needed. Although NIH and FDA guidelines recommend the use of a DSMB for phase I, II, and III pediatric clinical trials, the manner in which the DSMB is constituted has received little attention. In this study we reviewed the Institutional Review Board (IRB) applications submitted between 2008 and 2012 at our institution (Children’s Hospital Los Angeles) for phase I, II and III studies which were multi-site, sponsored and performed under a sponsor’s Investigation New Drug Application (IND) for the type of data and safety monitoring that was being used. Our results indicate that approximately two-third of the studies used an independent DSMB, 10% utilized a sponsor’s DSMB and the remaining studies (25%) did not utilize a DSMB and relied instead on safety monitoring by the Principal Investigator (PI) and the medical monitor/director. This pattern was observed across all study phases and for blinded as well as unblinded studies. Our result suggests that a Data and Safety Monitoring Plan (DSMP), although required by the IRB, is rarely submitted by the sponsor at the time of application. Instead the DSMP is submitted to the IRB by the PI on IRB supplied templates. IRB review of these completed templates were critical to ensuring DSMB related compliance. Additionally, a significant percent of sponsored clinical trials used the PI or an individual designated as medical director/monitor, rather than constituting a DSMB.


2012 ◽  
Vol 2012 ◽  
pp. 1-21 ◽  
Author(s):  
Chengjian Zhang

This paper presents a class of new numerical methods for nonlinear functional-integrodifferential equations, which are derived by an adaptation of Pouzet-Runge-Kutta methods originally introduced for standard Volterra integrodifferential equations. Based on the nonclassical Lipschitz condition, analytical and numerical stability is studied and some novel stability criteria are obtained. Numerical experiments further illustrate the theoretical results and the effectiveness of the methods. In the end, a comparison between the presented methods and the existed related methods is given.


2016 ◽  
Vol 2016 ◽  
pp. 1-7 ◽  
Author(s):  
Ricardo Aguilar-López ◽  
Juan L. Mata-Machuca

This paper proposes a synchronization methodology of two chaotic oscillators under the framework of identical synchronization and master-slave configuration. The proposed methodology is based on state observer design under the frame of control theory; the observer structure provides finite-time synchronization convergence by cancelling the upper bounds of the main nonlinearities of the chaotic oscillator. The above is showed via an analysis of the dynamic of the so called synchronization error. Numerical experiments corroborate the satisfactory results of the proposed scheme.


Sign in / Sign up

Export Citation Format

Share Document