Dependency of regret on accuracy of variance estimation for different versions of UCB strategy for Gaussian multi-armed bandits

Abstract We consider two variations of upper confidence bound strategy for Gaussian two-armed bandits. Rewards for the arms are assumed to have unknown expected values and unknown variances. It is demonstrated that expected regret values for both discussed strategies are continuous functions of reward variance. A set of Monte-Carlo simulations was performed to show the nature of the relation between variance estimation and losses. It is shown that the regret grows only slightly when the estimation error is fairly large, which allows to estimate the variance during the initial steps of the control and stop this estimation later.

Download Full-text

Numerical preservation issues in stochastic dynamical systems by $ \vartheta $-methods

Journal of Computational Dynamics ◽

10.3934/jcd.2021023 ◽

2021 ◽

Vol 0 (0) ◽

pp. 0

Author(s):

Raffaele D'Ambrosio ◽

Stefano Di Giovacchino

Keyword(s):

Monte Carlo ◽

Dynamical Systems ◽

Theoretical Analysis ◽

Monte Carlo Simulations ◽

Duffing Oscillator ◽

Stochastic Dynamical Systems ◽

Numerical Evidence ◽

Stochastic Oscillator ◽

Expected Values

<p style='text-indent:20px;'>This paper analyzes conservation issues in the discretization of certain stochastic dynamical systems by means of stochastic <inline-formula><tex-math id="M2">\begin{document}$ \vartheta $\end{document}</tex-math></inline-formula>-mehods. The analysis also takes into account the effects of the estimation of the expected values by means of Monte Carlo simulations. The theoretical analysis is supported by a numerical evidence on a given stochastic oscillator, inspired by the Duffing oscillator.</p>

Download Full-text

Comparison of straight line curve fit approaches for determining parameter variances and covariances

International Journal of Metrology and Quality Engineering ◽

10.1051/ijmqe/2020011 ◽

2020 ◽

Vol 11 ◽

pp. 14

Author(s):

Vishal Ramnath

Keyword(s):

Monte Carlo ◽

Monte Carlo Simulations ◽

Least Squares ◽

Weighted Least Squares ◽

Applied Pressure ◽

Ordinary Least Squares ◽

Pressure Balance ◽

Straight Line ◽

Covariance Term ◽

Expected Values

Pressure balances are known to have a linear straight line equation of the form y = ax + b that relates the applied pressure x to the effective area y, and recent work has investigated the use of Ordinary Least Squares (OLS), Weighted Least Squares (WLS), and Generalized Least Squares (GLS) regression schemes in order to quantify the expected values of the zero-pressure area A0 = b and distortion coefficient λ = a/b in pressure balance models of the form y = A0(1 + λx). The limitations with conventional OLS, WLS and GLS approaches is that whilst they may be used to quantify the uncertainties u(a) and u(b) and the covariance cov(a, b), it is technically challenging to analytically quantify the covariance term cov(A0, λ) without additional Monte Carlo simulations. In this paper, we revisit an earlier Weighted Total Least Squares with Correlation (WTLSC) algorithm to determine the variances u2(a) and u2(b) along with the covariance cov(a, b), and develop a simple analytical approach to directly infer the corresponding covariance cov(A0, λ) for pressure metrology uncertainty analysis work. Results are compared to OLS, WLS and GLS approaches and indicate that the WTLSC approach may be preferable as it avoids the need for Monte Carlo simulations and additional numerical post-processing to fit and quantify the covariance term, and is thus simpler and more suitable for industrial metrology pressure calibration laboratories. Novel aspects is that a Gnu Octave/Matlab program for easily implementing the WTLSC algorithm to calculate parameter expected values, variances and covariances is also supplied and reported.

Download Full-text

Low-voltage EDS of magnesium ferrite Dendrites in a FEG-SEM

Proceedings, annual meeting, Electron Microscopy Society of America ◽

10.1017/s0424820100164854 ◽

1996 ◽

Vol 54 ◽

pp. 478-479

Author(s):

Matthew T. Johnson ◽

Ian M. Anderson ◽

Jim Bentley ◽

C. Barry Carter

Keyword(s):

Monte Carlo ◽

Quantitative Analysis ◽

Monte Carlo Simulations ◽

Cross Section ◽

Spatial Resolution ◽

Low Voltage ◽

Magnesium Ferrite ◽

X Ray ◽

Interaction Volume ◽

Ferrite Spinel

Energy-dispersive X-ray spectrometry (EDS) performed at low (≤ 5 kV) accelerating voltages in the SEM has the potential for providing quantitative microanalytical information with a spatial resolution of ∼100 nm. In the present work, EDS analyses were performed on magnesium ferrite spinel [(MgxFe1−x)Fe2O4] dendrites embedded in a MgO matrix, as shown in Fig. 1. spatial resolution of X-ray microanalysis at conventional accelerating voltages is insufficient for the quantitative analysis of these dendrites, which have widths of the order of a few hundred nanometers, without deconvolution of contributions from the MgO matrix. However, Monte Carlo simulations indicate that the interaction volume for MgFe2O4 is ∼150 nm at 3 kV accelerating voltage and therefore sufficient to analyze the dendrites without matrix contributions.Single-crystal {001}-oriented MgO was reacted with hematite (Fe2O3) powder for 6 h at 1450°C in air and furnace cooled. The specimen was then cleaved to expose a clean cross-section suitable for microanalysis.

Download Full-text