scholarly journals “A Bias Recognized Is a Bias Sterilized”: The Effects of a Bias in Forecast Evaluation

Mathematics ◽  
2022 ◽  
Vol 10 (2) ◽  
pp. 171
Author(s):  
Nicolas Hardy
Keyword(s):  
Asymptotic Distribution ◽  
Null Model ◽  
Critical Values ◽  
Forecast Evaluation ◽  
Nuisance Parameters ◽  
Nested Model ◽  
Nominal Size ◽  
Out Of Sample ◽  
Size Distortions ◽  
Encompassing Test

Are traditional tests of forecast evaluation well behaved when the competing (nested) model is biased? No, they are not. In this paper, we show analytically and via simulations that, under the null hypothesis of no encompassing, a bias in the nested model may severely distort the size properties of traditional out-of-sample tests in economic forecasting. Not surprisingly, these size distortions depend on the magnitude of the bias and the persistency of the additional predictors. We consider two different cases: (i) There is both in-sample and out-of-sample bias in the nested model. (ii) The bias is present exclusively out-of-sample. To address the former case, we propose a modified encompassing test (MENC-NEW) robust to a bias in the null model. Akin to the ENC-NEW statistic, the asymptotic distribution of our test is a functional of stochastic integrals of quadratic Brownian motions. While this distribution is not pivotal, we can easily estimate the nuisance parameters. To address the second case, we derive the new asymptotic distribution of the ENC-NEW, showing that critical values may differ remarkably. Our Monte Carlo simulations reveal that the MENC-NEW (and the ENC-NEW with adjusted critical values) is reasonably well-sized even when the ENC-NEW (with standard critical values) exhibits rejections rates three times higher than the nominal size.

Asymptotic distribution of critical values

2009 ◽  
Vol 143 (1) ◽  
pp. 63-67 ◽  
Author(s):  
Tomohiro Fukaya ◽  
Masaki Tsukamoto
Keyword(s):  
Asymptotic Distribution ◽  
Critical Values

scholarly journals On the Correction of the Asymptotic Distribution of the Likelihood Ratio Statistic If Nuisance Parameters Are Estimated Based on an External Source

2014 ◽  
Vol 10 (2) ◽  
Author(s):  
Marianne Jonker ◽  
Aad Van der Vaart
Keyword(s):  
Confidence Interval ◽  
Asymptotic Distribution ◽  
Confidence Intervals ◽  
Likelihood Ratio ◽  
Statistical Models ◽  
Likelihood Ratio Statistic ◽  
External Source ◽  
Nuisance Parameters ◽  
Testing Hypotheses

AbstractIn practice, nuisance parameters in statistical models are often replaced by estimates based on an external source, for instance if estimates were published before or a second dataset is available. Next these estimates are assumed to be known when the parameter of interest is estimated, a hypothesis is tested or confidence intervals are constructed. By this assumption, the level of the test is, in general, higher than supposed and the coverage of the confidence interval is too low. In this article, we derive the asymptotic distribution of the likelihood ratio statistic if the nuisance parameters are estimated based on a dataset that is independent of the data used for estimating the parameter of interest. This distribution can be used for correctly testing hypotheses and constructing confidence intervals. Four theoretical and practical examples are given as illustration.

scholarly journals Constructing confidence intervals for QTL location.

Genetics ◽  
1994 ◽  
Vol 138 (4) ◽  
pp. 1301-1308 ◽  
Author(s):  
B Mangin ◽  
B Goffinet ◽  
A Rebaï
Keyword(s):  
Confidence Interval ◽  
Linear Model ◽  
Asymptotic Distribution ◽  
Confidence Intervals ◽  
Likelihood Ratio Test ◽  
Location Parameter ◽  
Nuisance Parameters ◽  
Ratio Test ◽  
Almost All ◽  
Map Location

Abstract We describe a method for constructing the confidence interval of the QTL location parameter. This method is developed in the local asymptotic framework, leading to a linear model at each position of the putative QTL. The idea is to construct a likelihood ratio test, using statistics whose asymptotic distribution does not depend on the nuisance parameters and in particular on the effect of the QTL. We show theoretical properties of the confidence interval built with this test, and compare it with the classical confidence interval using simulations. We show in particular, that our confidence interval has the correct probability of containing the true map location of the QTL, for almost all QTLs, whereas the classical confidence interval can be very biased for QTLs having small effect.

scholarly journals Inference in Time Series Regression When the Order of Integration of a Regressor is Unknown

1994 ◽  
Vol 10 (3-4) ◽  
pp. 672-700 ◽  
Author(s):  
Graham Elliott ◽  
James H. Stock
Keyword(s):  
Time Series ◽  
Unit Root ◽  
Critical Values ◽  
Asymptotic Power ◽  
Time Series Regression ◽  
Second Stage ◽  
Alternative Approach ◽  
Local Asymptotic Power ◽  
Size Distortions ◽  
Correct Size

The distribution of statistics testing restrictions on the coefficients in time series regressions can depend on the order of integration of the regressors. In practice, the order of integration is rarely known. We examine two conventional approaches to this problem — simply to ignore unit root problems or to use unit root pretests to determine the critical values for second-stage inference—and show that both exhibit substantial size distortions in empirically plausible situations. We then propose an alternative approach in which the second-stage critical values depend continuously on a first-stage statistic that is informative about the order of integration of the regressor. This procedure has the correct size asymptotically and good local asymptotic power.

scholarly journals Forecast evaluation of small nested model sets

2010 ◽  
Vol 25 (4) ◽  
pp. 574-594 ◽  
Author(s):  
Kirstin Hubrich ◽  
Kenneth D. West
Keyword(s):  
Forecast Evaluation ◽  
Nested Model ◽  
Model Sets

scholarly journals SPECIFICATION TESTS FOR LATTICE PROCESSES

2014 ◽  
Vol 31 (2) ◽  
pp. 294-336 ◽  
Author(s):  
Javier Hidalgo ◽  
Myung Hwan Seo
Keyword(s):  
Regression Model ◽  
Asymptotic Distribution ◽  
Nuisance Parameters ◽  
Monte Carlo Experiment ◽  
Finite Sample ◽  
Omnibus Test ◽  
Unknown Parameters ◽  
Specification Tests ◽  
Image Position ◽  
Lattice Processes

We consider an omnibus test for the correct specification of the dynamics of a sequence $\left\{ {x\left( t \right)} \right\}_{t \in Z^d } $ in a lattice. As it happens with causal models and d = 1, its asymptotic distribution is not pivotal and depends on the estimator of the unknown parameters of the model under the null hypothesis. One first main goal of the paper is to provide a transformation to obtain an asymptotic distribution that is free of nuisance parameters. Secondly, we propose a bootstrap analog of the transformation and show its validity. Thirdly, we discuss the results when $\left\{ {x\left( t \right)} \right\}_{t \in Z^d } $ are the errors of a parametric regression model. As a by product, we also discuss the asymptotic normality of the least squares estimator of the parameters of the regression model under very mild conditions. Finally, we present a small Monte Carlo experiment to shed some light on the finite sample behavior of our test.

Sign in / Sign up

Export Citation Format

Share Document