Model Averaging Techniques for Quantifying Conceptual Model Uncertainty

Vaccine effectiveness (VE) studies are often conducted after the introduction of new vaccines to ensure they provide protection in real-world settings. Although susceptible to confounding, the test-negative case-control study design is the most efficient method to assess VE post-licensure. Control of confounding is often needed during the analyses, which is most efficiently done through multivariable modeling. When a large number of potential confounders are being considered, it can be challenging to know which variables need to be included in the final model. This paper highlights the importance of considering model uncertainty by re-analyzing a Lyme VE study using several confounder selection methods. We propose an intuitive Bayesian Model Averaging (BMA) framework for this task and compare the performance of BMA to that of traditional single-best-model-selection methods. We demonstrate how BMA can be advantageous in situations when there is uncertainty about model selection by systematically considering alternative models and increasing transparency.

Download Full-text

Publication and Identification Biases in Measuring the Intertemporal Substitution of Labor Supply

10.31222/osf.io/nshqx ◽

2021 ◽

Author(s):

Tomas Havranek ◽

Roman Horvath ◽

Ali Elminejad

Keyword(s):

Labor Supply ◽

Publication Bias ◽

Model Uncertainty ◽

Model Averaging ◽

Extensive Margin ◽

Real Business Cycle ◽

Intertemporal Substitution ◽

The Mean ◽

Frisch Elasticity ◽

Frequentist Model Averaging

The intertemporal substitution (Frisch) elasticity of labor supply governs the predictions of real business cycle models and models of taxation. We show that, for the extensive margin elasticity, two biases conspire to systematically produce large positive estimates when the elasticity is in fact zero. Among 723 estimates in 36 studies, the mean reported elasticity is 0.5. One half of that number is due to publication bias: larger estimates are reported preferentially. The other half is due to identification bias: studies with less exogenous time variation in wages report larger elasticities. Net of the biases, the literature implies a zero mean elasticity and, with 95% confidence, is inconsistent with calibrations above 0.25. To derive these results we collect 23 variables that reflect the context in which the elasticity was obtained, use nonlinear techniques to correct for publication bias, and employ Bayesian and frequentist model averaging to address model uncertainty.

Download Full-text

Using Bayesian model averaging to improve ground motion predictions

Geophysical Journal International ◽

10.1093/gji/ggz486 ◽

2019 ◽

Vol 220 (2) ◽

pp. 1368-1378

Author(s):

M Bertin ◽

S Marin ◽

C Millet ◽

C Berge-Thierry

Keyword(s):

Ground Motion ◽

Model Uncertainty ◽

Bayesian Model ◽

Bayesian Model Averaging ◽

Selection Process ◽

Hazard Analysis ◽

Model Averaging ◽

Likelihood Estimation ◽

Predictive Performance ◽

Out Of Sample

SUMMARY In low-seismicity areas such as Europe, seismic records do not cover the whole range of variable configurations required for seismic hazard analysis. Usually, a set of empirical models established in such context (the Mediterranean Basin, northeast U.S.A., Japan, etc.) is considered through a logic-tree-based selection process. This approach is mainly based on the scientist’s expertise and ignores the uncertainty in model selection. One important and potential consequence of neglecting model uncertainty is that we assign more precision to our inference than what is warranted by the data, and this leads to overly confident decisions and precision. In this paper, we investigate the Bayesian model averaging (BMA) approach, using nine ground-motion prediction equations (GMPEs) issued from several databases. The BMA method has become an important tool to deal with model uncertainty, especially in empirical settings with large number of potential models and relatively limited number of observations. Two numerical techniques, based on the Markov chain Monte Carlo method and the maximum likelihood estimation approach, for implementing BMA are presented and applied together with around 1000 records issued from the RESORCE-2013 database. In the example considered, it is shown that BMA provides both a hierarchy of GMPEs and an improved out-of-sample predictive performance.

Download Full-text

Model Averaging and Its Use in Economics

Journal of Economic Literature ◽

10.1257/jel.20191385 ◽

2020 ◽

Vol 58 (3) ◽

pp. 644-719 ◽

Cited By ~ 3

Author(s):

Mark F. J. Steel

Keyword(s):

Model Uncertainty ◽

Bayesian Model Averaging ◽

Model Averaging ◽

Linear Regression Models ◽

Bayesian Procedures ◽

Wide Range ◽

Computational Resources ◽

Production Modeling ◽

Frequentist Model Averaging

The method of model averaging has become an important tool to deal with model uncertainty, for example in situations where a large amount of different theories exist, as are common in economics. Model averaging is a natural and formal response to model uncertainty in a Bayesian framework, and most of the paper deals with Bayesian model averaging. The important role of the prior assumptions in these Bayesian procedures is highlighted. In addition, frequentist model averaging methods are also discussed. Numerical techniques to implement these methods are explained, and I point the reader to some freely available computational resources. The main focus is on uncertainty regarding the choice of covariates in normal linear regression models, but the paper also covers other, more challenging, settings, with particular emphasis on sampling models commonly used in economics. Applications of model averaging in economics are reviewed and discussed in a wide range of areas including growth economics, production modeling, finance and forecasting macroeconomic quantities. (JEL C11, C15, C20, C52, O47).

Download Full-text

Can Cattle Basis Forecasts Be Improved? A Bayesian Model Averaging Approach

Journal of Agricultural and Applied Economics ◽

10.1017/aae.2018.35 ◽

2019 ◽

Vol 51 (02) ◽

pp. 249-266

Author(s):

Nicholas D. Payne ◽

Berna Karali ◽

Jeffrey H. Dorfman

Keyword(s):

Bayesian Approach ◽

Model Uncertainty ◽

Bayesian Model Averaging ◽

Model Performance ◽

Model Averaging ◽

Forecasting Model ◽

Forecast Errors ◽

Forecast Horizon ◽

Combining Forecasts ◽

The Bayesian Approach

AbstractBasis forecasting is important for producers and consumers of agricultural commodities in their risk management decisions. However, the best performing forecasting model found in previous studies varies substantially. Given this inconsistency, we take a Bayesian approach, which addresses model uncertainty by combining forecasts from different models. Results show model performance differs by location and forecast horizon, but the forecast from the Bayesian approach often performs favorably. In some cases, however, the simple moving averages have lower forecast errors. Besides the nearby basis, we also examine basis in a specific month and find that regression-based models outperform others in longer horizons.

Download Full-text