Configuration of Bayesian Model Averaging Training Window to Improve Seasonal Rainfall Ensemble Prediction

Abstract Merging forecasts from multiple models has the potential to combine the strengths of individual models and to better represent forecast uncertainty than the use of a single model. This study develops a Bayesian model averaging (BMA) method for merging forecasts from multiple models, giving greater weights to better performing models. The study aims for a BMA method that is capable of producing relatively stable weights in the presence of significant sampling variability, leading to robust forecasts for future events. The BMA method is applied to merge forecasts from multiple statistical models for seasonal rainfall forecasts over Australia using climate indices as predictors. It is shown that the fully merged forecasts effectively combine the best skills of the models to maximize the spatial coverage of positive skill. Overall, the skill is low for the first half of the year but more positive for the second half of the year. Models in the Pacific group contribute the most skill, and models in the Indian and extratropical groups also produce useful and sometimes distinct skills. The fully merged probabilistic forecasts are found to be reliable in representing forecast uncertainty spread. The forecast skill holds well when forecast lead time is increased from 0 to 1 month. The BMA method outperforms the approach of using a model with two fixed predictors chosen a priori and the approach of selecting the best model based on predictive performance.

Download Full-text

Multi-model ensemble prediction of terrestrial evapotranspiration across north China using Bayesian model averaging

Hydrological Processes ◽

10.1002/hyp.10832 ◽

2016 ◽

Vol 30 (16) ◽

pp. 2861-2879 ◽

Cited By ~ 20

Author(s):

Gaofeng Zhu ◽

Xin Li ◽

Kun Zhang ◽

Zhenyu Ding ◽

Tuo Han ◽

...

Keyword(s):

Bayesian Model ◽

North China ◽

Bayesian Model Averaging ◽

Model Averaging ◽

Ensemble Prediction ◽

Model Ensemble

Download Full-text

A Local Ensemble Prediction System for Fog and Low Clouds: Construction, Bayesian Model Averaging Calibration, and Validation

Journal of Applied Meteorology and Climatology ◽

10.1175/2008jamc1783.1 ◽

2008 ◽

Vol 47 (12) ◽

pp. 3072-3088 ◽

Cited By ~ 21

Author(s):

Stevie Roquelaure ◽

Thierry Bergot

Keyword(s):

Bayesian Model ◽

Bayesian Model Averaging ◽

Model Averaging ◽

Ensemble Prediction ◽

Prediction System ◽

Ensemble Prediction System ◽

Low Clouds ◽

Wide Range ◽

Ensemble Strategy ◽

Low Visibility

Abstract At main international airports, air traffic safety and economic issues related to poor visibility conditions are crucial. Meteorologists face the challenge of supplying airport authorities with accurate forecasts of fog and cloud ceiling. These events are difficult to forecast because conditions evolve on short space and time scales during their life cycle. To obtain accurate forecasts of fog and low clouds, the Code de Brouillard à l’Echelle Locale (the local scale fog code)–Interactions between Soil, Biosphere, and Atmosphere (COBEL–ISBA) local numerical forecast system was implemented at Charles de Gaulle International Airport in Paris. However, even with dedicated observations and initialization, uncertainties remain in both initial conditions and mesoscale forcings. A local ensemble prediction system (LEPS) has been designed around the COBEL–ISBA numerical model and tested to assess the predictability of low visibility procedures events, defined as a visibility less than 600 m and/or a ceiling below 60 m. This work describes and evaluates a local ensemble strategy for the prediction of low visibility procedures. A Bayesian model averaging method has been applied to calibrate the ensemble. The study shows that the use of LEPS for specific local event prediction is well adapted and useful for low visibility prediction in the aeronautic context. Moreover, a wide range of users, especially those with low cost–loss ratios, can expect economic savings with the use of this probabilistic system.

Download Full-text

A Comparison between Raw Ensemble Output, (Modified) Bayesian Model Averaging, and Extended Logistic Regression Using ECMWF Ensemble Precipitation Reforecasts

Monthly Weather Review ◽

10.1175/2010mwr3285.1 ◽

2010 ◽

Vol 138 (11) ◽

pp. 4199-4211 ◽

Cited By ~ 57

Author(s):

Maurice J. Schmeits ◽

Kees J. Kok

Keyword(s):

Logistic Regression ◽

Bias Correction ◽

Bayesian Model ◽

Bayesian Model Averaging ◽

Model Averaging ◽

Ensemble Prediction ◽

Ensemble Forecasts ◽

Ensemble Prediction System ◽

The Difference ◽

Model Output Statistics

Abstract Using a 20-yr ECMWF ensemble reforecast dataset of total precipitation and a 20-yr dataset of a dense precipitation observation network in the Netherlands, a comparison is made between the raw ensemble output, Bayesian model averaging (BMA), and extended logistic regression (LR). A previous study indicated that BMA and conventional LR are successful in calibrating multimodel ensemble forecasts of precipitation for a single forecast projection. However, a more elaborate comparison between these methods has not yet been made. This study compares the raw ensemble output, BMA, and extended LR for single-model ensemble reforecasts of precipitation; namely, from the ECMWF ensemble prediction system (EPS). The raw EPS output turns out to be generally well calibrated up to 6 forecast days, if compared to the area-mean 24-h precipitation sum. Surprisingly, BMA is less skillful than the raw EPS output from forecast day 3 onward. This is due to the bias correction in BMA, which applies model output statistics to individual ensemble members. As a result, the spread of the bias-corrected ensemble members is decreased, especially for the longer forecast projections. Here, an additive bias correction is applied instead and the equation for the probability of precipitation in BMA is also changed. These modifications to BMA are referred to as “modified BMA” and lead to a significant improvement in the skill of BMA for the longer projections. If the area-maximum 24-h precipitation sum is used as a predictand, both modified BMA and extended LR improve the raw EPS output significantly for the first 5 forecast days. However, the difference in skill between modified BMA and extended LR does not seem to be statistically significant. Yet, extended LR might be preferred, because incorporating predictors that are different from the predictand is straightforward, in contrast to BMA.

Download Full-text

Calibrated Surface Temperature Forecasts from the Canadian Ensemble Prediction System Using Bayesian Model Averaging

Monthly Weather Review ◽

10.1175/mwr3347.1 ◽

2007 ◽

Vol 135 (4) ◽

pp. 1364-1385 ◽

Cited By ~ 76

Author(s):

Laurence J. Wilson ◽

Stephane Beauregard ◽

Adrian E. Raftery ◽

Richard Verret

Keyword(s):

Surface Temperature ◽

Bias Correction ◽

Bayesian Model ◽

Bayesian Model Averaging ◽

Model Averaging ◽

Training Sample ◽

Ensemble Prediction ◽

Ensemble Forecasts ◽

Modest Improvement ◽

Component Models

Abstract Bayesian model averaging (BMA) has recently been proposed as a way of correcting underdispersion in ensemble forecasts. BMA is a standard statistical procedure for combining predictive distributions from different sources. The output of BMA is a probability density function (pdf), which is a weighted average of pdfs centered on the bias-corrected forecasts. The BMA weights reflect the relative contributions of the component models to the predictive skill over a training sample. The variance of the BMA pdf is made up of two components, the between-model variance, and the within-model error variance, both estimated from the training sample. This paper describes the results of experiments with BMA to calibrate surface temperature forecasts from the 16-member Canadian ensemble system. Using one year of ensemble forecasts, BMA was applied for different training periods ranging from 25 to 80 days. The method was trained on the most recent forecast period, then applied to the next day’s forecasts as an independent sample. This process was repeated through the year, and forecast quality was evaluated using rank histograms, the continuous rank probability score, and the continuous rank probability skill score. An examination of the BMA weights provided a useful comparative evaluation of the component models, both for the ensemble itself and for the ensemble augmented with the unperturbed control forecast and the higher-resolution deterministic forecast. Training periods around 40 days provided a good calibration of the ensemble dispersion. Both full regression and simple bias-correction methods worked well to correct the bias, except that the full regression failed to completely remove seasonal trend biases in spring and fall. Simple correction of the bias was sufficient to produce positive forecast skill out to 10 days with respect to climatology, which was improved by the BMA. The addition of the control forecast and the full-resolution model forecast to the ensemble produced modest improvement in the forecasts for ranges out to about 7 days. Finally, BMA produced significantly narrower 90% prediction intervals compared to a simple Gaussian bias correction, while achieving similar overall accuracy.

Download Full-text

Calibration Of Rainfall Ensemble Prediction Of ECMWF System 4 Using Bayesian Model Averaging

Agromet ◽

10.29244/j.agromet.34.1.20-33 ◽

2020 ◽

Vol 34 (1) ◽

pp. 20-33

Author(s):

Robi Muharsyah ◽

Tri Wahyu Hadi ◽

Sapto Wahyu Indratno

Keyword(s):

Bayesian Model ◽

Bayesian Model Averaging ◽

Model Averaging ◽

Skill Score ◽

Seasonal Forecast ◽

Probabilistic Forecast ◽

Ensemble Prediction ◽

Operating Characteristics ◽

Ensemble Prediction System ◽

Medium Range

Bayesian model averaging (BMA) is a statistical post-processing method for producing probabilistic forecasts from ensembles in the form of predictive PDFs. It is known that BMA is able to improve the reliability of probabilistic forecast of short range and medium range rainfall forecast. This study aims to develop the application of BMA to calibrate seasonal forecast (long range) in order to improved quality of seasonal forecast in Indonesia. The seasonal forecast used is monthly rainfall from the output of the ensemble prediction system European Center for Medium-Range Weather Forecasts (ECMWF) system 4 model (ECS4) and it is calibrated against observational data at 26 stations of the Agency for Meteorology Climatology and Geophysics of Republic of Indonesia (BMKG) in Java Island in 1981 – 2018. BMA predictive PDFs is generated with Gamma distribution approach which is obtained based on sequential training windows (JTS) and conditionals training windows (JTC). BMA-JTS approach is done by selecting the width of the 30-month training window as the optimal training period while the BMA-JTC is carried out with a cross-validation scheme for each month. In general, Both of BMA-JTS and BMA-JTC better than RAW models. BMA-JTC calibration results are varying according to spatial and temporal, but in general the result is better in the dry season and during the El Nino phase. BMA is able to improve the distribution characteristics of the RAW model ECS4 prediction which is shown by: a smaller value of Continuous Rank Probability Score (CRPS), a larger value of the Continuous Rank Probability Skill Score (CRPSS) and more flat form of the Verification Rank Histogram (VRH) than the RAW model. BMA also increases the skill, esolution and reliability of prediction of probability Below Normal (BN) and Above Normal (AN), which is known from the increasing Brier Skill Score (BSS), and the increasing area under curve of Relative Operating Characteristics (ROC) compared to the RAW model. Furthermore, the reliability of BN and AN of BMA results also has the category of “still very useful” and “perfect” compared to RAW models that are in the “dangerous”, “not useful” and “marginally useful” categories. The reliability of BMA results with the category “still very useful” and “perfect” show that the probabilistic forecast of BN and AN events can be used in making decisions related to seasonal forecast.

Download Full-text

Probabilistic Precipitation Forecasting over East Asia Using Bayesian Model Averaging

Weather and Forecasting ◽

10.1175/waf-d-18-0093.1 ◽

2019 ◽

Vol 34 (2) ◽

pp. 377-392 ◽

Cited By ~ 7

Author(s):

Luying Ji ◽

Xiefei Zhi ◽

Shoupeng Zhu ◽

Klaus Fraedrich

Keyword(s):

East Asia ◽

Bayesian Model ◽

Bayesian Model Averaging ◽

Model Averaging ◽

Heavy Precipitation ◽

Prediction Skill ◽

Ensemble Prediction ◽

Limited Ability ◽

Study Results ◽

Precipitation Events

Abstract Bayesian model averaging (BMA) was applied to improve the prediction skill of 1–15-day, 24-h accumulated precipitation over East Asia based on the ensemble prediction system (EPS) outputs of ECMWF, NCEP, and UKMO from the TIGGE datasets. Standard BMA deterministic forecasts were accurate for light-precipitation events but with limited ability for moderate- and heavy-precipitation events. The categorized BMA model based on precipitation categories was proposed to improve the BMA capacity for moderate and heavy precipitation in this study. Results showed that the categorized BMA deterministic forecasts were superior to the standard one, especially for moderate and heavy precipitation. The categorized BMA also provided a better calibrated probability of precipitation and a sharper prediction probability density function than the standard one and the raw ensembles. Moreover, BMA forecasts based on multimodel EPSs outperformed those based on a single-model EPS for all lead times. Comparisons between the two BMA models, logistic regression, and raw ensemble forecasts for probabilistic precipitation forecasts illustrated that the categorized BMA method performed best. For 10–15-day extended-range probabilistic forecasts, the initial BMA performances were inferior to the climatology forecasts, while they became much better after preprocessing the initial data with the running mean method. With increasing running steps, the BMA model generally had better performance for light to moderate precipitation but had limited ability for heavy precipitation. In general, the categorized BMA methodology combined with the running mean method improved the prediction skill of 1–15-day, 24-h accumulated precipitation over East Asia.

Download Full-text