scholarly journals Designing Multimodel Applications with Surrogate Forecast Systems

2020 ◽  
Vol 148 (6) ◽  
pp. 2233-2249
Author(s):  
Leonard A. Smith ◽  
Hailiang Du ◽  
Sarah Higgins
Keyword(s):  
Bayesian Model Averaging ◽  
Weather Forecasting ◽  
Model Averaging ◽  
Probabilistic Forecasting ◽  
Single Model ◽  
Forecast System ◽  
Mathematical System ◽  
Probabilistic Forecasts ◽  
Perfect Model ◽  
Seasonal Weather

Abstract Probabilistic forecasting is common in a wide variety of fields including geoscience, social science, and finance. It is sometimes the case that one has multiple probability forecasts for the same target. How is the information in these multiple nonlinear forecast systems best “combined”? Assuming stationarity, in the limit of a very large forecast–outcome archive, each model-based probability density function can be weighted to form a “multimodel forecast” that will, in expectation, provide at least as much information as the most informative single model forecast system. If one of the forecast systems yields a probability distribution that reflects the distribution from which the outcome will be drawn, Bayesian model averaging will identify this forecast system as the preferred system in the limit as the number of forecast–outcome pairs goes to infinity. In many applications, like those of seasonal weather forecasting, data are precious; the archive is often limited to fewer than 26 entries. In addition, no perfect model is in hand. It is shown that in this case forming a single “multimodel probabilistic forecast” can be expected to prove misleading. These issues are investigated in the surrogate model (here a forecast system) regime, where using probabilistic forecasts of a simple mathematical system allows many limiting behaviors of forecast systems to be quantified and compared with those under more realistic conditions.

scholarly journals Using Bayesian Model Averaging (BMA) to calibrate probabilistic surface temperature forecasts over Iran

2011 ◽  
Vol 29 (7) ◽  
pp. 1295-1303 ◽  
Author(s):  
I. Soltanzadeh ◽  
M. Azadi ◽  
G. A. Vakili
Keyword(s):  
Boundary Conditions ◽  
Bayesian Model ◽  
Bayesian Model Averaging ◽  
Model Averaging ◽  
Training Sample ◽  
Limited Area ◽  
Probabilistic Forecasts ◽  
Rank Histogram ◽  
Initial And Boundary Conditions ◽  
Environmental Prediction

Abstract. Using Bayesian Model Averaging (BMA), an attempt was made to obtain calibrated probabilistic numerical forecasts of 2-m temperature over Iran. The ensemble employs three limited area models (WRF, MM5 and HRM), with WRF used with five different configurations. Initial and boundary conditions for MM5 and WRF are obtained from the National Centers for Environmental Prediction (NCEP) Global Forecast System (GFS) and for HRM the initial and boundary conditions come from analysis of Global Model Europe (GME) of the German Weather Service. The resulting ensemble of seven members was run for a period of 6 months (from December 2008 to May 2009) over Iran. The 48-h raw ensemble outputs were calibrated using BMA technique for 120 days using a 40 days training sample of forecasts and relative verification data. The calibrated probabilistic forecasts were assessed using rank histogram and attribute diagrams. Results showed that application of BMA improved the reliability of the raw ensemble. Using the weighted ensemble mean forecast as a deterministic forecast it was found that the deterministic-style BMA forecasts performed usually better than the best member's deterministic forecast.

scholarly journals Short-Range Numerical Weather Prediction Using Time-Lagged Ensembles

2007 ◽  
Vol 22 (3) ◽  
pp. 580-595 ◽  
Author(s):  
Chungu Lu ◽  
Huiling Yuan ◽  
Barry E. Schwartz ◽  
Stanley G. Benjamin
Keyword(s):  
Short Range ◽  
Weather Forecasting ◽  
Weather Prediction ◽  
Cycle Model ◽  
Forecast System ◽  
Multilinear Regression ◽  
Probabilistic Forecasts ◽  
Regression Approach ◽  
Ensemble Mean ◽  
Time Lagged

Abstract A time-lagged ensemble forecast system is developed using a set of hourly initialized Rapid Update Cycle model deterministic forecasts. Both the ensemble-mean and probabilistic forecasts from this time-lagged ensemble system present a promising improvement in the very short-range weather forecasting of 1–3 h, which may be useful for aviation weather prediction and nowcasting applications. Two approaches have been studied to combine deterministic forecasts with different initialization cycles as the ensemble members. The first method uses a set of equally weighted time-lagged forecasts and produces a forecast by taking the ensemble mean. The second method adopts a multilinear regression approach to select a set of weights for different time-lagged forecasts. It is shown that although both methods improve short-range forecasts, the unequally weighted method provides the best results for all forecast variables at all levels. The time-lagged ensembles also provide a sample of statistics, which can be used to construct probabilistic forecasts.

scholarly journals Wind power application research on the fusion of the determination and ensemble prediction

2017 ◽  
Vol 14 ◽  
pp. 227-230
Author(s):  
Shi Lan ◽  
Xu Lina ◽  
Hao Yuzhu
Keyword(s):  
Wind Speed ◽  
Numerical Model ◽  
Wind Power ◽  
Wind Farm ◽  
Bayesian Model Averaging ◽  
Arima Model ◽  
Model Averaging ◽  
Absolute Error ◽  
Ensemble Prediction ◽  
Probabilistic Forecasting

Abstract. The fused product of wind speed for the wind farm is designed through the use of wind speed products of ensemble prediction from the European Centre for Medium-Range Weather Forecasts (ECMWF) and professional numerical model products on wind power based on Mesoscale Model5 (MM5) and Beijing Rapid Update Cycle (BJ-RUC), which are suitable for short-term wind power forecasting and electric dispatch. The single-valued forecast is formed by calculating the different ensemble statistics of the Bayesian probabilistic forecasting representing the uncertainty of ECMWF ensemble prediction. Using autoregressive integrated moving average (ARIMA) model to improve the time resolution of the single-valued forecast, and based on the Bayesian model averaging (BMA) and the deterministic numerical model prediction, the optimal wind speed forecasting curve and the confidence interval are provided. The result shows that the fusion forecast has made obvious improvement to the accuracy relative to the existing numerical forecasting products. Compared with the 0–24 h existing deterministic forecast in the validation period, the mean absolute error (MAE) is decreased by 24.3 % and the correlation coefficient (R) is increased by 12.5 %. In comparison with the ECMWF ensemble forecast, the MAE is reduced by 11.7 %, and R is increased 14.5 %. Additionally, MAE did not increase with the prolongation of the forecast ahead.

Probabilistic Visibility Forecasting Using Bayesian Model Averaging

2011 ◽  
Vol 139 (5) ◽  
pp. 1626-1636 ◽  
Author(s):  
Richard M. Chmielecki ◽  
Adrian E. Raftery
Keyword(s):  
Relative Humidity ◽  
Pacific Northwest ◽  
Bayesian Model ◽  
Bayesian Model Averaging ◽  
Weather Forecasting ◽  
Model Averaging ◽  
Ensemble Forecasts ◽  
Predictive Probability ◽  
The Pacific ◽  
Visibility Forecasting

Bayesian model averaging (BMA) is a statistical postprocessing technique that has been used in probabilistic weather forecasting to calibrate forecast ensembles and generate predictive probability density functions (PDFs) for weather quantities. The authors apply BMA to probabilistic visibility forecasting using a predictive PDF that is a mixture of discrete point mass and beta distribution components. Three approaches to developing predictive PDFs for visibility are developed, each using BMA to postprocess an ensemble of visibility forecasts. In the first approach, the ensemble is generated by a translation algorithm that converts predicted concentrations of hydrometeorological variables into visibility. The second approach augments the raw ensemble visibility forecasts with model forecasts of relative humidity and quantitative precipitation. In the third approach, the ensemble members are generated from relative humidity and precipitation alone. These methods are applied to 12-h ensemble forecasts from 2007 to 2008 and are tested against verifying observations recorded at Automated Surface Observing Stations in the Pacific Northwest. Each of the three methods produces predictive PDFs that are calibrated and sharp with respect to both climatology and the raw ensemble.

Probabilistic Wind Vector Forecasting Using Ensembles and Bayesian Model Averaging

2013 ◽  
Vol 141 (6) ◽  
pp. 2107-2119 ◽  
Author(s):  
J. McLean Sloughter ◽  
Tilmann Gneiting ◽  
Adrian E. Raftery
Keyword(s):  
Bayesian Model ◽  
Bayesian Model Averaging ◽  
Prediction Models ◽  
Weather Prediction ◽  
Model Averaging ◽  
Wind Vector ◽  
Bivariate Distributions ◽  
Ensemble Forecasts ◽  
Probabilistic Forecasts ◽  
Wide Range

Abstract Probabilistic forecasts of wind vectors are becoming critical as interest grows in wind as a clean and renewable source of energy, in addition to a wide range of other uses, from aviation to recreational boating. Unlike other common forecasting problems, which deal with univariate quantities, statistical approaches to wind vector forecasting must be based on bivariate distributions. The prevailing paradigm in weather forecasting is to issue deterministic forecasts based on numerical weather prediction models. Uncertainty can then be assessed through ensemble forecasts, where multiple estimates of the current state of the atmosphere are used to generate a collection of deterministic predictions. Ensemble forecasts are often uncalibrated, however, and Bayesian model averaging (BMA) is a statistical way of postprocessing these forecast ensembles to create calibrated predictive probability density functions (PDFs). It represents the predictive PDF as a weighted average of PDFs centered on the individual bias-corrected forecasts, where the weights reflect the forecasts’ relative contributions to predictive skill over a training period. In this paper the authors extend the BMA methodology to use bivariate distributions, enabling them to provide probabilistic forecasts of wind vectors. The BMA method is applied to 48-h-ahead forecasts of wind vectors over the North American Pacific Northwest in 2003 using the University of Washington mesoscale ensemble and is shown to provide better-calibrated probabilistic forecasts than the raw ensemble, which are also sharper than probabilistic forecasts derived from climatology.

scholarly journals Uncertainty analysis of hydrological multi-model ensembles based on CBP-BMA method

2018 ◽  
Vol 49 (5) ◽  
pp. 1636-1651 ◽  
Author(s):  
Shaokun He ◽  
Shenglian Guo ◽  
Zhangjun Liu ◽  
Jiabo Yin ◽  
Kebing Chen ◽  
...  
Keyword(s):  
Flood Control ◽  
Bayesian Model Averaging ◽  
Evaluation Criteria ◽  
Model Performance ◽  
Model Averaging ◽  
Probability Forecast ◽  
Average Deviation ◽  
Probabilistic Forecasts ◽  
Study Results ◽  
Model Ensembles

Abstract Quantification of the inherent uncertainty in hydrologic forecasting is essential for flood control and water resources management. The existing approaches, such as Bayesian model averaging (BMA), hydrologic uncertainty processor (HUP), copula-BMA (CBMA), aim at developing reliable probabilistic forecasts to characterize the uncertainty induced by model structures. In the probability forecast framework, these approaches either assume the probability density function (PDF) to follow a certain distribution, or are unable to reduce bias effectively for complex hydrological forecasts. To overcome these limitations, a copula Bayesian processor associated with BMA (CBP-BMA) method is proposed with ensemble lumped hydrological models. Comparing with the BMA and CBMA methods, the CBP-BMA method relaxes any assumption on the distribution of conditional PDFs. Several evaluation criteria, such as containing ratio, average bandwidth and average deviation amplitude of probabilistic application, are utilized to evaluate the model performance. The case study results demonstrate that the CBP-BMA method can improve hydrological forecasting precision with higher cover ratios more than 90%, which are increased by 4.4% and 3.2%, 2.2% and 1.7% over those of BMA and CBMA during the calibration and validation periods, respectively. The proposed CBP-BMA method provides an alternative approach for uncertainty estimation of hydrological multi-model forecasts.

scholarly journals Technical note: Combining quantile forecasts and predictive distributions of streamflows

2017 ◽  
Vol 21 (11) ◽  
pp. 5493-5502 ◽  
Author(s):  
Konrad Bogner ◽  
Katharina Liechti ◽  
Massimiliano Zappa
Keyword(s):  
Water Resource Management ◽  
Bayesian Model Averaging ◽  
Model Averaging ◽  
Relevant Information ◽  
Technical Note ◽  
Decision Makers ◽  
Forecast Combination ◽  
Predictive Distributions ◽  
Combination Methods ◽  
Probabilistic Forecasts

Abstract. The enhanced availability of many different hydro-meteorological modelling and forecasting systems raises the issue of how to optimally combine this great deal of information. Especially the usage of deterministic and probabilistic forecasts with sometimes widely divergent predicted future streamflow values makes it even more complicated for decision makers to sift out the relevant information. In this study multiple streamflow forecast information will be aggregated based on several different predictive distributions, and quantile forecasts. For this combination the Bayesian model averaging (BMA) approach, the non-homogeneous Gaussian regression (NGR), also known as the ensemble model output statistic (EMOS) techniques, and a novel method called Beta-transformed linear pooling (BLP) will be applied. By the help of the quantile score (QS) and the continuous ranked probability score (CRPS), the combination results for the Sihl River in Switzerland with about 5 years of forecast data will be compared and the differences between the raw and optimally combined forecasts will be highlighted. The results demonstrate the importance of applying proper forecast combination methods for decision makers in the field of flood and water resource management.

scholarly journals Bayesian Model Averaging’s Problematic Treatment of Extreme Weather and a Paradigm Shift That Fixes It

2008 ◽  
Vol 136 (12) ◽  
pp. 4641-4652 ◽  
Author(s):  
Craig H. Bishop ◽  
Kevin T. Shanley
Keyword(s):  
Paradigm Shift ◽  
Bayesian Model ◽  
Bayesian Model Averaging ◽  
Model Averaging ◽  
Extreme Weather ◽  
Ensemble Forecasts ◽  
Stochastic Perturbations ◽  
New Approach ◽  
Weather Forecasts ◽  
Probabilistic Forecasts

Abstract Methods of ensemble postprocessing in which continuous probability density functions are constructed from ensemble forecasts by centering functions around each of the ensemble members have come to be called Bayesian model averaging (BMA) or “dressing” methods. Here idealized ensemble forecasting experiments are used to show that these methods are liable to produce systematically unreliable probability forecasts of climatologically extreme weather. It is argued that the failure of these methods is linked to an assumption that the distribution of truth given the forecast can be sampled by adding stochastic perturbations to state estimates, even when these state estimates have a realistic climate. It is shown that this assumption is incorrect, and it is argued that such dressing techniques better describe the likelihood distribution of historical ensemble-mean forecasts given the truth for certain values of the truth. This paradigm shift leads to an approach that incorporates prior climatological information into BMA ensemble postprocessing through Bayes’s theorem. This new approach is shown to cure BMA’s ill treatment of extreme weather by providing a posterior BMA distribution whose probabilistic forecasts are reliable for both extreme and nonextreme weather forecasts.

Sign in / Sign up

Export Citation Format

Share Document