scholarly journals Seasonal Ensemble Forecasts: Are Recalibrated Single Models Better than Multimodels?

2009 ◽  
Vol 137 (4) ◽  
pp. 1460-1479 ◽  
Author(s):  
Andreas P. Weigel ◽  
Mark A. Liniger ◽  
Christof Appenzeller
Keyword(s):  
Ensemble Prediction ◽  
Model Errors ◽  
Seasonal Forecasts ◽  
Multimodel Ensemble ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
Near Surface ◽  
Temperature And Precipitation ◽  
Probabilistic Forecasts ◽  
The Difference

Abstract Multimodel ensemble combination (MMEC) has become an accepted technique to improve probabilistic forecasts from short- to long-range time scales. MMEC techniques typically widen ensemble spread, thus improving the dispersion characteristics and the reliability of the forecasts. This raises the question as to whether the same effect could be achieved in a potentially cheaper way by rescaling single model ensemble forecasts a posteriori such that they become reliable. In this study a climate conserving recalibration (CCR) technique is derived and compared with MMEC. With a simple stochastic toy model it is shown that both CCR and MMEC successfully improve forecast reliability. The difference between these two methods is that CCR conserves resolution but inevitably dilutes the potentially predictable signal while MMEC is in the ideal case able to fully retain the predictable signal and to improve resolution. Therefore, MMEC is conceptually to be preferred, particularly since the effect of CCR depends on the length of the data record and on distributional assumptions. In reality, however, multimodels consist only of a finite number of participating single models, and the model errors are often correlated. Under such conditions, and depending on the skill metric applied, CCR-corrected single models can on average have comparable skill as multimodel ensembles, particularly when the potential model predictability is low. Using seasonal near-surface temperature and precipitation forecasts of three models of the Development of a European Multimodel Ensemble System for Seasonal-to-Interannual Prediction (DEMETER) dataset, it is shown that the conclusions drawn from the toy-model experiments hold equally in a real multimodel ensemble prediction system. All in all, it is not possible to make a general statement on whether CCR or MMEC is the better method. Rather it seems that optimum forecasts can be obtained by a combination of both methods, but only if first MMEC and then CCR is applied. The opposite order—first CCR, then MMEC—is shown to be of only little effect, at least in the context of seasonal forecasts.

scholarly journals A Probabilistic Multimodel Ensemble Approach to Seasonal Prediction

2009 ◽  
Vol 24 (3) ◽  
pp. 812-828 ◽  
Author(s):  
Young-Mi Min ◽  
Vladimir N. Kryjov ◽  
Chung-Kyu Park
Keyword(s):  
Real Time ◽  
Gaussian Approximation ◽  
Seasonal Prediction ◽  
Asia Pacific ◽  
Ensemble Prediction ◽  
Prediction System ◽  
Sampling Errors ◽  
Seasonal Forecasts ◽  
Multimodel Ensemble ◽  
Ensemble Prediction System

Abstract A probabilistic multimodel ensemble prediction system (PMME) has been developed to provide operational seasonal forecasts at the Asia–Pacific Economic Cooperation (APEC) Climate Center (APCC). This system is based on an uncalibrated multimodel ensemble, with model weights inversely proportional to the errors in forecast probability associated with the model sampling errors, and a parametric Gaussian fitting method for the estimate of tercile-based categorical probabilities. It is shown that the suggested method is the most appropriate for use in an operational global prediction system that combines a large number of models, with individual model ensembles essentially differing in size and model weights in the forecast and hindcast datasets being inconsistent. Justification for the use of a Gaussian approximation of the precipitation probability distribution function for global forecasts is also provided. PMME retrospective and real-time forecasts are assessed. For above normal and below normal categories, temperature forecasts outperform climatology for a large part of the globe. Precipitation forecasts are definitely more skillful than random guessing for the extratropics and climatological forecasts for the tropics. The skill of real-time forecasts lies within the range of the interannual variability of the historical forecasts.

A Comparison between Raw Ensemble Output, (Modified) Bayesian Model Averaging, and Extended Logistic Regression Using ECMWF Ensemble Precipitation Reforecasts

2010 ◽  
Vol 138 (11) ◽  
pp. 4199-4211 ◽  
Author(s):  
Maurice J. Schmeits ◽  
Kees J. Kok
Keyword(s):  
Logistic Regression ◽  
Bias Correction ◽  
Bayesian Model ◽  
Bayesian Model Averaging ◽  
Model Averaging ◽  
Ensemble Prediction ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
The Difference ◽  
Model Output Statistics

Abstract Using a 20-yr ECMWF ensemble reforecast dataset of total precipitation and a 20-yr dataset of a dense precipitation observation network in the Netherlands, a comparison is made between the raw ensemble output, Bayesian model averaging (BMA), and extended logistic regression (LR). A previous study indicated that BMA and conventional LR are successful in calibrating multimodel ensemble forecasts of precipitation for a single forecast projection. However, a more elaborate comparison between these methods has not yet been made. This study compares the raw ensemble output, BMA, and extended LR for single-model ensemble reforecasts of precipitation; namely, from the ECMWF ensemble prediction system (EPS). The raw EPS output turns out to be generally well calibrated up to 6 forecast days, if compared to the area-mean 24-h precipitation sum. Surprisingly, BMA is less skillful than the raw EPS output from forecast day 3 onward. This is due to the bias correction in BMA, which applies model output statistics to individual ensemble members. As a result, the spread of the bias-corrected ensemble members is decreased, especially for the longer forecast projections. Here, an additive bias correction is applied instead and the equation for the probability of precipitation in BMA is also changed. These modifications to BMA are referred to as “modified BMA” and lead to a significant improvement in the skill of BMA for the longer projections. If the area-maximum 24-h precipitation sum is used as a predictand, both modified BMA and extended LR improve the raw EPS output significantly for the first 5 forecast days. However, the difference in skill between modified BMA and extended LR does not seem to be statistically significant. Yet, extended LR might be preferred, because incorporating predictors that are different from the predictand is straightforward, in contrast to BMA.

scholarly journals Relative effects of statistical preprocessing and postprocessing on a regional hydrological ensemble prediction system

2017 ◽  
Author(s):  
Sanjib Sharma ◽  
Ridwan Siddique ◽  
Seann Reed ◽  
Peter Ahnert ◽  
Pablo Mendoza ◽  
...  
Keyword(s):  
Surface Temperature ◽  
Warm Season ◽  
Hydrologic Model ◽  
Laboratory Research ◽  
Ensemble Prediction ◽  
Lead Times ◽  
Prediction System ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
Near Surface

Abstract. The relative roles of statistical weather preprocessing and streamflow postprocessing in hydrological ensemble forecasting at short- to medium-range forecast lead times (day 1–7) are investigated. For this purpose, a regional hydrologic ensemble prediction system (RHEPS) is developed and implemented. The RHEPS is comprised by the following components: i) hydrometeorological observations (multisensor precipitation estimates, gridded surface temperature, and gauged streamflow); ii) weather ensemble forecasts (precipitation and near-surface temperature) from the National Centers for Environmental Prediction 11-member Global Ensemble Forecast System Reforecast version 2 (GEFSRv2); iii) NOAA’s Hydrology Laboratory-Research Distributed Hydrologic Model (HL-RDHM); iv) heteroscedastic censored logistic regression (HCLR) as the statistical preprocessor; v) two statistical postprocessors, an autoregressive model with a single exogenous variable (ARX(1,1)) and quantile regression (QR); and vi) a comprehensive verification strategy. To implement the RHEPS, 1 to 7 days weather forecasts from the GEFSRv2 are used to force HL-RDHM and generate raw ensemble streamflow forecasts. Forecasting experiments are conducted in four nested basins in the U.S. middle Atlantic region, ranging in size from 381 to 12,362 km2. Results show that the HCLR preprocessed ensemble precipitation forecasts have greater skill than the raw forecasts. These improvements are more noticeable in the warm season at the longer lead times (> 3 days). Both postprocessors, ARX(1,1) and QR, show gains in skill relative to the raw ensemble flood forecasts but QR outperforms ARX(1,1). Preprocessing alone has little effect on improving the skill of the ensemble flood forecasts. Indeed, postprocessing alone performs similar, in terms of the relative mean error, skill, and reliability, to the more involved scenario that includes both preprocessing and postprocessing. We conclude that statistical preprocessing may not always be a necessary component of the ensemble flood forecasting chain.

scholarly journals Generation of Scenarios from Calibrated Ensemble Forecasts with a Dual-Ensemble Copula-Coupling Approach

2016 ◽  
Vol 144 (12) ◽  
pp. 4737-4750 ◽  
Author(s):  
Zied Ben Bouallègue ◽  
Tobias Heppelmann ◽  
Susanne E. Theis ◽  
Pierre Pinson
Keyword(s):  
Probability Distributions ◽  
Forecast Error ◽  
Computational Cost ◽  
Joint Probability ◽  
Ensemble Prediction ◽  
Small Scale ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
Probabilistic Forecasts ◽  
Joint Probability Distributions

Abstract Probabilistic forecasts in the form of ensembles of scenarios are required for complex decision-making processes. Ensemble forecasting systems provide such products but the spatiotemporal structures of the forecast uncertainty is lost when statistical calibration of the ensemble forecasts is applied for each lead time and location independently. Nonparametric approaches allow the reconstruction of spatiotemporal joint probability distributions at a small computational cost. For example, the ensemble copula coupling (ECC) method rebuilds the multivariate aspect of the forecast from the original ensemble forecasts. Based on the assumption of error stationarity, parametric methods aim to fully describe the forecast dependence structures. In this study, the concept of ECC is combined with past data statistics in order to account for the autocorrelation of the forecast error. The new approach, called d-ECC, is applied to wind forecasts from the high-resolution Consortium for Small-Scale Modeling (COSMO) ensemble prediction system (EPS) run operationally at the German Weather Service (COSMO-DE-EPS). Scenarios generated by ECC and d-ECC are compared and assessed in the form of time series by means of multivariate verification tools and within a product-oriented framework. Verification results over a 3-month period show that the innovative method d-ECC performs as well as or even outperforms ECC in all investigated aspects.

Probabilistic forecast of low-level wind shear over Jeju international airport using non-homogeneous regression model

2021 ◽  
pp. 002072092199706
Author(s):  
Young-Gon Lee ◽  
Chansoo Kim
Keyword(s):  
Reliability Analysis ◽  
Wind Shear ◽  
Probabilistic Forecast ◽  
Ensemble Prediction ◽  
Prediction System ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
International Airport ◽  
Probabilistic Forecasts ◽  
Exchangeable Model

Ensemble verification of low-level wind shear (LLWS) is an important issue in airplane landing operation and management. However, there have been few studies on the probabilistic forecasts of LLWS obtained from ensemble prediction system. In this study, we analyzed a reliability analysis to verify LLWS ensemble member forecasts and observation based on the limited grid points around Jeju International Airport in Jeju. Homogeneous and non-homogeneous regression models were used to reduce the bias and dispersion existing ensemble prediction system and to provide probabilistic forecast. Prior to applying probabilistic forecast model, reliability analysis was conducted by using rank histogram to identify the statistical consistency of LLWS ensemble forecasts and corresponding observations. Based on the results of our study, we found that LLWS ensemble forecasts had a consistent positive bias, indicating over-forecasting, and were under-dispersed for all seasons. To correct such biases, homogeneous regression and non-homogeneous regressions as EMOS (Ensemble Model Output Statistics) and EMOS exchangeable model by assuming exchangeable ensemble members were applied. The prediction skills of the methods were compared by the mean absolute error and continuous ranked probability score. We found that the prediction skills of probabilistic forecasts of EMOS exchangeable model were superior to the bias-corrected forecasts in terms of deterministic prediction.

scholarly journals Perturbing Surface Initial Conditions in a Regional Ensemble Prediction System

2016 ◽  
Vol 144 (9) ◽  
pp. 3377-3390 ◽  
Author(s):  
Martin Bellus ◽  
Yong Wang ◽  
Florian Meier
Keyword(s):  
Initial Conditions ◽  
Lower Troposphere ◽  
Ensemble Prediction ◽  
Limited Area ◽  
Breeding Method ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
Near Surface ◽  
Surface Data ◽  
Screen Level

Two techniques for perturbing surface initial conditions in the regional ensemble system Aire Limitée Adaptation Dynamique Développement International-Limited Area Ensemble Forecasting (ALADIN-LAEF) are presented and investigated in this paper. The first technique is the noncycling surface breeding (NCSB), which combines short-range surface forecasts driven by perturbed atmospheric forcing and the breeding method for generating the perturbations on surface initial conditions. The second technique, which is currently used in the ALADIN-LAEF operational version, applies an ensemble of surface data assimilations (ESDA) in which the observations are randomly perturbed. Both techniques are evaluated over a two-month period from late spring to summer. The results show that the evaluation is more favorable to ESDA. In general, the ensemble forecasts of the observed near-surface meteorological variables (screen-level variables) of ESDA are more skillful than NCSB, in particular for 2-m temperature they are statistically more consistent and reliable. A slightly better statistical reliability for 2-m relative humidity and 10-m wind has been found as well. This could be attributed to the introduction of surface data assimilation in ESDA, which provides more accurate surface initial conditions. Moreover, the observation perturbation in ESDA helps to better estimate the initial condition uncertainties. For the forecast of precipitation and the upper-air variables in the lower troposphere, both ESDA and NCSB perform very similarly, having neutral impact.

Ensemble Forecasts Used for Extreme Wave Warning and Platform Unmanning Strategies

2012 ◽  
Author(s):  
Øistein Hagen ◽  
Bjørn Riise
Keyword(s):  
False Alarm ◽  
False Alarm Rate ◽  
Probabilistic Model ◽  
Ensemble Forecasting ◽  
Ensemble Prediction ◽  
Extreme Wave ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
Safety Margins ◽  
Probabilistic Forecasts

Surface waves often represent the most critical environmental parameter for offshore activities, and reliable forecasts are in many cases essential for safe operation. Currently, decisions on safety during extreme wave situations are generally taken based on deterministic wave forecasts, and uncertainty related to forecasts are acknowledged and accounted for via safety margins. However, unmanning criteria reflecting a consistent probabilistic model for the forecast uncertainty are generally lacking. In 1998 the European Centre for Medium-Range Weather Forecasts (ECMWF) introduced a new method for ocean wave forecasting based on the Ensemble Forecasting technique. In this paper we address how the ensemble prediction system (EPS) can be used to establish a probabilistic model applicable for platform unmanning criteria and procedures. Before ensemble forecasting techniques can be introduced operationally for this purpose it is important to investigate how the decision process can be improved with this method. By improvement we expect the probability forecasts to yield a higher hit rate and a lower false alarm rate. Extreme wave warnings are simulated based on a data series established for a 10-year period and the hit and false-alarm rate from deterministic and probabilistic forecasts are compared with measured data. The ensemble forecasts are used to: • establish a model for uncertainty of forecasts for bad weather • update safety margins used at present such that target safety is obtained if deterministic forecasts are used • discuss a revised scheme where the EPS forecasts are applied on an operational level

scholarly journals DEVELOPMENT OF A EUROPEAN MULTIMODEL ENSEMBLE SYSTEM FOR SEASONAL-TO-INTERANNUAL PREDICTION (DEMETER)

2004 ◽  
Vol 85 (6) ◽  
pp. 853-872 ◽  
Author(s):  
T. N. Palmer ◽  
A. Alessandri ◽  
U. Andersen ◽  
P. Cantelaube ◽  
M. Davey ◽  
...  
Keyword(s):  
Initial Conditions ◽  
Economic Value ◽  
Ensemble Prediction ◽  
Model Ensemble ◽  
Single Model ◽  
Multimodel Ensemble ◽  
Coupled Models ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
Climate Simulations

A multi-model ensemble-based system for seasonal-to-interannual prediction has been developed in a joint European project known as DEMETER (Development of a European Multimodel Ensemble Prediction System for Seasonal to Interannual Prediction). The DEMETER system comprises seven global atmosphere–ocean coupled models, each running from an ensemble of initial conditions. Comprehensive hindcast evaluation demonstrates the enhanced reliability and skill of the multimodel ensemble over a more conventional single-model ensemble approach. In addition, innovative examples of the application of seasonal ensemble forecasts in malaria and crop yield prediction are discussed. The strategy followed in DEMETER deals with important problems such as communication across disciplines, downscaling of climate simulations, and use of probabilistic forecast information in the applications sector, illustrating the economic value of seasonal-to-interannual prediction for society as a whole.

scholarly journals Seasonal Forecasting of the Onset of the Rainy Season in West Africa

Atmosphere ◽  
2019 ◽  
Vol 10 (9) ◽  
pp. 528 ◽  
Author(s):  
Manuel Rauch ◽  
Jan Bliefernicht ◽  
Patrick Laux ◽  
Seyni Salack ◽  
Moussa Waongo ◽  
...  
Keyword(s):  
West Africa ◽  
Fuzzy Rule ◽  
Seasonal Forecasting ◽  
Ensemble Prediction ◽  
Graphical Methods ◽  
Seasonal Forecasts ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
Anomaly Correlation Coefficient ◽  
Computationally Intensive

Seasonal forecasts for monsoonal rainfall characteristics like the onset of the rainy seasons (ORS) are crucial for national weather services in semi-arid regions to better support decision-making in rain-fed agriculture. In this study an approach for seasonal forecasting of the ORS is proposed using precipitation information from a global seasonal ensemble prediction system. It consists of a quantile–quantile-transformation for eliminating systematic differences between ensemble forecasts and observations, a fuzzy-rule based method for estimating the ORS date and graphical methods for an improved visualization of probabilistic ORS forecasts. The performance of the approach is tested for several climate zones (the Sahel, Sudan and Guinean zone) in West Africa for a period of eleven years (2000 to 2010), using hindcasts from the Seasonal Forecasting System 4 of ECMWF. We indicated that seasonal ORS forecasts can be skillful for individual years and specific regions (e.g., the Guinean coasts), but also associated with large uncertainties. A spatial verification of the ORS fields emphasizes the importance of selecting appropriate performance measures (e.g., the anomaly correlation coefficient) to avoid an overestimation of the forecast skill. The graphical methods consist of several common formats used in seasonal forecasting and a new index-based method for a quicker interpretation of probabilistic ORS forecast. The new index can also be applied to other seasonal forecast variables, providing an important alternative to the common forecast formats used in seasonal forecasting. Moreover, the forecasting approach proposed in this study is not computationally intensive and is therefore operational applicable for forecasting centers in tropical and subtropical regions where computing power and bandwidth are often limited.

scholarly journals The Discrete Brier and Ranked Probability Skill Scores

2007 ◽  
Vol 135 (1) ◽  
pp. 118-124 ◽  
Author(s):  
Andreas P. Weigel ◽  
Mark A. Liniger ◽  
Christof Appenzeller
Keyword(s):  
Correction Term ◽  
Analytical Formula ◽  
Skill Score ◽  
Ensemble Prediction ◽  
Ensemble Size ◽  
Ensemble Prediction System ◽  
Near Surface ◽  
Skill Scores ◽  
Probabilistic Forecasts ◽  
New Formulation

Abstract The Brier skill score (BSS) and the ranked probability skill score (RPSS) are widely used measures to describe the quality of categorical probabilistic forecasts. They quantify the extent to which a forecast strategy improves predictions with respect to a (usually climatological) reference forecast. The BSS can thereby be regarded as the special case of an RPSS with two forecast categories. From the work of Müller et al., it is known that the RPSS is negatively biased for ensemble prediction systems with small ensemble sizes, and that a debiased version, the RPSSD, can be obtained quasi empirically by random resampling from the reference forecast. In this paper, an analytical formula is derived to directly calculate the RPSS bias correction for any ensemble size and combination of probability categories, thus allowing an easy implementation of the RPSSD. The correction term itself is identified as the “intrinsic unreliability” of the ensemble prediction system. The performance of this new formulation of the RPSSD is illustrated in two examples. First, it is applied to a synthetic random white noise climate, and then, using the ECMWF Seasonal Forecast System 2, to seasonal predictions of near-surface temperature in several regions of different predictability. In both examples, the skill score is independent of ensemble size while the associated confidence thresholds decrease as the number of ensemble members and forecast/observation pairs increase.

Sign in / Sign up

Export Citation Format

Share Document