Simplifying a hydrological ensemble prediction system with a backward greedy selection of members – Part 1: Optimization criteria

Abstract. Hydrological Ensemble Prediction System (HEPS), obtained by forcing rainfall-runoff models with Meteorological Ensemble Prediction Systems (MEPS), have been recognized as useful approaches to quantify uncertainties of hydrological forecasting systems. This task is complex both in terms of the coupling of information and computational time, which may create an operational barrier. The main objective of the current work is to assess the degree of simplification (reduction of members) of a HEPS configured with 16 lumped hydrological models driven by the 50 weather ensemble forecasts from the European Center for Medium-range Weather Forecasts (ECMWF). Here, the selection of the most relevant members is proposed using a Backward greedy technique with k-fold cross-validation, allowing an optimal use of the information. The methodology draws from a multi-criterion score that represents the combination of resolution, reliability, consistency, and diversity. Results show that the degree of reduction of members can be established in terms of maximum number of members required (complexity of the HEPS) or the maximization of the relationship between the different scores (performance).

Download Full-text

Simplifying a hydrological ensemble prediction system with a backward greedy selection of members – Part 1: Optimization criteria

Hydrology and Earth System Sciences ◽

10.5194/hess-15-3307-2011 ◽

2011 ◽

Vol 15 (11) ◽

pp. 3307-3325 ◽

Cited By ~ 11

Author(s):

D. Brochero ◽

F. Anctil ◽

C. Gagné

Keyword(s):

Computational Time ◽

Ensemble Prediction ◽

Hydrological Models ◽

Rainfall Runoff ◽

Ensemble Forecasts ◽

Ensemble Prediction System ◽

Weather Forecasts ◽

Prediction Systems ◽

The Relationship ◽

Selection Of

Abstract. Hydrological Ensemble Prediction Systems (HEPS), obtained by forcing rainfall-runoff models with Meteorological Ensemble Prediction Systems (MEPS), have been recognized as useful approaches to quantify uncertainties of hydrological forecasting systems. This task is complex both in terms of the coupling of information and computational time, which may create an operational barrier. The main objective of the current work is to assess the degree of simplification (reduction of the number of hydrological members) that can be achieved with a HEPS configured using 16 lumped hydrological models driven by the 50 weather ensemble forecasts from the European Centre for Medium-range Weather Forecasts (ECMWF). Here, Backward Greedy Selection (BGS) is proposed to assess the weight that each model must represent within a subset that offers similar or better performance than a reference set of 800 hydrological members. These hydrological models' weights represent the participation of each hydrological model within a simplified HEPS which would issue real-time forecasts in a relatively short computational time. The methodology uses a variation of the k-fold cross-validation, allowing an optimal use of the information, and employs a multi-criterion framework that represents the combination of resolution, reliability, consistency, and diversity. Results show that the degree of reduction of members can be established in terms of maximum number of members required (complexity of the HEPS) or the maximization of the relationship between the different scores (performance).

Download Full-text

Test of a Poor Man’s Ensemble Prediction System for Short-Range Probability Forecasting

Monthly Weather Review ◽

10.1175/mwr2911.1 ◽

2005 ◽

Vol 133 (7) ◽

pp. 1825-1839 ◽

Cited By ~ 36

Author(s):

A. Arribas ◽

K. B. Robertson ◽

K. R. Mylne

Keyword(s):

Short Range ◽

Weather Prediction ◽

Ensemble Prediction ◽

Small Subset ◽

Prediction System ◽

Ensemble Forecasts ◽

Sea Level Pressure ◽

Ensemble Prediction System ◽

To Come ◽

Prediction Systems

Abstract Current operational ensemble prediction systems (EPSs) are designed specifically for medium-range forecasting, but there is also considerable interest in predictability in the short range, particularly for potential severe-weather developments. A possible option is to use a poor man’s ensemble prediction system (PEPS) comprising output from different numerical weather prediction (NWP) centers. By making use of a range of different models and independent analyses, a PEPS provides essentially a random sampling of both the initial condition and model evolution errors. In this paper the authors investigate the ability of a PEPS using up to 14 models from nine operational NWP centers. The ensemble forecasts are verified for a 101-day period and five variables: mean sea level pressure, 500-hPa geopotential height, temperature at 850 hPa, 2-m temperature, and 10-m wind speed. Results are compared with the operational ECMWF EPS, using the ECMWF analysis as the verifying “truth.” It is shown that, despite its smaller size, PEPS is an efficient way of producing ensemble forecasts and can provide competitive performance in the short range. The best relative performance is found to come from hybrid configurations combining output from a small subset of the ECMWF EPS with other different NWP models.

Download Full-text

On the Impact of Short-Range Meteorological Forecasts for Ensemble Streamflow Predictions

Journal of Hydrometeorology ◽

10.1175/2008jhm959.1 ◽

2008 ◽

Vol 9 (6) ◽

pp. 1301-1317 ◽

Cited By ~ 53

Author(s):

Guillaume Thirel ◽

Fabienne Rousset-Regimbeau ◽

Eric Martin ◽

Florence Habets

Keyword(s):

Short Range ◽

Ensemble Prediction ◽

Prediction System ◽

Streamflow Prediction ◽

Ensemble Forecasts ◽

Ensemble Prediction System ◽

Ensemble Streamflow Prediction ◽

Prediction Systems ◽

Set Up ◽

The Impact

Abstract Ensemble streamflow prediction systems are emerging in the international scientific community in order to better assess hydrologic threats. Two ensemble streamflow prediction systems (ESPSs) were set up at Météo-France using ensemble forecasts from the European Centre for Medium-Range Weather Forecasts (ECMWF) Ensemble Prediction System for the first one, and from the Prévision d’Ensemble Action de Recherche Petite Echelle Grande Echelle (PEARP) ensemble prediction system of Météo-France for the second. This paper presents the evaluation of their capacities to better anticipate severe hydrological events and more generally to estimate the quality of both ESPSs on their globality. The two ensemble predictions were used as input for the same hydrometeorological model. The skills of both ensemble streamflow prediction systems were evaluated over all of France for the precipitation input and streamflow prediction during a 569-day period and for a 2-day short-range scale. The ensemble streamflow prediction system based on the PEARP data was the best for floods and small basins, and the ensemble streamflow prediction system based on the ECMWF data seemed the best adapted for low flows and large basins.

Download Full-text

A Comparison of the ECMWF, MSC, and NCEP Global Ensemble Prediction Systems

Monthly Weather Review ◽

10.1175/mwr2905.1 ◽

2005 ◽

Vol 133 (5) ◽

pp. 1076-1097 ◽

Cited By ~ 406

Author(s):

Roberto Buizza ◽

P. L. Houtekamer ◽

Gerald Pellerin ◽

Zoltan Toth ◽

Yuejian Zhu ◽

...

Keyword(s):

Ensemble Forecasting ◽

Ensemble Prediction ◽

Forecast Errors ◽

Ensemble Forecasts ◽

Ensemble Prediction System ◽

Weather Forecasts ◽

Prediction Systems ◽

Environmental Prediction ◽

Global Systems

Abstract The present paper summarizes the methodologies used at the European Centre for Medium-Range Weather Forecasts (ECMWF), the Meteorological Service of Canada (MSC), and the National Centers for Environmental Prediction (NCEP) to simulate the effect of initial and model uncertainties in ensemble forecasting. The characteristics of the three systems are compared for a 3-month period between May and July 2002. The main conclusions of the study are the following:the performance of ensemble prediction systems strongly depends on the quality of the data assimilation system used to create the unperturbed (best) initial condition and the numerical model used to generate the forecasts;a successful ensemble prediction system should simulate the effect of both initial and model-related uncertainties on forecast errors; andfor all three global systems, the spread of ensemble forecasts is insufficient to systematically capture reality, suggesting that none of them is able to simulate all sources of forecast uncertainty.The relative strengths and weaknesses of the three systems identified in this study can offer guidelines for the future development of ensemble forecasting techniques.

Download Full-text

Multivariate Probabilistic Analysis and Predictability of Medium-Range Ensemble Weather Forecasts

Monthly Weather Review ◽

10.1175/mwr-d-14-00015.1 ◽

2014 ◽

Vol 142 (11) ◽

pp. 4074-4090 ◽

Cited By ~ 10

Author(s):

Jessica Keune ◽

Christian Ohlwein ◽

Andreas Hense

Keyword(s):

Covariance Matrix ◽

Full Rank ◽

Ensemble Prediction ◽

Prediction System ◽

Ensemble Prediction System ◽

Weather Forecasts ◽

Verification Methods ◽

Uncertainty Estimates ◽

Medium Range ◽

Prediction Systems

Abstract Ensemble weather forecasting has been operational for two decades now. However, the related uncertainty analysis in terms of probabilistic postprocessing still focuses on single variables, grid points, or stations. Inevitable dependencies in space and time and between variables are often ignored. To address this problem, two probabilistic postprocessing methods are presented, which are multivariate versions of Gaussian fit and kernel dressing, respectively. The multivariate case requires the estimation of a full rank, invertible covariance matrix. For this purpose, a Graphical Least Absolute Shrinkage and Selection Operators (GLASSO) estimator has been employed that is based on sparse undirected graphical models regularized by an L1 penalty term in order to parameterize the full rank inverse covariance. In all cases, the result is a multidimensional probability density. The forecasts used to test the approach are station forecasts of 2-m temperature and surface pressure from four main global ensemble prediction systems (EPS) with medium-range weather forecasts: the NCEP Global Ensemble Forecast System (GEFS), the Met Office Global and Regional Ensemble Prediction System (MOGREPS), the Canadian Meteorological Centre (CMC) Global Ensemble Prediction System (GEPS), and the ECMWF EPS. To evaluate the multivariate probabilistic postprocessing, especially the uncertainty estimates, common verification methods such as the analysis rank histogram and the continuous ranked probability score (CRPS) are applied. Furthermore, a multivariate extension of the CRPS, the energy score, allows for the verification of a complete medium-range forecast as well as for determining its predictability. It is shown that the predictability is similar for all of the examined ensemble prediction systems, whereas the GLASSO proved to be a useful tool for calibrating the commonly observed underdispersion of ensemble forecasts during the first few lead days by using information from the full covariance matrix.

Download Full-text

The Soverato flood in Southern Italy: performance of global and limited-area ensemble forecasts

Nonlinear Processes in Geophysics ◽

10.5194/npg-10-261-2003 ◽

2003 ◽

Vol 10 (3) ◽

pp. 261-274 ◽

Cited By ~ 16

Author(s):

A. Montani ◽

C. Marsigli ◽

F. Nerozzi ◽

T. Paccagnella ◽

S. Tibaldi ◽

...

Keyword(s):

High Resolution ◽

Southern Italy ◽

Ensemble Prediction ◽

Limited Area ◽

Prediction System ◽

Probabilistic Prediction ◽

Ensemble Forecasts ◽

Ensemble Prediction System ◽

Limited Area Model ◽

Area Model

Abstract. The predictability of the flood event affecting Soverato (Southern Italy) in September 2000 is investigated by considering three different configurations of ECMWF ensemble: the operational Ensemble Prediction System (EPS), the targeted EPS and a high-resolution version of EPS. For each configuration, three successive runs of ECMWF ensemble with the same verification time are grouped together so as to generate a highly-populated "super-ensemble". Then, five members are selected from the super-ensemble and used to provide initial and boundary conditions for the integrations with a limited-area model, whose runs generate a Limited-area Ensemble Prediction System (LEPS). The relative impact of targeting the initial perturbations against increasing the horizontal resolution is assessed for the global ensembles as well as for the properties transferred to LEPS integrations, the attention being focussed on the probabilistic prediction of rainfall over a localised area. At the 108, 84 and 60- hour forecast ranges, the overall performance of the global ensembles is not particularly accurate and the best results are obtained by the high-resolution version of EPS. The LEPS performance is very satisfactory in all configurations and the rainfall maps show probability peaks in the correct regions. LEPS products would have been of great assistance to issue flood risk alerts on the basis of limited-area ensemble forecasts. For the 60-hour forecast range, the sensitivity of the results to the LEPS ensemble size is discussed by comparing a 5-member against a 51-member LEPS, where the limited-area model is nested on all EPS members. Little sensitivity is found as concerns the detection of the regions most likely affected by heavy precipitation, the probability peaks being approximately the same in both configurations.

Download Full-text

Predictions of 2010’s Tropical Cyclones Using the GFS and Ensemble-Based Data Assimilation Methods

Monthly Weather Review ◽

10.1175/mwr-d-11-00079.1 ◽

2011 ◽

Vol 139 (10) ◽

pp. 3243-3247 ◽

Cited By ~ 55

Author(s):

Thomas M. Hamill ◽

Jeffrey S. Whitaker ◽

Daryl T. Kleist ◽

Michael Fiorino ◽

Stanley G. Benjamin

Keyword(s):

Data Assimilation ◽

Three Dimensional ◽

Ensemble Prediction ◽

Ensemble Forecasts ◽

Ensemble Prediction System ◽

Weather Forecasts ◽

Successful Testing ◽

Gfs Model ◽

Environmental Prediction ◽

Statistical Interpolation

Abstract Experimental ensemble predictions of tropical cyclone (TC) tracks from the ensemble Kalman filter (EnKF) using the Global Forecast System (GFS) model were recently validated for the 2009 Northern Hemisphere hurricane season by Hamill et al. A similar suite of tests is described here for the 2010 season. Two major changes were made this season: 1) a reduction in the resolution of the GFS model, from 2009’s T384L64 (~31 km at 25°N) to 2010’s T254L64 (~47 km at 25°N), and some changes in model physics; and 2) the addition of a limited test of deterministic forecasts initialized from a hybrid three-dimensional variational data assimilation (3D-Var)/EnKF method. The GFS/EnKF ensembles continued to produce reduced track errors relative to operational ensemble forecasts created by the National Centers for Environmental Prediction (NCEP), the Met Office (UKMO), and the Canadian Meteorological Centre (CMC). The GFS/EnKF was not uniformly as skillful as the European Centre for Medium-Range Weather Forecasts (ECMWF) ensemble prediction system. GFS/EnKF track forecasts had slightly higher error than ECMWF at longer leads, especially in the western North Pacific, and exhibited poorer calibration between spread and error than in 2009, perhaps in part because of lower model resolution. Deterministic forecasts from the hybrid were competitive with deterministic EnKF ensemble-mean forecasts and superior in track error to those initialized from the operational variational algorithm, the Gridpoint Statistical Interpolation (GSI). Pending further successful testing, the National Oceanic and Atmospheric Administration (NOAA) intends to implement the global hybrid system operationally for data assimilation.

Download Full-text

A Similarity-Based Implementation of the Schaake Shuffle

Monthly Weather Review ◽

10.1175/mwr-d-15-0227.1 ◽

2016 ◽

Vol 144 (5) ◽

pp. 1909-1921 ◽

Cited By ~ 35

Author(s):

Roman Schefzik

Keyword(s):

Prediction Models ◽

Initial Conditions ◽

Weather Prediction ◽

Similarity Criterion ◽

Ensemble Prediction ◽

Ensemble Forecasts ◽

Weather Forecasts ◽

Prediction Horizon ◽

Prediction Systems ◽

Specific Implementation

Contemporary weather forecasts are typically based on ensemble prediction systems, which consist of multiple runs of numerical weather prediction models that vary with respect to the initial conditions and/or the parameterization of the atmosphere. Ensemble forecasts are frequently biased and show dispersion errors and thus need to be statistically postprocessed. However, current postprocessing approaches are often univariate and apply to a single weather quantity at a single location and for a single prediction horizon only, thereby failing to account for potentially crucial dependence structures. Nonparametric multivariate postprocessing methods based on empirical copulas, such as ensemble copula coupling or the Schaake shuffle, can address this shortcoming. A specific implementation of the Schaake shuffle, called the SimSchaake approach, is introduced. The SimSchaake method aggregates univariately postprocessed ensemble forecasts using dependence patterns from past observations. Specifically, the observations are taken from historical dates at which the ensemble forecasts resembled the current ensemble prediction with respect to a specific similarity criterion. The SimSchaake ensemble outperforms all reference ensembles in an application to ensemble forecasts for 2-m temperature from the European Centre for Medium-Range Weather Forecasts.

Download Full-text

Sensitivity of Global Ensemble Forecasts to the Initial Ensemble Mean and Perturbations: Comparison of EnKF, Singular Vector, and 4D-Var Approaches

Monthly Weather Review ◽

10.1175/2010mwr3296.1 ◽

2010 ◽

Vol 138 (10) ◽

pp. 3886-3904 ◽

Cited By ~ 4

Author(s):

Mark Buehner ◽

Ahmed Mahidjiba

Keyword(s):

Ensemble Prediction ◽

Boreal Summer ◽

Prediction System ◽

Ensemble Forecasts ◽

Ensemble Prediction System ◽

Var Analysis ◽

Initial Ensemble ◽

Ensemble Mean ◽

The Tropics ◽

The Impact

Abstract This study examines the sensitivity of global ensemble forecasts to the use of different approaches for specifying both the initial ensemble mean and perturbations. The current operational ensemble prediction system of the Meteorological Service of Canada uses the ensemble Kalman filter (EnKF) to define both the ensemble mean and perturbations. To evaluate the impact of different approaches for obtaining the initial ensemble perturbations, the operational EnKF approach is compared with using either no initial perturbations or perturbations obtained using singular vectors (SVs). The SVs are computed using the (dry) total-energy norm with a 48-h optimization time interval. Random linear combinations of 60 SVs are computed for each of three regions. Next, the impact of replacing the initial ensemble mean, currently the EnKF ensemble mean analysis, with the higher-resolution operational four-dimensional variational data assimilation (4D-Var) analysis is evaluated. For this comparison, perturbations are provided by the EnKF. All experiments are performed over two-month periods during both the boreal summer and winter using a system very similar to the global ensemble prediction system that became operational on 10 July 2007. Relative to the operational configuration that relies on the EnKF, the use of SVs to compute initial perturbations produces small, but statistically significant differences in probabilistic forecast scores in favor of the EnKF both in the tropics and, for a limited set of forecast lead times, in the summer hemisphere extratropics, whereas the results are very similar in the winter hemisphere extratropics. Both approaches lead to significantly better ensemble forecasts than with no initial perturbations, though results are quite similar in the tropics when using SVs and no perturbations. The use of an initial-time norm that does not include information on analysis uncertainty and the lack of linearized moist processes in the calculation of the SVs are two factors that limit the quality of the resulting SV-based ensemble forecasts. Relative to the operational configuration, use of the 4D-Var analysis to specify the initial ensemble mean results in improved probabilistic forecast scores during the boreal summer period in the southern extratropics and tropics, but a near-neutral impact otherwise.

Download Full-text

An evaluation of the canadian global meteorological ensemble prediction system for short-term hydrological forecasting

Hydrology and Earth System Sciences Discussions ◽

10.5194/hessd-6-4891-2009 ◽

2009 ◽

Vol 6 (4) ◽

pp. 4891-4917

Author(s):

J. A. Velázquez ◽

T. Petit ◽

A. Lavoie ◽

M.-A. Boucher ◽

R. Turcotte ◽

...

Keyword(s):

Initial Conditions ◽

Ensemble Prediction ◽

Prediction System ◽

Forecast Horizon ◽

Time Step ◽

Ensemble Forecasts ◽

Ensemble Prediction System ◽

Hydrological Forecasting ◽

Updating Procedure ◽

First Time

Abstract. Hydrological forecasting consists in the assessment of future streamflow. Current deterministic forecasts do not give any information concerning the uncertainty, which might be limiting in a decision-making process. Ensemble forecasts are expected to fill this gap. In July 2007, the Meteorological Service of Canada has improved its ensemble prediction system, which has been operational since 1998. It uses the GEM model to generate a 20-member ensemble on a 100 km grid, at mid-latitudes. This improved system is used for the first time for hydrological ensemble predictions. Five watersheds in Quebec (Canada) are studied: Chaudière, Châteauguay, Du Nord, Kénogami and Du Lièvre. An interesting 17-day rainfall event has been selected in October 2007. Forecasts are produced in a 3 h time step for a 3-day forecast horizon. The deterministic forecast is also available and it is compared with the ensemble ones. In order to correct the bias of the ensemble, an updating procedure has been applied to the output data. Results showed that ensemble forecasts are more skilful than the deterministic ones, as measured by the Continuous Ranked Probability Score (CRPS), especially for 72 h forecasts. However, the hydrological ensemble forecasts are under dispersed: a situation that improves with the increasing length of the prediction horizons. We conjecture that this is due in part to the fact that uncertainty in the initial conditions of the hydrological model is not taken into account.

Download Full-text