scholarly journals Why Should Ensemble Spread Match the RMSE of the Ensemble Mean?

2014 ◽  
Vol 15 (4) ◽  
pp. 1708-1713 ◽  
Author(s):  
V. Fortin ◽  
M. Abaza ◽  
F. Anctil ◽  
R. Turcotte
Keyword(s):  
Good Practice ◽  
Ensemble Prediction ◽  
Square Root ◽  
Prediction System ◽  
Mean Square ◽  
Ensemble Spread ◽  
Ensemble Prediction System ◽  
Correct Equation ◽  
Ensemble Mean ◽  
The Impact

Abstract When evaluating the reliability of an ensemble prediction system, it is common to compare the root-mean-square error of the ensemble mean to the average ensemble spread. While this is indeed good practice, two different and inconsistent methodologies have been used over the last few years in the meteorology and hydrology literature to compute the average ensemble spread. In some cases, the square root of average ensemble variance is used, and in other cases, the average of ensemble standard deviation is computed instead. The second option is incorrect. To avoid the perpetuation of practices that are not supported by probability theory, the correct equation for computing the average ensemble spread is obtained and the impact of using the wrong equation is illustrated.

scholarly journals Sensitivity of Global Ensemble Forecasts to the Initial Ensemble Mean and Perturbations: Comparison of EnKF, Singular Vector, and 4D-Var Approaches

2010 ◽  
Vol 138 (10) ◽  
pp. 3886-3904 ◽  
Author(s):  
Mark Buehner ◽  
Ahmed Mahidjiba
Keyword(s):  
Ensemble Prediction ◽  
Boreal Summer ◽  
Prediction System ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
Var Analysis ◽  
Initial Ensemble ◽  
Ensemble Mean ◽  
The Tropics ◽  
The Impact

Abstract This study examines the sensitivity of global ensemble forecasts to the use of different approaches for specifying both the initial ensemble mean and perturbations. The current operational ensemble prediction system of the Meteorological Service of Canada uses the ensemble Kalman filter (EnKF) to define both the ensemble mean and perturbations. To evaluate the impact of different approaches for obtaining the initial ensemble perturbations, the operational EnKF approach is compared with using either no initial perturbations or perturbations obtained using singular vectors (SVs). The SVs are computed using the (dry) total-energy norm with a 48-h optimization time interval. Random linear combinations of 60 SVs are computed for each of three regions. Next, the impact of replacing the initial ensemble mean, currently the EnKF ensemble mean analysis, with the higher-resolution operational four-dimensional variational data assimilation (4D-Var) analysis is evaluated. For this comparison, perturbations are provided by the EnKF. All experiments are performed over two-month periods during both the boreal summer and winter using a system very similar to the global ensemble prediction system that became operational on 10 July 2007. Relative to the operational configuration that relies on the EnKF, the use of SVs to compute initial perturbations produces small, but statistically significant differences in probabilistic forecast scores in favor of the EnKF both in the tropics and, for a limited set of forecast lead times, in the summer hemisphere extratropics, whereas the results are very similar in the winter hemisphere extratropics. Both approaches lead to significantly better ensemble forecasts than with no initial perturbations, though results are quite similar in the tropics when using SVs and no perturbations. The use of an initial-time norm that does not include information on analysis uncertainty and the lack of linearized moist processes in the calculation of the SVs are two factors that limit the quality of the resulting SV-based ensemble forecasts. Relative to the operational configuration, use of the 4D-Var analysis to specify the initial ensemble mean results in improved probabilistic forecast scores during the boreal summer period in the southern extratropics and tropics, but a near-neutral impact otherwise.

scholarly journals Regional Differences in the Prediction of Extratropical Cyclones by the ECMWF Ensemble Prediction System

2009 ◽  
Vol 137 (3) ◽  
pp. 893-911 ◽  
Author(s):  
Lizzie S. R. Froude
Keyword(s):  
Regional Differences ◽  
Propagation Speed ◽  
Ensemble Prediction ◽  
Extratropical Cyclones ◽  
Spatial Distributions ◽  
Prediction System ◽  
Ensemble Spread ◽  
Ensemble Prediction System ◽  
Mean Error ◽  
Ensemble Mean

Abstract A regional study of the prediction of extratropical cyclones by the European Centre for Medium-Range Weather Forecasts (ECMWF) Ensemble Prediction System (EPS) has been performed. An objective feature-tracking method has been used to identify and track the cyclones along the forecast trajectories. Forecast error statistics have then been produced for the position, intensity, and propagation speed of the storms. In previous work, data limitations meant it was only possible to present the diagnostics for the entire Northern Hemisphere (NH) or Southern Hemisphere. A larger data sample has allowed the diagnostics to be computed separately for smaller regions around the globe and has made it possible to explore the regional differences in the prediction of storms by the EPS. Results show that in the NH there is a larger ensemble mean error in the position of storms over the Atlantic Ocean. Further analysis revealed that this is mainly due to errors in the prediction of storm propagation speed rather than in direction. Forecast storms propagate too slowly in all regions, but the bias is about 2 times as large in the NH Atlantic region. The results show that storm intensity is generally overpredicted over the ocean and underpredicted over the land and that the absolute error in intensity is larger over the ocean than over the land. In the NH, large errors occur in the prediction of the intensity of storms that originate as tropical cyclones but then move into the extratropics. The ensemble is underdispersive for the intensity of cyclones (i.e., the spread is smaller than the mean error) in all regions. The spatial patterns of the ensemble mean error and ensemble spread are very different for the intensity of cyclones. Spatial distributions of the ensemble mean error suggest that large errors occur during the growth phase of storm development, but this is not indicated by the spatial distributions of the ensemble spread. In the NH there are further differences. First, the large errors in the prediction of the intensity of cyclones that originate in the tropics are not indicated by the spread. Second, the ensemble mean error is larger over the Pacific Ocean than over the Atlantic, whereas the opposite is true for the spread. The use of a storm-tracking approach, to both weather forecasters and developers of forecast systems, is also discussed.

A Unified Scheme of Stochastic Physics and Bias Correction in an Ensemble Model to Reduce Both Random and Systematic Errors

2019 ◽  
Vol 34 (6) ◽  
pp. 1675-1691 ◽  
Author(s):  
Yu Xia ◽  
Jing Chen ◽  
Jun Du ◽  
Xiefei Zhi ◽  
Jingzhuo Wang ◽  
...  
Keyword(s):  
Bias Correction ◽  
Weather Prediction ◽  
Systematic Errors ◽  
Ensemble Prediction ◽  
Ensemble Model ◽  
Prediction System ◽  
Ensemble Prediction System ◽  
Probabilistic Forecasts ◽  
Ensemble Mean ◽  
Stochastic Physics

Abstract This study experimented with a unified scheme of stochastic physics and bias correction within a regional ensemble model [Global and Regional Assimilation and Prediction System–Regional Ensemble Prediction System (GRAPES-REPS)]. It is intended to improve ensemble prediction skill by reducing both random and systematic errors at the same time. Three experiments were performed on top of GRAPES-REPS. The first experiment adds only the stochastic physics. The second experiment adds only the bias correction scheme. The third experiment adds both the stochastic physics and bias correction. The experimental period is one month from 1 to 31 July 2015 over the China domain. Using 850-hPa temperature as an example, the study reveals the following: 1) the stochastic physics can effectively increase the ensemble spread, while the bias correction cannot. Therefore, ensemble averaging of the stochastic physics runs can reduce more random error than the bias correction runs. 2) Bias correction can significantly reduce systematic error, while the stochastic physics cannot. As a result, the bias correction greatly improved the quality of ensemble mean forecasts but the stochastic physics did not. 3) The unified scheme can greatly reduce both random and systematic errors at the same time and performed the best of the three experiments. These results were further confirmed by verification of the ensemble mean, spread, and probabilistic forecasts of many other atmospheric fields for both upper air and the surface, including precipitation. Based on this study, we recommend that operational numerical weather prediction centers adopt this unified scheme approach in ensemble models to achieve the best forecasts.

scholarly journals On the Impact of the Choice of Global Ensemble in Forcing a Regional Ensemble System

2016 ◽  
Vol 31 (2) ◽  
pp. 515-530 ◽  
Author(s):  
Florian Weidle ◽  
Yong Wang ◽  
Geert Smet
Keyword(s):  
Large Scale ◽  
Ensemble Forecasting ◽  
Ensemble Prediction ◽  
Lateral Boundary Condition ◽  
Limited Area ◽  
Prediction System ◽  
Ensemble Prediction System ◽  
Driving Model ◽  
Environmental Prediction ◽  
The Impact

Abstract It is quite common that in a regional ensemble system the large-scale initial condition (IC) perturbations and the lateral boundary condition (LBC) perturbations are taken from a global ensemble prediction system (EPS). The choice of global EPS as a driving model can have a significant impact on the performance of the regional EPS. This study investigates the impact of large-scale IC/LBC perturbations obtained from different global EPSs on the forecast quality of a regional EPS. For this purpose several experiments are conducted where the Aire Limitée Adaption dynamique Développement International–Limited Area Ensemble Forecasting (ALADIN-LAEF) regional ensemble is forced by two of the world’s leading global ensembles, the European Centre for Medium-Range Weather Forecasts’ Ensemble Prediction System (ECMWF-EPS) and the Global Ensemble Forecasting System (GEFS) from the National Centers for Environmental Prediction (NCEP), which provide the IC and LBC perturbations. The investigation is carried out for a 51-day period during summer 2010 over central Europe. The results indicate that forcing of the regional ensemble with GEFS performs better for surface parameters, whereas at upper levels forcing with ECMWF-EPS is superior. Using perturbations from GEFS lead to a considerably higher spread in ALADIN-LAEF, which is beneficial near the surface where regional EPSs are usually underdispersive. At upper levels, forcing with GEFS leads to an overdispersion of ALADIN-LAEF as a result of the large spread of some parameters, where forcing ALADIN-LAEF with ECMWF-EPS provides statistically more reliable forecasts. The results indicate that the best global EPS might not always provide the best ICs and LBCs for a regional ensemble.

scholarly journals Oil spill model uncertainty quantification using an atmospheric ensemble

Ocean Science ◽  
2021 ◽  
Vol 17 (4) ◽  
pp. 919-934
Author(s):  
Konstantinos Kampouris ◽  
Vassilios Vervatis ◽  
John Karagiorgos ◽  
Sarantis Sofianos
Keyword(s):  
Oil Spill ◽  
Model Uncertainty ◽  
Performance Metrics ◽  
Aegean Sea ◽  
Mitigation Strategies ◽  
Ensemble Prediction ◽  
Prediction System ◽  
Atmospheric Forcing ◽  
Ensemble Prediction System ◽  
The Impact

Abstract. We investigate the impact of atmospheric forcing uncertainties on the prediction of the dispersion of pollutants in the marine environment. Ensemble simulations consisting of 50 members were carried out using the ECMWF ensemble prediction system and the oil spill model MEDSLIK-II in the Aegean Sea. A deterministic control run using the unperturbed wind of the ECMWF high-resolution system served as reference for the oil spill prediction. We considered the oil spill rates and duration to be similar to major accidents of the past (e.g., the Prestige case) and we performed simulations for different seasons and oil spill types. Oil spill performance metrics and indices were introduced in the context of probabilistic hazard assessment. Results suggest that oil spill model uncertainties were sensitive to the atmospheric forcing uncertainties, especially to phase differences in the intensity and direction of the wind among members. An oil spill ensemble prediction system based on model uncertainty of the atmospheric forcing, shows great potential for predicting pathways of oil spill transport alongside a deterministic simulation, increasing the reliability of the model prediction and providing important information for the control and mitigation strategies in the event of an oil spill accident.

A large ensemble decadal prediction system with MPI-ESM

2021 ◽  
Author(s):  
Sebastian Brune ◽  
Vimal Koul ◽  
David Marcolino Nielsen ◽  
Laura Hövel ◽  
Holger Pohlmann ◽  
...  
Keyword(s):  
North Atlantic ◽  
Prediction Skill ◽  
Ensemble Prediction ◽  
Prediction System ◽  
Decadal Prediction ◽  
Ensemble Size ◽  
Large Ensemble ◽  
Ensemble Prediction System ◽  
Atlantic Sector ◽  
Ensemble Mean

<p>Current state-of-the-art decadal ensemble prediction systems are run with an ensemble size of 10 to 40 members, their retrospective forecasts of the past are used to assess the system's prediction skill. Here, we present an attempt for a large ensemble decadal prediction system for the time period 1960-today, with an ensemble size of 80 members, based on the low resolution version of the Max Planck Institute Earth system model (MPI-ESM-LR). The ensemble is forced with CMIP6 conditions and initialized every year in November through a weakly coupled assimilation using atmospheric reanalyses via nudging and observed oceanic temperature and salinity profiles via a 16-member ensemble Kalman filter. To generate ensemble members beyond 16, we use additional physical perturbations at stratospheric height. The analysis of our large ensemble prediction system presented here aims for answering two questions: (1) How does the ensemble mean deterministic prediction skill for global and North Atlantic key climate indices change with ensemble size? (2) How well may the 80-member ensemble serve as a basis for a robust statistical analysis of probabilities of extremes in the North Atlantic sector? Preliminary results for global and regional air surface temperature show that in terms of ensemble mean ACC and full ensemble CPRSS with reference data, the 80-member ensemble leads to similar prediction skill as the 16-member ensemble. This indicates that the additional ensemble members may lead to a better sampling of the distribution of model trajectories, paving the way for a more robust statistical probabilistic analysis.</p>

scholarly journals Model Error Representation in an Operational Ensemble Kalman Filter

2009 ◽  
Vol 137 (7) ◽  
pp. 2126-2143 ◽  
Author(s):  
P. L. Houtekamer ◽  
Herschel L. Mitchell ◽  
Xingxiu Deng
Keyword(s):  
Kalman Filter ◽  
Ensemble Kalman Filter ◽  
Model Error ◽  
Ensemble Prediction ◽  
Isotropic Model ◽  
Error Sources ◽  
Ensemble Prediction System ◽  
Stochastic Perturbations ◽  
Ensemble Mean ◽  
The Impact

Since 12 January 2005, an ensemble Kalman filter (EnKF) has been used operationally at the Meteorological Service of Canada to provide the initial conditions for the medium-range forecasts of the ensemble prediction system. One issue in EnKF development is how to best account for model error. It is shown that in a perfect-model environment, without any model error or model error simulation, the EnKF spread remains representative of the ensemble mean error with respect to a truth integration. Consequently, the EnKF can be used to quantify the impact of the various error sources in a data-assimilation cycle on the quality of the ensemble mean. Using real rather than simulated observations, but still not simulating model error in any manner, the rms ensemble spread is found to be too small by approximately a factor of 2. It is then attempted to account for model error by using various combinations of the following four different approaches: (i) additive isotropic model error perturbations; (ii) different versions of the model for different ensemble members; (iii) stochastic perturbations to physical tendencies; and (iv) stochastic kinetic energy backscatter. The addition of isotropic model error perturbations is found to have the biggest impact. The identification of model error sources could lead to a more realistic, likely anisotropic, parameterization. Using different versions of the model has a small but clearly positive impact and consequently both (i) and (ii) are used in the operational EnKF. The use of approaches (iii) and (iv) did not lead to further improvements.

scholarly journals Representation of model error in a convective-scale ensemble prediction system

2014 ◽  
Vol 21 (1) ◽  
pp. 19-39 ◽  
Author(s):  
L. H. Baker ◽  
A. C. Rudd ◽  
S. Migliorini ◽  
R. N. Bannister
Keyword(s):  
Model Error ◽  
Ensemble Prediction ◽  
Initial Condition ◽  
Prediction System ◽  
Ensemble Prediction System ◽  
Near Surface ◽  
Skill Scores ◽  
The Uk ◽  
Parameter Values ◽  
The Impact

Abstract. In this paper ensembles of forecasts (of up to six hours) are studied from a convection-permitting model with a representation of model error due to unresolved processes. The ensemble prediction system (EPS) used is an experimental convection-permitting version of the UK Met Office's 24-member Global and Regional Ensemble Prediction System (MOGREPS). The method of representing model error variability, which perturbs parameters within the model's parameterisation schemes, has been modified and we investigate the impact of applying this scheme in different ways. These are: a control ensemble where all ensemble members have the same parameter values; an ensemble where the parameters are different between members, but fixed in time; and ensembles where the parameters are updated randomly every 30 or 60 min. The choice of parameters and their ranges of variability have been determined from expert opinion and parameter sensitivity tests. A case of frontal rain over the southern UK has been chosen, which has a multi-banded rainfall structure. The consequences of including model error variability in the case studied are mixed and are summarised as follows. The multiple banding, evident in the radar, is not captured for any single member. However, the single band is positioned in some members where a secondary band is present in the radar. This is found for all ensembles studied. Adding model error variability with fixed parameters in time does increase the ensemble spread for near-surface variables like wind and temperature, but can actually decrease the spread of the rainfall. Perturbing the parameters periodically throughout the forecast does not further increase the spread and exhibits "jumpiness" in the spread at times when the parameters are perturbed. Adding model error variability gives an improvement in forecast skill after the first 2–3 h of the forecast for near-surface temperature and relative humidity. For precipitation skill scores, adding model error variability has the effect of improving the skill in the first 1–2 h of the forecast, but then of reducing the skill after that. Complementary experiments were performed where the only difference between members was the set of parameter values (i.e. no initial condition variability). The resulting spread was found to be significantly less than the spread from initial condition variability alone.

scholarly journals On the Impact of Short-Range Meteorological Forecasts for Ensemble Streamflow Predictions

2008 ◽  
Vol 9 (6) ◽  
pp. 1301-1317 ◽  
Author(s):  
Guillaume Thirel ◽  
Fabienne Rousset-Regimbeau ◽  
Eric Martin ◽  
Florence Habets
Keyword(s):  
Short Range ◽  
Ensemble Prediction ◽  
Prediction System ◽  
Streamflow Prediction ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
Ensemble Streamflow Prediction ◽  
Prediction Systems ◽  
Set Up ◽  
The Impact

Abstract Ensemble streamflow prediction systems are emerging in the international scientific community in order to better assess hydrologic threats. Two ensemble streamflow prediction systems (ESPSs) were set up at Météo-France using ensemble forecasts from the European Centre for Medium-Range Weather Forecasts (ECMWF) Ensemble Prediction System for the first one, and from the Prévision d’Ensemble Action de Recherche Petite Echelle Grande Echelle (PEARP) ensemble prediction system of Météo-France for the second. This paper presents the evaluation of their capacities to better anticipate severe hydrological events and more generally to estimate the quality of both ESPSs on their globality. The two ensemble predictions were used as input for the same hydrometeorological model. The skills of both ensemble streamflow prediction systems were evaluated over all of France for the precipitation input and streamflow prediction during a 569-day period and for a 2-day short-range scale. The ensemble streamflow prediction system based on the PEARP data was the best for floods and small basins, and the ensemble streamflow prediction system based on the ECMWF data seemed the best adapted for low flows and large basins.

Sign in / Sign up

Export Citation Format

Share Document