Ensemble mean forecast skill and applications with the T213 ensemble prediction system

Abstract This study experimented with a unified scheme of stochastic physics and bias correction within a regional ensemble model [Global and Regional Assimilation and Prediction System–Regional Ensemble Prediction System (GRAPES-REPS)]. It is intended to improve ensemble prediction skill by reducing both random and systematic errors at the same time. Three experiments were performed on top of GRAPES-REPS. The first experiment adds only the stochastic physics. The second experiment adds only the bias correction scheme. The third experiment adds both the stochastic physics and bias correction. The experimental period is one month from 1 to 31 July 2015 over the China domain. Using 850-hPa temperature as an example, the study reveals the following: 1) the stochastic physics can effectively increase the ensemble spread, while the bias correction cannot. Therefore, ensemble averaging of the stochastic physics runs can reduce more random error than the bias correction runs. 2) Bias correction can significantly reduce systematic error, while the stochastic physics cannot. As a result, the bias correction greatly improved the quality of ensemble mean forecasts but the stochastic physics did not. 3) The unified scheme can greatly reduce both random and systematic errors at the same time and performed the best of the three experiments. These results were further confirmed by verification of the ensemble mean, spread, and probabilistic forecasts of many other atmospheric fields for both upper air and the surface, including precipitation. Based on this study, we recommend that operational numerical weather prediction centers adopt this unified scheme approach in ensemble models to achieve the best forecasts.

Download Full-text

Sensitivity of Global Ensemble Forecasts to the Initial Ensemble Mean and Perturbations: Comparison of EnKF, Singular Vector, and 4D-Var Approaches

Monthly Weather Review ◽

10.1175/2010mwr3296.1 ◽

2010 ◽

Vol 138 (10) ◽

pp. 3886-3904 ◽

Cited By ~ 4

Author(s):

Mark Buehner ◽

Ahmed Mahidjiba

Keyword(s):

Ensemble Prediction ◽

Boreal Summer ◽

Prediction System ◽

Ensemble Forecasts ◽

Ensemble Prediction System ◽

Var Analysis ◽

Initial Ensemble ◽

Ensemble Mean ◽

The Tropics ◽

The Impact

Abstract This study examines the sensitivity of global ensemble forecasts to the use of different approaches for specifying both the initial ensemble mean and perturbations. The current operational ensemble prediction system of the Meteorological Service of Canada uses the ensemble Kalman filter (EnKF) to define both the ensemble mean and perturbations. To evaluate the impact of different approaches for obtaining the initial ensemble perturbations, the operational EnKF approach is compared with using either no initial perturbations or perturbations obtained using singular vectors (SVs). The SVs are computed using the (dry) total-energy norm with a 48-h optimization time interval. Random linear combinations of 60 SVs are computed for each of three regions. Next, the impact of replacing the initial ensemble mean, currently the EnKF ensemble mean analysis, with the higher-resolution operational four-dimensional variational data assimilation (4D-Var) analysis is evaluated. For this comparison, perturbations are provided by the EnKF. All experiments are performed over two-month periods during both the boreal summer and winter using a system very similar to the global ensemble prediction system that became operational on 10 July 2007. Relative to the operational configuration that relies on the EnKF, the use of SVs to compute initial perturbations produces small, but statistically significant differences in probabilistic forecast scores in favor of the EnKF both in the tropics and, for a limited set of forecast lead times, in the summer hemisphere extratropics, whereas the results are very similar in the winter hemisphere extratropics. Both approaches lead to significantly better ensemble forecasts than with no initial perturbations, though results are quite similar in the tropics when using SVs and no perturbations. The use of an initial-time norm that does not include information on analysis uncertainty and the lack of linearized moist processes in the calculation of the SVs are two factors that limit the quality of the resulting SV-based ensemble forecasts. Relative to the operational configuration, use of the 4D-Var analysis to specify the initial ensemble mean results in improved probabilistic forecast scores during the boreal summer period in the southern extratropics and tropics, but a near-neutral impact otherwise.

Download Full-text

A Regional Ensemble Prediction System Based on Moist Targeted Singular Vectors and Stochastic Parameter Perturbations

Monthly Weather Review ◽

10.1175/2007mwr2109.1 ◽

2008 ◽

Vol 136 (2) ◽

pp. 443-462 ◽

Cited By ~ 41

Author(s):

Xiaoli Li ◽

Martin Charron ◽

Lubos Spacek ◽

Guillem Candille

Keyword(s):

Deep Convection ◽

Horizontal Resolution ◽

Forecast Skill ◽

Convective Precipitation ◽

Ensemble Prediction ◽

Limited Area ◽

Prediction System ◽

Ensemble Prediction System ◽

Stochastic Perturbations ◽

Singular Vectors

Abstract A regional ensemble prediction system (REPS) with the limited-area version of the Canadian Global Environmental Multiscale (GEM) model at 15-km horizontal resolution is developed and tested. The total energy norm singular vectors (SVs) targeted over northeastern North America are used for initial and boundary perturbations. Two SV perturbation strategies are tested: dry SVs with dry simplified physics and moist SVs with simplified physics, including stratiform condensation and convective precipitation as well as dry processes. Model physics uncertainties are partly accounted for by stochastically perturbing two parameters: the threshold vertical velocity in the trigger function of the Kain–Fritsch deep convection scheme, and the threshold humidity in the Sundqvist explicit scheme. The perturbations are obtained from first-order Markov processes. Short-range ensemble forecasts in summer with 16 members are performed for five different experiments. The experiments employ different perturbation and piloting strategies, and two different surface schemes. Verification focuses on quantitative precipitation forecasts and is done using a range of probabilistic measures. Results indicate that using moist SVs instead of dry SVs has a stronger impact on precipitation than on dynamical fields. Forecast skill for precipitation is greatly influenced by the dominant synoptic weather systems. For stratiform precipitation caused by strong baroclinic systems, the forecast skill is improved in the moist SV experiments relative to the dry SV experiments. For convective precipitation rates in the range 15–50 mm (24 h)−1 produced by weak synoptic baroclinic systems, all experiments exhibit noticeably poorer forecast skills. Skill improvements due to the Interactions between Soil, Biosphere, and Atmosphere (ISBA) surface scheme and stochastic perturbations are also observed.

Download Full-text

A large ensemble decadal prediction system with MPI-ESM

10.5194/egusphere-egu21-9263 ◽

2021 ◽

Author(s):

Sebastian Brune ◽

Vimal Koul ◽

David Marcolino Nielsen ◽

Laura Hövel ◽

Holger Pohlmann ◽

...

Keyword(s):

North Atlantic ◽

Prediction Skill ◽

Ensemble Prediction ◽

Prediction System ◽

Decadal Prediction ◽

Ensemble Size ◽

Large Ensemble ◽

Ensemble Prediction System ◽

Atlantic Sector ◽

Ensemble Mean

<p>Current state-of-the-art decadal ensemble prediction systems are run with an ensemble size of 10 to 40 members, their retrospective forecasts of the past are used to assess the system's prediction skill. Here, we present an attempt for a large ensemble decadal prediction system for the time period 1960-today, with an ensemble size of 80 members, based on the low resolution version of the Max Planck Institute Earth system model (MPI-ESM-LR). The ensemble is forced with CMIP6 conditions and initialized every year in November through a weakly coupled assimilation using atmospheric reanalyses via nudging and observed oceanic temperature and salinity profiles via a 16-member ensemble Kalman filter. To generate ensemble members beyond 16, we use additional physical perturbations at stratospheric height. The analysis of our large ensemble prediction system presented here aims for answering two questions: (1) How does the ensemble mean deterministic prediction skill for global and North Atlantic key climate indices change with ensemble size? (2) How well may the 80-member ensemble serve as a basis for a robust statistical analysis of probabilities of extremes in the North Atlantic sector? Preliminary results for global and regional air surface temperature show that in terms of ensemble mean ACC and full ensemble CPRSS with reference data, the 80-member ensemble leads to similar prediction skill as the 16-member ensemble. This indicates that the additional ensemble members may lead to a better sampling of the distribution of model trajectories, paving the way for a more robust statistical probabilistic analysis.</p>

Download Full-text

Regional Differences in the Prediction of Extratropical Cyclones by the ECMWF Ensemble Prediction System

Monthly Weather Review ◽

10.1175/2008mwr2610.1 ◽

2009 ◽

Vol 137 (3) ◽

pp. 893-911 ◽

Cited By ~ 19

Author(s):

Lizzie S. R. Froude

Keyword(s):

Regional Differences ◽

Propagation Speed ◽

Ensemble Prediction ◽

Extratropical Cyclones ◽

Spatial Distributions ◽

Prediction System ◽

Ensemble Spread ◽

Ensemble Prediction System ◽

Mean Error ◽

Ensemble Mean

Abstract A regional study of the prediction of extratropical cyclones by the European Centre for Medium-Range Weather Forecasts (ECMWF) Ensemble Prediction System (EPS) has been performed. An objective feature-tracking method has been used to identify and track the cyclones along the forecast trajectories. Forecast error statistics have then been produced for the position, intensity, and propagation speed of the storms. In previous work, data limitations meant it was only possible to present the diagnostics for the entire Northern Hemisphere (NH) or Southern Hemisphere. A larger data sample has allowed the diagnostics to be computed separately for smaller regions around the globe and has made it possible to explore the regional differences in the prediction of storms by the EPS. Results show that in the NH there is a larger ensemble mean error in the position of storms over the Atlantic Ocean. Further analysis revealed that this is mainly due to errors in the prediction of storm propagation speed rather than in direction. Forecast storms propagate too slowly in all regions, but the bias is about 2 times as large in the NH Atlantic region. The results show that storm intensity is generally overpredicted over the ocean and underpredicted over the land and that the absolute error in intensity is larger over the ocean than over the land. In the NH, large errors occur in the prediction of the intensity of storms that originate as tropical cyclones but then move into the extratropics. The ensemble is underdispersive for the intensity of cyclones (i.e., the spread is smaller than the mean error) in all regions. The spatial patterns of the ensemble mean error and ensemble spread are very different for the intensity of cyclones. Spatial distributions of the ensemble mean error suggest that large errors occur during the growth phase of storm development, but this is not indicated by the spatial distributions of the ensemble spread. In the NH there are further differences. First, the large errors in the prediction of the intensity of cyclones that originate in the tropics are not indicated by the spread. Second, the ensemble mean error is larger over the Pacific Ocean than over the Atlantic, whereas the opposite is true for the spread. The use of a storm-tracking approach, to both weather forecasters and developers of forecast systems, is also discussed.

Download Full-text

Why Should Ensemble Spread Match the RMSE of the Ensemble Mean?

Journal of Hydrometeorology ◽

10.1175/jhm-d-14-0008.1 ◽

2014 ◽

Vol 15 (4) ◽

pp. 1708-1713 ◽

Cited By ~ 75

Author(s):

V. Fortin ◽

M. Abaza ◽

F. Anctil ◽

R. Turcotte

Keyword(s):

Good Practice ◽

Ensemble Prediction ◽

Square Root ◽

Prediction System ◽

Mean Square ◽

Ensemble Spread ◽

Ensemble Prediction System ◽

Correct Equation ◽

Ensemble Mean ◽

The Impact

Abstract When evaluating the reliability of an ensemble prediction system, it is common to compare the root-mean-square error of the ensemble mean to the average ensemble spread. While this is indeed good practice, two different and inconsistent methodologies have been used over the last few years in the meteorology and hydrology literature to compute the average ensemble spread. In some cases, the square root of average ensemble variance is used, and in other cases, the average of ensemble standard deviation is computed instead. The second option is incorrect. To avoid the perpetuation of practices that are not supported by probability theory, the correct equation for computing the average ensemble spread is obtained and the impact of using the wrong equation is illustrated.

Download Full-text

Potential Forecast Skill of Ensemble Prediction and Spread and Skill Distributions of the ECMWF Ensemble Prediction System

Monthly Weather Review ◽

10.1175/1520-0493(1997)125<0099:pfsoep>2.0.co;2 ◽

1997 ◽

Vol 125 (1) ◽

pp. 99-119 ◽

Cited By ~ 140

Author(s):

Roberto Buizza

Keyword(s):

Forecast Skill ◽

Ensemble Prediction ◽

Prediction System ◽

Ensemble Prediction System

Download Full-text

Typhoon Ensemble Prediction System Developed at the Japan Meteorological Agency

Monthly Weather Review ◽

10.1175/2009mwr2697.1 ◽

2009 ◽

Vol 137 (8) ◽

pp. 2592-2604 ◽

Cited By ~ 41

Author(s):

Munehiko Yamaguchi ◽

Ryota Sakai ◽

Masayuki Kyoda ◽

Takuya Komori ◽

Takashi Kadowaki

Keyword(s):

Deterministic Model ◽

Global Analysis ◽

North Pacific Ocean ◽

Japan Meteorological Agency ◽

Ensemble Prediction ◽

Prediction System ◽

Global Spectral Model ◽

Vector Method ◽

Ensemble Prediction System ◽

Ensemble Mean

Abstract The Japan Meteorological Agency (JMA) Typhoon Ensemble Prediction System (TEPS) and its performance are described. In February 2008, JMA started an operation of TEPS that was designed for providing skillful tropical cyclone (TC) track predictions in both deterministic and probabilistic ways. TEPS consists of 1 nonperturbed prediction and 10 perturbed predictions based on the lower-resolution version (TL319L60) of the JMA Global Spectral Model (GSM; TL959L60) and a global analysis for JMA/GSM. A singular vector method is employed to create initial perturbations. Focusing on TCs in the western North Pacific Ocean and the South China Sea (0°–60°N, 100°E–180°), TEPS runs 4 times a day, initiated at 0000, 0600, 1200, and 1800 UTC with a prediction range of 132 h. The verifications of TEPS during the quasi-operational period from May to December 2007 indicate that the ensemble mean track predictions statistically have better performance as compared with the control (nonperturbed) predictions: the error reduction in the 5-day predictions is 40 km on average. Moreover, it is found that the ensemble spread of tracks is an indicator of position error, indicating that TEPS will be useful in presenting confidence information on TC track predictions. For 2008 when TEPS was in operational use, however, it was also found that the ensemble mean was significantly worse than the deterministic model (JMA/GSM) out to 84 h.

Download Full-text

Predictability of Frontal Waves and Cyclones

Weather and Forecasting ◽

10.1175/waf-d-15-0039.1 ◽

2015 ◽

Vol 30 (5) ◽

pp. 1291-1302 ◽

Cited By ~ 5

Author(s):

Thomas H. A. Frame ◽

John Methven ◽

Nigel M. Roberts ◽

Helen A. Titley

Keyword(s):

Lead Time ◽

Spatial Scales ◽

Statistical Properties ◽

Forecast Skill ◽

Ensemble Prediction ◽

Prediction System ◽

Number Density ◽

Ensemble Prediction System ◽

Probability Forecasts ◽

Stochastic Physics

Abstract The statistical properties and skill in predictions of objectively identified and tracked cyclonic features (frontal waves and cyclones) are examined in the 15-day version of the Met Office Global and Regional Ensemble Prediction System (MOGREPS-15). The number density of cyclonic features is found to decline with increasing lead time, with analysis fields containing weak features that are not sustained past the first day of the forecast. This loss of cyclonic features is associated with a decline in area-averaged enstrophy with increasing lead time. Both feature number density and area-averaged enstrophy saturate by around 7 days into the forecast. It is found that the feature number density and area-averaged enstrophy of forecasts produced using model versions that include stochastic energy backscatter saturate at higher values than forecasts produced without stochastic physics. The ability of MOGREPS-15 to predict the locations of cyclonic features of different strengths is evaluated at different spatial scales by examining the Brier skill (relative to the analysis climatology) of strike probability forecasts: the probability that a cyclonic feature center is located within a specified radius. The radius at which skill is maximized increases with lead time from 650 km at 12 h to 950 km at 7 days. The skill is greatest for the most intense features. Forecast skill remains above zero at these scales out to 14 days for the most intense cyclonic features, but only out to 8 days when all features are included irrespective of intensity.

Download Full-text

Verification of a short-range ensemble precipitation prediction system over Iberia

Advances in Geosciences ◽

10.5194/adgeo-25-55-2010 ◽

2010 ◽

Vol 25 ◽

pp. 55-63 ◽

Cited By ~ 3

Author(s):

D. Santos-Muñoz ◽

M. L. Martin ◽

A. Morata ◽

F. Valero ◽

A. Pascual

Keyword(s):

Short Range ◽

Initial Conditions ◽

Ensemble Prediction ◽

Positive Contribution ◽

Prediction System ◽

Good Reliability ◽

Ensemble Prediction System ◽

Probabilistic Forecasts ◽

Ensemble Mean ◽

The Mean

Abstract. The purpose of this paper is the verification of a short-range ensemble prediction system (SREPS) built with five different model physical process parameterization schemes and two different initial conditions from global models, allowing to construct several versions of the non-hydrostatic mesoscale MM5 model for a 1-month period of October 2006. From the SREPS, flow-dependent probabilistic forecasts are provided by means of predictive probability distributions over the Iberian Peninsula down to 10-km grid spacing. In order to carry out the verification, 25 km grid of observational precipitation records over Spain from the Spanish Climatic Network has been used to evaluate the ensemble accuracy together with the mean model performance and forecast variability by means of comparisons between such records and the ensemble forecasts. This verification has been carried out upscaling the 10 km probabilistic forecast to the observational data grid. Temporal evolution of precipitation forecasts for spatial averaged ensemble members and the ensemble mean is shown, illustrating the consistency of the SREPS. Such evolutions summarize the SREPS information, showing each of the members as well as the ensemble mean evolutions. The Talagrand diagram derived from the SREPS results shows underdispersion which indicates some bias behaviour. The Relative Operating Characteristic (ROC) curve shows a very outstanding area, indicating potential usefulness of the forecasting system. The forecast probability and the mean observed frequency present good agreement with the SREPS results close to the no-skill line. Because the probability has a good reliability and a positive contribution to the brier skill score, a positive value of this skill is obtained. Moreover, the probabilistic meteogram of the spatial daily mean precipitation values shows the range of forecast values, providing discrete probability information in different quantile intervals. The epsgram shows different daily distributions, indicating the predictability of each day.

Download Full-text