scholarly journals Regional Differences in the Prediction of Extratropical Cyclones by the ECMWF Ensemble Prediction System

2009 ◽  
Vol 137 (3) ◽  
pp. 893-911 ◽  
Author(s):  
Lizzie S. R. Froude
Keyword(s):  
Regional Differences ◽  
Propagation Speed ◽  
Ensemble Prediction ◽  
Extratropical Cyclones ◽  
Spatial Distributions ◽  
Prediction System ◽  
Ensemble Spread ◽  
Ensemble Prediction System ◽  
Mean Error ◽  
Ensemble Mean

Abstract A regional study of the prediction of extratropical cyclones by the European Centre for Medium-Range Weather Forecasts (ECMWF) Ensemble Prediction System (EPS) has been performed. An objective feature-tracking method has been used to identify and track the cyclones along the forecast trajectories. Forecast error statistics have then been produced for the position, intensity, and propagation speed of the storms. In previous work, data limitations meant it was only possible to present the diagnostics for the entire Northern Hemisphere (NH) or Southern Hemisphere. A larger data sample has allowed the diagnostics to be computed separately for smaller regions around the globe and has made it possible to explore the regional differences in the prediction of storms by the EPS. Results show that in the NH there is a larger ensemble mean error in the position of storms over the Atlantic Ocean. Further analysis revealed that this is mainly due to errors in the prediction of storm propagation speed rather than in direction. Forecast storms propagate too slowly in all regions, but the bias is about 2 times as large in the NH Atlantic region. The results show that storm intensity is generally overpredicted over the ocean and underpredicted over the land and that the absolute error in intensity is larger over the ocean than over the land. In the NH, large errors occur in the prediction of the intensity of storms that originate as tropical cyclones but then move into the extratropics. The ensemble is underdispersive for the intensity of cyclones (i.e., the spread is smaller than the mean error) in all regions. The spatial patterns of the ensemble mean error and ensemble spread are very different for the intensity of cyclones. Spatial distributions of the ensemble mean error suggest that large errors occur during the growth phase of storm development, but this is not indicated by the spatial distributions of the ensemble spread. In the NH there are further differences. First, the large errors in the prediction of the intensity of cyclones that originate in the tropics are not indicated by the spread. Second, the ensemble mean error is larger over the Pacific Ocean than over the Atlantic, whereas the opposite is true for the spread. The use of a storm-tracking approach, to both weather forecasters and developers of forecast systems, is also discussed.

scholarly journals TIGGE: Comparison of the Prediction of Northern Hemisphere Extratropical Cyclones by Different Ensemble Prediction Systems

2010 ◽  
Vol 25 (3) ◽  
pp. 819-836 ◽  
Author(s):  
Lizzie S. R. Froude
Keyword(s):  
Northern Hemisphere ◽  
Propagation Speed ◽  
Research Programme ◽  
Ensemble Prediction ◽  
Extratropical Cyclones ◽  
Ensemble Prediction System ◽  
Cyclone Intensity ◽  
Ensemble Mean ◽  
Environmental Prediction ◽  
And Control

Abstract The Observing System Research and Predictability Experiment (THORPEX) Interactive Grand Global Ensemble (TIGGE) is a World Weather Research Programme project. One of its main objectives is to enhance collaboration on the development of ensemble prediction between operational centers and universities by increasing the availability of ensemble prediction system (EPS) data for research. This study analyzes the prediction of Northern Hemisphere extratropical cyclones by nine different EPSs archived as part of the TIGGE project for the 6-month time period of 1 February 2008–31 July 2008, which included a sample of 774 cyclones. An objective feature tracking method has been used to identify and track the cyclones along the forecast trajectories. Forecast verification statistics have then been produced [using the European Centre for Medium-Range Weather Forecasts (ECMWF) operational analysis as the truth] for cyclone position, intensity, and propagation speed, showing large differences between the different EPSs. The results show that the ECMWF ensemble mean and control have the highest level of skill for all cyclone properties. The Japanese Meteorological Administration (JMA), the National Centers for Environmental Prediction (NCEP), the Met Office (UKMO), and the Canadian Meteorological Centre (CMC) have 1 day less skill for the position of cyclones throughout the forecast range. The relative performance of the different EPSs remains the same for cyclone intensity except for NCEP, which has larger errors than for position. NCEP, the Centro de Previsão de Tempo e Estudos Climáticos (CPTEC), and the Australian Bureau of Meteorology (BoM) all have faster intensity error growth in the earlier part of the forecast. They are also very underdispersive and significantly underpredict intensities, perhaps due to the comparatively low spatial resolutions of these EPSs not being able to accurately model the tilted structure essential to cyclone growth and decay. There is very little difference between the levels of skill of the ensemble mean and control for cyclone position, but the ensemble mean provides an advantage over the control for all EPSs except CPTEC in cyclone intensity and there is an advantage for propagation speed for all EPSs. ECMWF and JMA have an excellent spread–skill relationship for cyclone position. The EPSs are all much more underdispersive for cyclone intensity and propagation speed than for position, with ECMWF and CMC performing best for intensity and CMC performing best for propagation speed. ECMWF is the only EPS to consistently overpredict cyclone intensity, although the bias is small. BoM, NCEP, UKMO, and CPTEC significantly underpredict intensity and, interestingly, all the EPSs underpredict the propagation speed, that is, the cyclones move too slowly on average in all EPSs.

scholarly journals Why Should Ensemble Spread Match the RMSE of the Ensemble Mean?

2014 ◽  
Vol 15 (4) ◽  
pp. 1708-1713 ◽  
Author(s):  
V. Fortin ◽  
M. Abaza ◽  
F. Anctil ◽  
R. Turcotte
Keyword(s):  
Good Practice ◽  
Ensemble Prediction ◽  
Square Root ◽  
Prediction System ◽  
Mean Square ◽  
Ensemble Spread ◽  
Ensemble Prediction System ◽  
Correct Equation ◽  
Ensemble Mean ◽  
The Impact

Abstract When evaluating the reliability of an ensemble prediction system, it is common to compare the root-mean-square error of the ensemble mean to the average ensemble spread. While this is indeed good practice, two different and inconsistent methodologies have been used over the last few years in the meteorology and hydrology literature to compute the average ensemble spread. In some cases, the square root of average ensemble variance is used, and in other cases, the average of ensemble standard deviation is computed instead. The second option is incorrect. To avoid the perpetuation of practices that are not supported by probability theory, the correct equation for computing the average ensemble spread is obtained and the impact of using the wrong equation is illustrated.

scholarly journals The Prediction of Extratropical Storm Tracks by the ECMWF and NCEP Ensemble Prediction Systems

2007 ◽  
Vol 135 (7) ◽  
pp. 2545-2567 ◽  
Author(s):  
Lizzie S. R. Froude ◽  
Lennart Bengtsson ◽  
Kevin I. Hodges
Keyword(s):  
Propagation Speed ◽  
Ensemble Member ◽  
Ensemble Prediction ◽  
Extratropical Cyclones ◽  
Mean Error ◽  
Predictive Skill ◽  
Ensemble Mean ◽  
The Difference ◽  
Prediction Systems ◽  
Environmental Prediction

Abstract The prediction of extratropical cyclones by the European Centre for Medium-Range Weather Forecasts (ECMWF) and the National Centers for Environmental Prediction (NCEP) ensemble prediction systems (EPSs) has been investigated using an objective feature tracking methodology to identify and track the cyclones along the forecast trajectories. Overall the results show that the ECMWF EPS has a slightly higher level of skill than the NCEP EPS in the Northern Hemisphere (NH). However in the Southern Hemisphere (SH), NCEP has higher predictive skill than ECMWF for the intensity of the cyclones. The results from both EPSs indicate a higher level of predictive skill for the position of extratropical cyclones than their intensity and show that there is a larger spread in intensity than position. Further analysis shows that the predicted propagation speed of cyclones is generally too slow for the ECMWF EPS and shows a slight bias for the intensity of the cyclones to be overpredicted. This is also true for the NCEP EPS in the SH. For the NCEP EPS in the NH the intensity of the cyclones is underpredicted. There is small bias in both the EPS for the cyclones to be displaced toward the poles. For each ensemble forecast of each cyclone, the predictive skill of the ensemble member that best predicts the cyclone’s position and intensity was computed. The results are very encouraging showing that the predictive skill of the best ensemble member is significantly higher than that of the control forecast in terms of both the position and intensity of the cyclones. The prediction of cyclones before they are identified as 850-hPa vorticity centers in the analysis cycle was also considered. It is shown that an indication of extratropical cyclones can be given by at least 1 ensemble member 7 days before they are identified in the analysis. Further analysis of the ECMWF EPS shows that the ensemble mean has a higher level of skill than the control forecast, particularly for the intensity of the cyclones, from day 3 of the forecast. There is a higher level of skill in the NH than the SH and the spread in the SH is correspondingly larger. The difference between the ensemble mean error and spread is very small for the position of the cyclones, but the spread of the ensemble is smaller than the ensemble mean error for the intensity of the cyclones in both hemispheres. Results also show that the ECMWF control forecast has ½ to 1 day more skill than the perturbed members, for both the position and intensity of the cyclones, throughout the forecast.

A Unified Scheme of Stochastic Physics and Bias Correction in an Ensemble Model to Reduce Both Random and Systematic Errors

2019 ◽  
Vol 34 (6) ◽  
pp. 1675-1691 ◽  
Author(s):  
Yu Xia ◽  
Jing Chen ◽  
Jun Du ◽  
Xiefei Zhi ◽  
Jingzhuo Wang ◽  
...  
Keyword(s):  
Bias Correction ◽  
Weather Prediction ◽  
Systematic Errors ◽  
Ensemble Prediction ◽  
Ensemble Model ◽  
Prediction System ◽  
Ensemble Prediction System ◽  
Probabilistic Forecasts ◽  
Ensemble Mean ◽  
Stochastic Physics

Abstract This study experimented with a unified scheme of stochastic physics and bias correction within a regional ensemble model [Global and Regional Assimilation and Prediction System–Regional Ensemble Prediction System (GRAPES-REPS)]. It is intended to improve ensemble prediction skill by reducing both random and systematic errors at the same time. Three experiments were performed on top of GRAPES-REPS. The first experiment adds only the stochastic physics. The second experiment adds only the bias correction scheme. The third experiment adds both the stochastic physics and bias correction. The experimental period is one month from 1 to 31 July 2015 over the China domain. Using 850-hPa temperature as an example, the study reveals the following: 1) the stochastic physics can effectively increase the ensemble spread, while the bias correction cannot. Therefore, ensemble averaging of the stochastic physics runs can reduce more random error than the bias correction runs. 2) Bias correction can significantly reduce systematic error, while the stochastic physics cannot. As a result, the bias correction greatly improved the quality of ensemble mean forecasts but the stochastic physics did not. 3) The unified scheme can greatly reduce both random and systematic errors at the same time and performed the best of the three experiments. These results were further confirmed by verification of the ensemble mean, spread, and probabilistic forecasts of many other atmospheric fields for both upper air and the surface, including precipitation. Based on this study, we recommend that operational numerical weather prediction centers adopt this unified scheme approach in ensemble models to achieve the best forecasts.

scholarly journals Sensitivity of Global Ensemble Forecasts to the Initial Ensemble Mean and Perturbations: Comparison of EnKF, Singular Vector, and 4D-Var Approaches

2010 ◽  
Vol 138 (10) ◽  
pp. 3886-3904 ◽  
Author(s):  
Mark Buehner ◽  
Ahmed Mahidjiba
Keyword(s):  
Ensemble Prediction ◽  
Boreal Summer ◽  
Prediction System ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
Var Analysis ◽  
Initial Ensemble ◽  
Ensemble Mean ◽  
The Tropics ◽  
The Impact

Abstract This study examines the sensitivity of global ensemble forecasts to the use of different approaches for specifying both the initial ensemble mean and perturbations. The current operational ensemble prediction system of the Meteorological Service of Canada uses the ensemble Kalman filter (EnKF) to define both the ensemble mean and perturbations. To evaluate the impact of different approaches for obtaining the initial ensemble perturbations, the operational EnKF approach is compared with using either no initial perturbations or perturbations obtained using singular vectors (SVs). The SVs are computed using the (dry) total-energy norm with a 48-h optimization time interval. Random linear combinations of 60 SVs are computed for each of three regions. Next, the impact of replacing the initial ensemble mean, currently the EnKF ensemble mean analysis, with the higher-resolution operational four-dimensional variational data assimilation (4D-Var) analysis is evaluated. For this comparison, perturbations are provided by the EnKF. All experiments are performed over two-month periods during both the boreal summer and winter using a system very similar to the global ensemble prediction system that became operational on 10 July 2007. Relative to the operational configuration that relies on the EnKF, the use of SVs to compute initial perturbations produces small, but statistically significant differences in probabilistic forecast scores in favor of the EnKF both in the tropics and, for a limited set of forecast lead times, in the summer hemisphere extratropics, whereas the results are very similar in the winter hemisphere extratropics. Both approaches lead to significantly better ensemble forecasts than with no initial perturbations, though results are quite similar in the tropics when using SVs and no perturbations. The use of an initial-time norm that does not include information on analysis uncertainty and the lack of linearized moist processes in the calculation of the SVs are two factors that limit the quality of the resulting SV-based ensemble forecasts. Relative to the operational configuration, use of the 4D-Var analysis to specify the initial ensemble mean results in improved probabilistic forecast scores during the boreal summer period in the southern extratropics and tropics, but a near-neutral impact otherwise.

A large ensemble decadal prediction system with MPI-ESM

2021 ◽  
Author(s):  
Sebastian Brune ◽  
Vimal Koul ◽  
David Marcolino Nielsen ◽  
Laura Hövel ◽  
Holger Pohlmann ◽  
...  
Keyword(s):  
North Atlantic ◽  
Prediction Skill ◽  
Ensemble Prediction ◽  
Prediction System ◽  
Decadal Prediction ◽  
Ensemble Size ◽  
Large Ensemble ◽  
Ensemble Prediction System ◽  
Atlantic Sector ◽  
Ensemble Mean

<p>Current state-of-the-art decadal ensemble prediction systems are run with an ensemble size of 10 to 40 members, their retrospective forecasts of the past are used to assess the system's prediction skill. Here, we present an attempt for a large ensemble decadal prediction system for the time period 1960-today, with an ensemble size of 80 members, based on the low resolution version of the Max Planck Institute Earth system model (MPI-ESM-LR). The ensemble is forced with CMIP6 conditions and initialized every year in November through a weakly coupled assimilation using atmospheric reanalyses via nudging and observed oceanic temperature and salinity profiles via a 16-member ensemble Kalman filter. To generate ensemble members beyond 16, we use additional physical perturbations at stratospheric height. The analysis of our large ensemble prediction system presented here aims for answering two questions: (1) How does the ensemble mean deterministic prediction skill for global and North Atlantic key climate indices change with ensemble size? (2) How well may the 80-member ensemble serve as a basis for a robust statistical analysis of probabilities of extremes in the North Atlantic sector? Preliminary results for global and regional air surface temperature show that in terms of ensemble mean ACC and full ensemble CPRSS with reference data, the 80-member ensemble leads to similar prediction skill as the 16-member ensemble. This indicates that the additional ensemble members may lead to a better sampling of the distribution of model trajectories, paving the way for a more robust statistical probabilistic analysis.</p>

Ensemble mean forecast skill and applications with the T213 ensemble prediction system

2016 ◽  
Vol 33 (11) ◽  
pp. 1297-1305
Author(s):  
Sijia Li ◽  
Yuan Wang ◽  
Huiling Yuan ◽  
Jinjie Song ◽  
Xin Xu
Keyword(s):  
Forecast Skill ◽  
Ensemble Prediction ◽  
Prediction System ◽  
Ensemble Prediction System ◽  
Ensemble Mean

Comparison of a 51-Member Low-Resolution (TL399L62) Ensemble with a 6-Member High-Resolution (TL799L91) Lagged-Forecast Ensemble

2008 ◽  
Vol 136 (9) ◽  
pp. 3343-3362 ◽  
Author(s):  
Roberto Buizza
Keyword(s):  
Forecast Error ◽  
Ensemble Prediction ◽  
Individual Member ◽  
Ensemble Spread ◽  
Error Statistics ◽  
Ensemble Prediction System ◽  
Optimal Weights ◽  
Probabilistic Forecasts ◽  
Ensemble Mean ◽  
Medium Range

Abstract The 51-member TL399L62 ECMWF ensemble prediction system (EPS51) is compared with a lagged ensemble system based on the six most recent ECMWF TL799L91 forecasts (LAG6). The EPS51 and LAG6 systems are compared to two 6-member ensembles with a “weighted” ensemble-mean: EPS6wEM and LAG6wEM. EPS6wEM includes six members of EPS51 and has the ensemble mean constructed giving optimal weights to its members, while LAG6wEM includes the LAG6 six members and has the ensemble mean constructed giving optimal weights to its members. In these weighted ensembles, the optimal weights are based on 50-day forecast error statistics of each individual member (in EPS51 and LAG6 the ensemble mean is constructed giving the same weight to each individual member). The EPS51, LAG6, EPS6wEM, and LAG6wEM ensembles are compared for a 7-month period (from 1 April to 30 October 2006—213 cases) and for two of the most severe storms that hit the Scandinavian countries since 1969. The study shows that EPS51 has the best-tuned ensemble spread, and provides the best probabilistic forecasts, with differences in predictability between EPS51 and LAG6 or LAG6wEM probabilistic forecasts of geopotential height anomalies of up to 24 h. In terms of ensemble mean, EPS51 gives the best forecast from forecast day 4, but before forecast day 4 LAG6wEM provides a slightly better forecast, with differences in predictability smaller than 2 h up to forecast day 6, and of about 6 h afterward. The comparison also shows that a larger ensemble size is more important in the medium range rather than in the short range. Overall, these results indicate that if the aim of ensemble prediction is to generate not only a single (most likely) scenario but also a probabilistic forecast, than EPS51 has a higher skill than the lagged ensemble system based on LAG6 or LAG6wEM.

scholarly journals Sensitivity of Ensemble Forecast Verification to Model Bias

2018 ◽  
Vol 146 (3) ◽  
pp. 781-796 ◽  
Author(s):  
Jingzhuo Wang ◽  
Jing Chen ◽  
Jun Du ◽  
Yutao Zhang ◽  
Yu Xia ◽  
...  
Keyword(s):  
Bias Correction ◽  
Ensemble Prediction ◽  
Prediction System ◽  
Model Bias ◽  
Ensemble Spread ◽  
Ensemble Prediction System ◽  
Forecast Bias ◽  
Probabilistic Forecasts ◽  
Before And After

This study demonstrates how model bias can adversely affect the quality assessment of an ensemble prediction system (EPS) by verification metrics. A regional EPS [Global and Regional Assimilation and Prediction Enhanced System-Regional Ensemble Prediction System (GRAPES-REPS)] was verified over a period of one month over China. Three variables (500-hPa and 2-m temperatures, and 250-hPa wind) are selected to represent “strong” and “weak” bias situations. Ensemble spread and probabilistic forecasts are compared before and after a bias correction. The results show that the conclusions drawn from ensemble verification about the EPS are dramatically different with or without model bias. This is true for both ensemble spread and probabilistic forecasts. The GRAPES-REPS is severely underdispersive before the bias correction but becomes calibrated afterward, although the improvement in the spread’s spatial structure is much less; the spread–skill relation is also improved. The probabilities become much sharper and almost perfectly reliable after the bias is removed. Therefore, it is necessary to remove forecast biases before an EPS can be accurately evaluated since an EPS deals only with random error but not systematic error. Only when an EPS has no or little forecast bias, can ensemble verification metrics reliably reveal the true quality of an EPS without removing forecast bias first. An implication is that EPS developers should not be expected to introduce methods to dramatically increase ensemble spread (either by perturbation method or statistical calibration) to achieve reliability. Instead, the preferred solution is to reduce model bias through prediction system developments and to focus on the quality of spread (not the quantity of spread). Forecast products should also be produced from the debiased but not the raw ensemble.

scholarly journals Typhoon Ensemble Prediction System Developed at the Japan Meteorological Agency

2009 ◽  
Vol 137 (8) ◽  
pp. 2592-2604 ◽  
Author(s):  
Munehiko Yamaguchi ◽  
Ryota Sakai ◽  
Masayuki Kyoda ◽  
Takuya Komori ◽  
Takashi Kadowaki
Keyword(s):  
Deterministic Model ◽  
Global Analysis ◽  
North Pacific Ocean ◽  
Japan Meteorological Agency ◽  
Ensemble Prediction ◽  
Prediction System ◽  
Global Spectral Model ◽  
Vector Method ◽  
Ensemble Prediction System ◽  
Ensemble Mean

Abstract The Japan Meteorological Agency (JMA) Typhoon Ensemble Prediction System (TEPS) and its performance are described. In February 2008, JMA started an operation of TEPS that was designed for providing skillful tropical cyclone (TC) track predictions in both deterministic and probabilistic ways. TEPS consists of 1 nonperturbed prediction and 10 perturbed predictions based on the lower-resolution version (TL319L60) of the JMA Global Spectral Model (GSM; TL959L60) and a global analysis for JMA/GSM. A singular vector method is employed to create initial perturbations. Focusing on TCs in the western North Pacific Ocean and the South China Sea (0°–60°N, 100°E–180°), TEPS runs 4 times a day, initiated at 0000, 0600, 1200, and 1800 UTC with a prediction range of 132 h. The verifications of TEPS during the quasi-operational period from May to December 2007 indicate that the ensemble mean track predictions statistically have better performance as compared with the control (nonperturbed) predictions: the error reduction in the 5-day predictions is 40 km on average. Moreover, it is found that the ensemble spread of tracks is an indicator of position error, indicating that TEPS will be useful in presenting confidence information on TC track predictions. For 2008 when TEPS was in operational use, however, it was also found that the ensemble mean was significantly worse than the deterministic model (JMA/GSM) out to 84 h.

Sign in / Sign up

Export Citation Format

Share Document