scholarly journals An Assessment of the Performance of the Operational Global Ensemble Forecast Systems in Predicting the Forecast Uncertainty

2017 ◽  
Vol 32 (1) ◽  
pp. 149-164 ◽  
Author(s):  
Carlee F. Loeser ◽  
Michael A. Herrera ◽  
Istvan Szunyogh
Keyword(s):  
Large Scale ◽  
Forecast Error ◽  
Ensemble Forecast ◽  
Forecast Uncertainty ◽  
Ensemble Forecasts ◽  
Atmospheric Flow ◽  
Weather Forecasts ◽  
Using Data ◽  
Ensemble Systems ◽  
First Time

Abstract This study investigates the efficiency of the major operational global ensemble forecast systems of the world in capturing the spatiotemporal evolution of the forecast uncertainty. Using data from 2015, it updates the results of an earlier study based on data from 2012. It also tests, for the first time on operational ensemble data, two quantitative relationships to aid in the interpretation of the raw ensemble forecasts. One of these relationships provides a flow-dependent prediction of the reliability of the ensemble in capturing the uncertain forecast features, while the other predicts the 95th percentile value of the magnitude of the forecast error. It is found that, except for the system of the Met Office, the main characteristics of the ensemble forecast systems have changed little between 2012 and 2015. The performance of the UKMO ensemble improved in predicting the overall magnitude of the uncertainty, but its ability to predict the dominant uncertain forecast features was degraded. A common serious limitation of the ensemble systems remains that they all have major difficulties with predicting the large-scale atmospheric flow in the long (longer than 10 days) forecast range. These difficulties are due to the inability of the ensemble members to maintain large-scale waves in the forecasts, which presents a stumbling block in the way of extending the skill of numerical weather forecasts to the subseasonal range. The two tested predictive relationships were found to provide highly accurate predictions of the flow-dependent reliability of the ensemble predictions and the 95th percentile value of the magnitude of the forecast error for the operational ensemble forecast systems.

scholarly journals Sensitivity of Forecast Uncertainty to Different Microphysics Schemes within a Convection-Allowing Ensemble during SoWMEX-IOP8

2021 ◽  
Author(s):  
Chin-Hung Chen ◽  
Kao-Shen Chung ◽  
Shu-Chih Yang ◽  
Li-Hsin Chen ◽  
Pay-Liam Lin ◽  
...  
Keyword(s):  
Large Scale ◽  
Initial Conditions ◽  
Forecast Error ◽  
Convective System ◽  
State Variables ◽  
Forecast Uncertainty ◽  
Ensemble Forecasts ◽  
Weather System ◽  
Microphysics Scheme ◽  
Double Moment

AbstractA mesoscale convective system that occurred in southwestern Taiwan on 15 June 2008 is simulated using convection-allowing ensemble forecasts to investigate the forecast uncertainty associated with four microphysics schemes—the Goddard Cumulus Ensemble (GCE), Morrison (MOR), WRF single-moment 6-class (WSM6), and WRF double-moment 6-class (WDM6) schemes. First, the essential features of the convective structure, hydrometeor distribution, and microphysical tendencies for the different microphysics schemes are presented through deterministic forecasts. Second, ensemble forecasts with the same initial conditions are employed to estimate the forecast uncertainty produced by the different ensembles with the fixed microphysics scheme. GCE has the largest spread in most state variables due to its most efficient phase conversion between water species. By contrast, MOR results in the least spread. WSM6 and WDM6 have similar vertical spread structures due to their similar ice-phase formulae. However, WDM6 produces more ensemble spread than WSM6 does below the melting layer, resulting from its double-moment treatment of warm rain processes. The model simulations with the four microphysics schemes demonstrate upscale error growth through spectrum analysis of the root-mean difference total energy (RMDTE). The RMDTE results reveal that the GCE and WDM6 schemes are more sensitive to initial condition uncertainty, whereas the MOR and WSM6 schemes are relatively less sensitive to that for this event. Overall, the diabatic heating–cooling processes connect the convective-scale cloud microphysical processes to the large-scale dynamical and thermodynamical fields, and they significantly affect the forecast error signatures in the multiscale weather system.

scholarly journals Decomposition of a New Proper Score for Verification of Ensemble Forecasts

2015 ◽  
Vol 143 (5) ◽  
pp. 1517-1532 ◽  
Author(s):  
H. M. Christensen
Keyword(s):  
Standard Deviation ◽  
Ensemble Forecast ◽  
Ensemble Prediction ◽  
Brier Score ◽  
Continuous Variables ◽  
Ensemble Forecasts ◽  
Atmospheric Flow ◽  
Ensemble Prediction System ◽  
Weather Forecasts ◽  
Medium Range

Abstract A new proper score, the error-spread score (ES), has recently been proposed for evaluation of ensemble forecasts of continuous variables. The ES is formulated with respect to the moments of the ensemble forecast. It is particularly sensitive to evaluating how well an ensemble forecast represents uncertainty: is the probabilistic forecast well calibrated? In this paper, it is shown that the ES can be decomposed into its reliability, resolution, and uncertainty components in a similar way to the Brier score. The first term evaluates the reliability of the forecast standard deviation and skewness, rewarding systems where the forecast moments reliably indicate the properties of the verification. The second term evaluates the resolution of the forecast standard deviation and skewness, and rewards systems where the forecast moments vary from the climatological moments according to the predictability of the atmospheric flow. The uncertainty term depends only on the observed error distribution and is independent of the forecast standard deviation or skewness. The decomposition was demonstrated using forecasts made with the European Centre for Medium-Range Weather Forecasts ensemble prediction system, and was able to identify the source of the skill in the forecasts at different latitudes.

scholarly journals More on the Scale Dependence of the Predictability of Precipitation Patterns: Extension to the 2009–13 CAPS Spring Experiment Ensemble Forecasts

2017 ◽  
Vol 145 (9) ◽  
pp. 3625-3646 ◽  
Author(s):  
Madalina Surcel ◽  
Isztar Zawadzki ◽  
M. K. Yau ◽  
Ming Xue ◽  
Fanyou Kong
Keyword(s):  
Large Scale ◽  
Forecast Error ◽  
Quasi Equilibrium ◽  
Relative Role ◽  
Ensemble Forecasts ◽  
Adjustment Time ◽  
Convective Adjustment ◽  
Microphysical Parameterization ◽  
Microphysical Parameterization Scheme

This paper analyzes the scale and case dependence of the predictability of precipitation in the Storm-Scale Ensemble Forecast (SSEF) system run by the Center for Analysis and Prediction of Storms (CAPS) during the NOAA Hazardous Weather Testbed Spring Experiments of 2008–13. The effect of different types of ensemble perturbation methodologies is quantified as a function of spatial scale. It is found that uncertainties in the large-scale initial and boundary conditions and in the model microphysical parameterization scheme can result in the loss of predictability at scales smaller than 200 km after 24 h. Also, these uncertainties account for most of the forecast error. Other types of ensemble perturbation methodologies were not found to be as important for the quantitative precipitation forecasts (QPFs). The case dependences of predictability and of the sensitivity to the ensemble perturbation methodology were also analyzed. Events were characterized in terms of the extent of the precipitation coverage and of the convective-adjustment time scale [Formula: see text], an indicator of whether convection is in equilibrium with the large-scale forcing. It was found that events characterized by widespread precipitation and small [Formula: see text] values (representative of quasi-equilibrium convection) were usually more predictable than nonequilibrium cases. No significant statistical relationship was found between the relative role of different perturbation methodologies and precipitation coverage or [Formula: see text].

scholarly journals Diagnosis and Optimization of Ensemble Forecasts

2008 ◽  
Vol 136 (3) ◽  
pp. 1054-1074 ◽  
Author(s):  
Tomislava Vukicevic ◽  
Isidora Jankov ◽  
John McGinley
Keyword(s):  
Initial Conditions ◽  
Forecast Error ◽  
Absolute Error ◽  
Ensemble Forecast ◽  
Stochastic Representation ◽  
Ensemble Forecasts ◽  
Ensemble Technique ◽  
Consensus Forecasts ◽  
Almost All ◽  
The Impact

Abstract In the current study, a technique that offers a way to evaluate ensemble forecast uncertainties produced either by initial conditions or different model versions, or both, is presented. The technique consists of first diagnosing the performance of the forecast ensemble and then optimizing the ensemble forecast using results of the diagnosis. The technique is based on the explicit evaluation of probabilities that are associated with the Gaussian stochastic representation of the weather analysis and forecast. It combines an ensemble technique for evaluating the analysis error covariance and the standard Monte Carlo approach for computing samples from a known Gaussian distribution. The technique was demonstrated in a tutorial manner on two relatively simple examples to illustrate the impact of ensemble characteristics including ensemble size, various observation strategies, and configurations including different model versions and varying initial conditions. In addition, the authors assessed improvements in the consensus forecasts gained by optimal weighting of the ensemble members based on time-varying, prior-probabilistic skill measures. The results with different observation configurations indicate that, as observations become denser, there is a need for larger-sized ensembles and/or more accuracy among individual members for the ensemble forecast to exhibit prediction skill. The main conclusions relative to ensembles built up with different physics configurations were, first, that almost all members typically exhibited some skill at some point in the model run, suggesting that all should be retained to acquire the best consensus forecast; and, second, that the normalized probability metric can be used to determine what sets of weights or physics configurations are performing best. A comparison of forecasts derived from a simple ensemble mean to forecasts from a mean developed from variably weighting the ensemble members based on prior performance by the probabilistic measure showed that the latter had substantially reduced mean absolute error. The study also indicates that a weighting scheme that utilized more prior cycles showed additional reduction in forecast error.

scholarly journals Spatial Postprocessing of Ensemble Forecasts for Temperature Using Nonhomogeneous Gaussian Regression

2015 ◽  
Vol 143 (3) ◽  
pp. 955-971 ◽  
Author(s):  
Kira Feldmann ◽  
Michael Scheuerer ◽  
Thordis L. Thorarinsdottir
Keyword(s):  
Bayesian Model Averaging ◽  
Forecast Error ◽  
Gaussian Random Field ◽  
Rank Correlation ◽  
Model Averaging ◽  
Small Scale ◽  
Ensemble Forecasts ◽  
Weather Forecasts ◽  
Scale Modelling ◽  
Gaussian Regression

Abstract Statistical postprocessing techniques are commonly used to improve the skill of ensembles from numerical weather forecasts. This paper considers spatial extensions of the well-established nonhomogeneous Gaussian regression (NGR) postprocessing technique for surface temperature and a recent modification thereof in which the local climatology is included in the regression model to permit locally adaptive postprocessing. In a comparative study employing 21-h forecasts from the Consortium for Small Scale Modelling ensemble predictive system over Germany (COSMO-DE), two approaches for modeling spatial forecast error correlations are considered: a parametric Gaussian random field model and the ensemble copula coupling (ECC) approach, which utilizes the spatial rank correlation structure of the raw ensemble. Additionally, the NGR methods are compared to both univariate and spatial versions of the ensemble Bayesian model averaging (BMA) postprocessing technique.

The Impact of Initial Condition and Warm Conveyor Belt Forecast Uncertainty on Variability in the Downstream Waveguide in an ECWMF Case Study

2019 ◽  
Vol 147 (11) ◽  
pp. 4071-4089 ◽  
Author(s):  
Jeremy D. Berman ◽  
Ryan D. Torn
Keyword(s):  
Conveyor Belt ◽  
Forecast Uncertainty ◽  
Ensemble Forecasts ◽  
Rossby Wave Breaking ◽  
Weather Forecasts ◽  
The North ◽  
Weather Events ◽  
High Impact Weather ◽  
Sensitivity Method ◽  
The Impact

Abstract Perturbations to the potential vorticity (PV) waveguide, which can result from latent heat release within the warm conveyor belt (WCB) of midlatitude cyclones, can lead to the downstream radiation of Rossby waves, and in turn high-impact weather events. Previous studies have hypothesized that forecast uncertainty associated with diabatic heating in WCBs can result in large downstream forecast variability; however, these studies have not established a direct connection between the two. This study evaluates the potential impact of latent heating variability in the WCB on subsequent downstream forecasts by applying the ensemble-based sensitivity method to European Centre for Medium-Range Weather Forecasts (ECMWF) ensemble forecasts of a cyclogenesis event over the North Atlantic. For this case, ensemble members with a more amplified ridge are associated with greater negative PV advection by the irrotational wind, which is associated with stronger lower-tropospheric southerly moisture transport east of the upstream cyclone in the WCB. This transport is sensitive to the pressure trough to the south of the cyclone along the cold front, which in turn is modulated by earlier differences in the motion of the air masses on either side of the front. The position of the cold air behind the front is modulated by upstream tropopause-based PV anomalies, such that a deeper pressure trough is associated with a more progressive flow pattern, originating from Rossby wave breaking over the North Pacific. Overall, these results suggest that more accurate forecasts of upstream PV anomalies and WCBs may reduce forecast uncertainty in the downstream waveguide.

scholarly journals “Jumpiness” of the ECMWF and Met Office EPS Control and Ensemble-Mean Forecasts

2009 ◽  
Vol 137 (11) ◽  
pp. 3823-3836 ◽  
Author(s):  
Ervin Zsoter ◽  
Roberto Buizza ◽  
David Richardson
Keyword(s):  
Time Series ◽  
Weak Link ◽  
Forecast Error ◽  
Forecast Uncertainty ◽  
Flip Flop ◽  
Lower Error ◽  
Ensemble Mean ◽  
Analysis Error ◽  
Ensemble Systems

Abstract This work investigates the inconsistency between forecasts issued at different times but valid for the same time, and shows that ensemble-mean forecasts are less inconsistent than corresponding control forecasts. The “jumpiness” index, the concepts of different forecast jumps—the “flip,” “flip-flop,” and “flip-flop-flip”—and the inconsistency correlation between time series of inconsistency indices are introduced to measure the consistency/inconsistency of consecutive forecasts. These new measures are used to compare the behavior of the ECMWF and the Met Office control and ensemble-mean forecasts for an 18-month period over Europe. Results indicate that for both the ECMWF and the Met Office ensembles, the ensemble-mean forecast is less inconsistent than the control forecast. However, they also indicate that the ensemble mean follows its corresponding control forecast more closely than the controls (or the ensemble means) of the two ensemble systems following each other, thus suggesting weaknesses in both ensemble systems in the simulation of forecast uncertainty due to model or analysis error. Results also show that there is only a weak link between forecast jumpiness and forecast error (i.e., forecasts with lower inconsistency do not necessarily have, on average, lower error).

scholarly journals Predictability of a Mediterranean Tropical-Like Storm Downstream of the Extratropical Transition of Hurricane Helene (2006)

2013 ◽  
Vol 141 (6) ◽  
pp. 1943-1962 ◽  
Author(s):  
Florian P. Pantillon ◽  
Jean-Pierre Chaboureau ◽  
Patrick J. Mascart ◽  
Christine Lac
Keyword(s):  
Tropical Cyclone ◽  
Lead Time ◽  
Large Scale ◽  
Synoptic Situation ◽  
British Isles ◽  
Extratropical Transition ◽  
Ensemble Forecasts ◽  
Weather Forecasts ◽  
Warm Core ◽  
Medium Range

Abstract The extratropical transition (ET) of a tropical cyclone is known as a source of forecast uncertainty that can propagate far downstream. The present study focuses on the predictability of a Mediterranean tropical-like storm (Medicane) on 26 September 2006 downstream of the ET of Hurricane Helene from 22 to 25 September. While the development of the Medicane was missed in the deterministic forecasts from the European Centre for Medium-Range Weather Forecasts (ECMWF) initialized before and during ET, it was contained in the ECMWF ensemble forecasts in more than 10% of the 50 members up to 108-h lead time. The 200 ensemble members initialized at 0000 UTC from 20 to 23 September were clustered into two nearly equiprobable scenarios after the synoptic situation over the Mediterranean. In the first and verifying scenario, Helene was steered northeastward by an upstream trough during ET and contributed to the building of a downstream ridge. A trough elongated farther downstream toward Italy and enabled the development of the Medicane in 9 of 102 members. In the second and nonverifying scenario, Helene turned southeastward during ET and the downstream ridge building was reduced. A large-scale low over the British Isles dominated the circulation in Europe and only 1 of 98 members forecasted the Medicane. The two scenarios resulted from a different phasing between Helene and the upstream trough. Sensitivity experiments performed with the Méso-NH model further revealed that initial perturbations targeted on Helene and the upstream trough were sufficient in forecasting the warm-core Medicane at 84- and 108-h lead time.

scholarly journals Hidden Error Variance Theory. Part II: An Instrument That Reveals Hidden Error Variance Distributions from Ensemble Forecasts and Observations

2013 ◽  
Vol 141 (5) ◽  
pp. 1469-1483 ◽  
Author(s):  
Craig H. Bishop ◽  
Elizabeth A. Satterfield ◽  
Kevin T. Shanley
Keyword(s):  
Forecast Error ◽  
Error Variance ◽  
True Error ◽  
Ensemble Forecasts ◽  
Optimal Weights ◽  
Long Time ◽  
Forecast Error Variance ◽  
Hidden Parameters ◽  
Variance Parameters ◽  
Using Data

Abstract In Part I of this study, a model of the distribution of true error variances given an ensemble variance is shown to be defined by six parameters that also determine the optimal weights for the static and flow-dependent parts of hybrid error variance models. Two of the six parameters (the climatological mean of forecast error variance and the climatological minimum of ensemble variance) are straightforward to estimate. The other four parameters are (i) the variance of the climatological distribution of the true conditional error variances, (ii) the climatological minimum of the true conditional error variance, (iii) the relative variance of the distribution of ensemble variances given a true conditional error variance, and (iv) the parameter that defines the mean response of the ensemble variances to changes in the true error variance. These parameters are hidden because they are defined in terms of condition-dependent forecast error variance, which is unobservable if the condition is not sufficiently repeatable. Here, a set of equations that enable these hidden parameters to be accurately estimated from a long time series of (observation minus forecast, ensemble variance) data pairs is presented. The accuracy of the equations is demonstrated in tests using data from long data assimilation cycles with differing model error variance parameters as well as synthetically generated data. This newfound ability to estimate these hidden parameters provides new tools for assessing the quality of ensemble forecasts, tuning hybrid error variance models, and postprocessing ensemble forecasts.

Error Growth Dynamics within Convection-Allowing Ensemble Forecasts over Central U.S. Regions for Days of Active Convection

2021 ◽  
Author(s):  
Xiaoran Zhuang ◽  
Ming Xue ◽  
Jinzhong Min ◽  
Zhiming Kang ◽  
Naigeng Wu ◽  
...  
Keyword(s):  
Diurnal Cycle ◽  
Large Scale ◽  
Response Times ◽  
Forecast Error ◽  
Small Scale ◽  
Forecast Errors ◽  
Error Growth ◽  
Ensemble Forecasts ◽  
Scale Error ◽  
Active Convection

AbstractError growth is investigated based on convection-allowing ensemble forecasts starting from 0000 UTC for 14 active convection events over central to eastern U.S. regions from spring 2018. The analysis domain is divided into the NW, NE, SE and SW quadrants (subregions). Total difference energy and its decompositions are used to measure and analyze error growth at and across scales. Special attention is paid to the dominant types of convection with respect to their forcing mechanisms in the four subregions and the associated difference in precipitation diurnal cycles. The discussions on the average behaviors of error growth in each region are supplemented by 4 representative cases. Results show that the meso-γ-scale error growth is directly linked to precipitation diurnal cycle while meso-α-scale error growth has strong link to large scale forcing. Upscale error growth is evident in all regions/cases but up-amplitude growth within own scale plays different roles in different regions/cases.When large-scale flow is important (as in the NE region), precipitation is strongly modulated by the large-scale forcing and becomes more organized with time, and upscale transfer of forecast error is stronger. On the other hand, when local instability plays more dominant roles (as in the SE region), precipitation is overall least organized and has the weakest diurnal variations. Its associated errors at the γ– and β-scale can reach their peaks sooner and meso-α-scale error tends to rely more on growth of error with its own scale. Small-scale forecast errors are directly impacted by convective activities and have short response time to convection while increasingly larger scale errors have longer response times and delayed phase within the diurnal cycle.

Sign in / Sign up

Export Citation Format

Share Document