scholarly journals Simulation-based comparison of multivariate ensemble post-processing methods

2020 ◽  
Vol 27 (2) ◽  
pp. 349-371 ◽  
Author(s):  
Sebastian Lerch ◽  
Sándor Baran ◽  
Annette Möller ◽  
Jürgen Groß ◽  
Roman Schefzik ◽  
...  
Keyword(s):  
Covariance Structure ◽  
Post Processing ◽  
Copula Functions ◽  
Ensemble Forecasts ◽  
Processing Methods ◽  
Practical Applications ◽  
Weather Forecasts ◽  
Simulation Based ◽  
Mean Variance

Abstract. Many practical applications of statistical post-processing methods for ensemble weather forecasts require accurate modeling of spatial, temporal, and inter-variable dependencies. Over the past years, a variety of approaches has been proposed to address this need. We provide a comprehensive review and comparison of state-of-the-art methods for multivariate ensemble post-processing. We focus on generally applicable two-step approaches where ensemble predictions are first post-processed separately in each margin and multivariate dependencies are restored via copula functions in a second step. The comparisons are based on simulation studies tailored to mimic challenges occurring in practical applications and allow ready interpretation of the effects of different types of misspecifications in the mean, variance, and covariance structure of the ensemble forecasts on the performance of the post-processing methods. Overall, we find that the Schaake shuffle provides a compelling benchmark that is difficult to outperform, whereas the forecast quality of parametric copula approaches and variants of ensemble copula coupling strongly depend on the misspecifications at hand.

scholarly journals Simulation-based comparison of multivariate ensemble post-processing methods

2020 ◽  
Author(s):  
Sebastian Lerch ◽  
Sándor Baran ◽  
Annette Möller ◽  
Jürgen Groß ◽  
Roman Schefzik ◽  
...  
Keyword(s):  
Covariance Structure ◽  
Post Processing ◽  
Copula Functions ◽  
Ensemble Forecasts ◽  
Processing Methods ◽  
Practical Applications ◽  
Weather Forecasts ◽  
Simulation Based ◽  
Mean Variance

Abstract. Many practical applications of statistical post-processing methods for ensemble weather forecasts require to accurately model spatial, temporal and inter-variable dependencies. Over the past years, a variety of approaches has been proposed to address this need. We provide a comprehensive review and comparison of state of the art methods for multivariate ensemble post-processing. We focus on generally applicable two-step approaches where ensemble predictions are first post-processed separately in each margin, and multivariate dependencies are restored via copula functions in a second step. The comparisons are based on simulation studies tailored to mimic challenges occurring in practical applications and allow to readily interpret the effects of different types of misspecifications in the mean, variance and covariance structure of the ensemble forecasts on the performance of the post-processing methods. Overall, we find that the Schaake shuffle provides a compelling benchmark that is difficult to outperform, whereas the forecast quality of parametric copula approaches and variants of ensemble copula coupling strongly depend on the misspecifications at hand.

scholarly journals From research to applications – examples of operational ensemble post-processing in France using machine learning

2020 ◽  
Vol 27 (2) ◽  
pp. 329-347 ◽  
Author(s):  
Maxime Taillardat ◽  
Olivier Mestre
Keyword(s):  
Machine Learning ◽  
Heat Waves ◽  
Computation Time ◽  
Ensemble Prediction ◽  
Post Processing ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
Practical Applications ◽  
Medium Resolution ◽  
Hourly Rainfall

Abstract. Statistical post-processing of ensemble forecasts, from simple linear regressions to more sophisticated techniques, is now a well-known procedure for correcting biased and poorly dispersed ensemble weather predictions. However, practical applications in national weather services are still in their infancy compared to deterministic post-processing. This paper presents two different applications of ensemble post-processing using machine learning at an industrial scale. The first is a station-based post-processing of surface temperature and subsequent interpolation to a grid in a medium-resolution ensemble system. The second is a gridded post-processing of hourly rainfall amounts in a high-resolution ensemble prediction system. The techniques used rely on quantile regression forests (QRFs) and ensemble copula coupling (ECC), chosen for their robustness and simplicity of training regardless of the variable subject to calibration. Moreover, some variants of classical techniques used, such as QRF and ECC, were developed in order to adjust to operational constraints. A forecast anomaly-based QRF is used for temperature for a better prediction of cold and heat waves. A variant of ECC for hourly rainfall was built, accounting for more realistic longer rainfall accumulations. We show that both forecast quality and forecast value are improved compared to the raw ensemble. Finally, comments about model size and computation time are made.

scholarly journals A low-cost post-processing technique improves weather forecasts around the world

2021 ◽  
Vol 2 (1) ◽  
Author(s):  
Timothy David Hewson ◽  
Fatima Maria Pillosu
Keyword(s):  
Flash Flood ◽  
Low Cost ◽  
Extreme Rainfall ◽  
Processing Technique ◽  
Processing Method ◽  
Weather Type ◽  
Post Processing ◽  
Ensemble Forecasts ◽  
Weather Forecasts ◽  
Strong Focus

AbstractComputer-generated weather forecasts divide the Earth’s surface into gridboxes, each currently spanning about 400 km2, and predict one value per gridbox. If weather varies markedly within a gridbox, forecasts for specific sites inevitably fail. Here we present a statistical post-processing method for ensemble forecasts that accounts for the degree of variation within each gridbox, bias on the gridbox scale, and the weather dependence of each. When applying this post-processing, skill improves substantially across the globe; for extreme rainfall, for example, useful forecasts extend 5 days ahead, compared to less than 1 day without post-processing. Skill improvements are attributed to creation of huge calibration datasets by aggregating, globally rather than locally, forecast-observation differences wherever and whenever the observed “weather type” was similar. A strong focus on meteorological understanding also contributes. We suggest that applications for our methodology include improved flash flood warnings, physics-related insights into model weaknesses and global pointwise re-analyses.

scholarly journals Using Machine Learning to Generate Storm-Scale Probabilistic Guidance of Severe Weather Hazards in the Warn-on-Forecast System

2021 ◽  
Author(s):  
Montgomery L. Flora ◽  
Corey K. Potvin ◽  
Patrick S. Skinner ◽  
Shawn Handler ◽  
Amy McGovern
Keyword(s):  
Machine Learning ◽  
Storm Track ◽  
Severe Weather ◽  
Storm Tracks ◽  
State Variables ◽  
Post Processing ◽  
Short Term ◽  
Ensemble Forecasts ◽  
Weather Forecasts ◽  
Environment Variables

AbstractA primary goal of the National Oceanic and Atmospheric Administration Warn-on-Forecast (WoF) project is to provide rapidly updating probabilistic guidance to human forecasters for short-term (e.g., 0-3 h) severe weather forecasts. Post-processing is required to maximize the usefulness of probabilistic guidance from an ensemble of convection-allowing model forecasts. Machine learning (ML) models have become popular methods for post-processing severe weather guidance since they can leverage numerous variables to discover useful patterns in complex datasets. In this study, we develop and evaluate a series of ML models to produce calibrated, probabilistic severe weather guidance from WoF System (WoFS) output.Our dataset includes WoFS ensemble forecasts available every 5 minutes out to 150 min of lead time from the 2017-2019 NOAA Hazardous Weather Testbed Spring Forecasting Experiments (81 dates). Using a novel ensemble storm track identification method, we extracted three sets of predictors from the WoFS forecasts: intra-storm state variables, near-storm environment variables, and morphological attributes of the ensemble storm tracks. We then trained random forests, gradient-boosted trees, and logistic regression algorithms to predict which WoFS 30-min ensemble storm tracks will overlap a tornado, severe hail, and/or severe wind report. To provide rigorous baselines against which to evaluate the skill of the ML models, we extracted the ensemble probabilities of hazard-relevant WoFS variables exceeding tuned thresholds from each ensemble storm track. The three ML algorithms discriminated well for all three hazards and produced more reliable probabilities than the baseline predictions. Overall, the results suggest that ML-based post-processing of dynamical ensemble output can improve short term, storm-scale severe weather probabilistic guidance.

scholarly journals From research to applications – Examples of operational ensemble post-processing in France using machine learning

2020 ◽  
Author(s):  
Maxime Taillardat ◽  
Olivier Mestre
Keyword(s):  
Machine Learning ◽  
Heat Waves ◽  
Computation Time ◽  
Ensemble Prediction ◽  
Post Processing ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
Practical Applications ◽  
Medium Resolution ◽  
Hourly Rainfall

Abstract. Statistical post-processing of ensemble forecasts, from simple linear regressions to more sophisticated techniques, is now a well-known procedure in order to correct biased and misdispersed ensemble weather predictions. However, practical applications in National Weather Services is still in its infancy compared to deterministic post-processing. This paper presents two different applications of ensemble post-processing using machine learning at an industrial scale. The first is a station-based post-processing of surface temperature in a medium resolution ensemble system. The second is a gridded post-processing of hourly rainfall amounts in a high resolution ensemble prediction system. The techniques used rely on quantile regression forests (QRF) and ensemble copula coupling (ECC), chosen for their robustness and simplicity of training whatever the variable subject to calibration. Moreover, some variants of classical techniques used such as QRF or ECC have been developed in order to adjust to operational constraints. A forecast anomaly-based QRF is used for temperature for a better prediction of cold and heat waves. A variant of ECC for hourly rainfall is built, accounting for more realistic longer rainfall accumulations. It is shown that forecast quality as well as forecast value is improved compared to the raw ensemble. At last, comments about model size and computation time are made.

scholarly journals Calibrated ensemble forecasts of the height of new snow using quantile regression forests and ensemble model output statistics

2021 ◽  
Vol 28 (3) ◽  
pp. 467-480
Author(s):  
Guillaume Evin ◽  
Matthieu Lafaysse ◽  
Maxime Taillardat ◽  
Michaël Zamo
Keyword(s):  
Quantile Regression ◽  
Specific Humidity ◽  
Ensemble Model ◽  
Model Output ◽  
Post Processing ◽  
Ensemble Forecasts ◽  
Mountain Areas ◽  
Processing Methods ◽  
Ensemble Model Output Statistics ◽  
Model Output Statistics

Abstract. Height of new snow (HN) forecasts help to prevent critical failures of infrastructures in mountain areas, e.g. transport networks and ski resorts. The French national meteorological service, Météo-France, operates a probabilistic forecasting system based on ensemble meteorological forecasts and a detailed snowpack model to provide ensembles of HN forecasts. These forecasts are, however, biased and underdispersed. As for many weather variables, post-processing methods can be used to alleviate these drawbacks and obtain meaningful 1 to 4 d HN forecasts. In this paper, we compare the skill of two post-processing methods. The first approach is an ensemble model output statistics (EMOS) method, which can be described as a nonhomogeneous regression with a censored shifted Gamma distribution. The second approach is based on quantile regression forests, using different meteorological and snow predictors. Both approaches are evaluated using a 22 year reforecast. Thanks to a larger number of predictors, the quantile regression forest is shown to be a powerful alternative to EMOS for the post-processing of HN ensemble forecasts. The gain of performance is large in all situations but is particularly marked when raw forecasts completely miss the snow event. This type of situation happens when the rain–snow transition elevation is overestimated by the raw forecasts (rain instead of snow in the raw forecasts) or when there is no precipitation in the forecast. In that case, quantile regression forests improve the predictions using the other weather predictors (wind, temperature, and specific humidity).

scholarly journals Calibrated ensemble forecasts of the height of new snow using quantile regression forests and Ensemble Model Output Statistics

2021 ◽  
Author(s):  
Guillaume Evin ◽  
Matthieu Lafaysse ◽  
Maxime Taillardat ◽  
Michaël Zamo
Keyword(s):  
Quantile Regression ◽  
Specific Humidity ◽  
Ensemble Model ◽  
Model Output ◽  
Post Processing ◽  
Ensemble Forecasts ◽  
Mountain Areas ◽  
Processing Methods ◽  
Ensemble Model Output Statistics ◽  
Model Output Statistics

Abstract. Height of new snow (HN) forecasts help to prevent critical failures of infrastructures in mountain areas, e.g. transport networks, ski resorts. The French national meteorological service, Meteo-France, operates a probabilistic forecasting system based on ensemble meteorological forecasts and a detailed snowpack model to provide ensembles of HN forecasts. These forecasts are however significantly biased and underdispersed. As for many weather variables, post-processing methods can be used to alleviate these drawbacks and obtain meaningful 1-day to 4-day HN forecasts. In this paper, we compare the skill of two post-processing methods. The first approach is an ensemble model output statistics (EMOS) method, which can be described as a Nonhomogeneous Regression with a Censored Shifted Gamma distribution. The second approach is based on quantile regression forests, using different meteorological and snow predictors. Both approaches are evaluated using a 22-year reforecast. Thanks to a larger number of predictors, the quantile regression forest is shown to be a powerful alternative to EMOS for the post-processing of HN ensemble forecasts. The gain of performance is important in all situations but is particularly marked when raw forecasts completely miss the snow event. This type of situations happens when the rain-snow transition elevation is overestimated by the raw forecasts (rain instead of snow in the raw forecasts) or when there is no precipitation in the forecast. In that case, quantile regression forests improve the predictions using the other weather predictors (wind, temperature, specific humidity).

Statistical post-processing of ensemble forecasts at the Belgian met service

2021 ◽  
Author(s):  
Jonathan Demaeyer ◽  
Bert Van schaeybroeck ◽  
Stéphane Vannitsem
Keyword(s):  
Maximum Temperature ◽  
Post Processing ◽  
Ensemble Forecasts ◽  
Weather Forecasts ◽  
Operational Application ◽  
Uncertainty Estimates ◽  
The Mean ◽  
Model Output Statistics ◽  
Ensemble Weather Forecasts ◽  
Meteorological Institute

<p>Statistical post-processing of ensemble weather forecasts has become an essential step in the forecasting chain as it enables the correction of biases and reliable uncertainty estimates of ensembles (Gneiting, 2014).  One algorithm recently proposed to perform the correction of ensemble weather forecasts is a linear member-by-member (MBM) Model Output Statistics (MOS) system, post-processing each member of the ECMWF ensemble (Van Schaeybroeck & Vannitsem, 2015). This method consists in correcting the mean and variability of the ensemble members in line with the observed climatology. At the same time, it calibrates the ensemble spread such as to match, on average, the mean square error of the ensemble mean. The MBM method calibrates the ensemble forecasts based on the station observations by minimizing the continuous ranked probability score (CRPS).</p><p><span>Using this method, the Royal Meteorological Institute of Belgium has started in 2020 its new postprocessing program by developing an operational application to perform the calibration of the ECMWF ensemble forecasts at the stations points for the minimum and maximum temperature, and for wind gusts. </span>In this report, we will first describe briefly the postprocessing methods being used and the architecture of the application. We will then present the results over the first few months of operation. Finally, we will discuss the future developments of this application and of the program.</p><p><span><br></span></p><p><span> </span> <span> </span></p><p><strong>Gneiting</strong>, <strong>T.</strong>, 2014: Calibration of medium-range weather forecasts. <em>ECMWF Technical Memorandum</em> <strong>No. 719</strong></p><p><span> </span> <span> </span></p><p><strong>Van Schaeybroeck</strong>, <strong>B.</strong> & <strong>Vannitsem</strong>, <strong>S.</strong>, 2015: Ensemble post-processing using member-by-member approaches: theoretical aspects. <em>Quarterly Journal of the Royal Meteorological Society</em>, <strong>141</strong>, 807–818.</p>

Sign in / Sign up

Export Citation Format

Share Document