A low-cost post-processing technique improves weather forecasts around the world

AbstractComputer-generated weather forecasts divide the Earth’s surface into gridboxes, each currently spanning about 400 km2, and predict one value per gridbox. If weather varies markedly within a gridbox, forecasts for specific sites inevitably fail. Here we present a statistical post-processing method for ensemble forecasts that accounts for the degree of variation within each gridbox, bias on the gridbox scale, and the weather dependence of each. When applying this post-processing, skill improves substantially across the globe; for extreme rainfall, for example, useful forecasts extend 5 days ahead, compared to less than 1 day without post-processing. Skill improvements are attributed to creation of huge calibration datasets by aggregating, globally rather than locally, forecast-observation differences wherever and whenever the observed “weather type” was similar. A strong focus on meteorological understanding also contributes. We suggest that applications for our methodology include improved flash flood warnings, physics-related insights into model weaknesses and global pointwise re-analyses.

Download Full-text

Ensemble-Based Analysis of the May 2010 Extreme Rainfall in Tennessee and Kentucky

Monthly Weather Review ◽

10.1175/mwr-d-13-00020.1 ◽

2014 ◽

Vol 142 (1) ◽

pp. 222-239 ◽

Cited By ~ 15

Author(s):

Samantha L. Lynch ◽

Russ S. Schumacher

Keyword(s):

Mississippi River ◽

Flash Flood ◽

Extreme Rainfall ◽

Heavy Rain ◽

Heavy Precipitation ◽

Extreme Rainfall Event ◽

Ensemble Forecasts ◽

Surface Cyclone ◽

Upper Level ◽

Surrounding Areas

Abstract From 1 to 3 May 2010, persistent heavy rainfall occurred in the Ohio and Mississippi River valleys due to two successive quasi-stationary mesoscale convective systems (MCSs), with locations in central Tennessee accumulating more than 483 mm of rain, and the city of Nashville experiencing a historic flash flood. This study uses operational global ensemble forecasts from the European Centre for Medium-Range Weather Forecasts (ECMWF) to diagnose atmospheric processes and assess forecast uncertainty in this event. Several ensemble analysis methods are used to examine the processes that led to the development and maintenance of this precipitation system. Differences between ensemble members that correctly predicted heavy precipitation and those that did not were determined, in order to pinpoint the processes that were favorable or detrimental to the system's development. Statistical analysis was used to determine how synoptic-scale flows were correlated to 5-day area-averaged precipitation. The precipitation throughout Nashville and the surrounding areas occurred ahead of an upper-level trough located over the central United States. The distribution of precipitation was found to be closely related to the strength of this trough and an associated surface cyclone. In particular, when the upper-level trough was elongated, the surface cyclone remained weaker with a narrower low-level jet from the south. This caused the plume of moisture from the Caribbean Sea to be concentrated over Tennessee and Kentucky, where, in conjunction with focused ascent, heavy rain fell. Relatively small differences in the wind and pressure fields led to important differences in the precipitation forecasts and highlighted some of the uncertainties associated with predicting this extreme rainfall event.

Download Full-text

Simulation-based comparison of multivariate ensemble post-processing methods

Nonlinear Processes in Geophysics ◽

10.5194/npg-27-349-2020 ◽

2020 ◽

Vol 27 (2) ◽

pp. 349-371 ◽

Cited By ~ 1

Author(s):

Sebastian Lerch ◽

Sándor Baran ◽

Annette Möller ◽

Jürgen Groß ◽

Roman Schefzik ◽

...

Keyword(s):

Covariance Structure ◽

Post Processing ◽

Copula Functions ◽

Ensemble Forecasts ◽

Processing Methods ◽

Practical Applications ◽

Weather Forecasts ◽

Simulation Based ◽

Mean Variance

Abstract. Many practical applications of statistical post-processing methods for ensemble weather forecasts require accurate modeling of spatial, temporal, and inter-variable dependencies. Over the past years, a variety of approaches has been proposed to address this need. We provide a comprehensive review and comparison of state-of-the-art methods for multivariate ensemble post-processing. We focus on generally applicable two-step approaches where ensemble predictions are first post-processed separately in each margin and multivariate dependencies are restored via copula functions in a second step. The comparisons are based on simulation studies tailored to mimic challenges occurring in practical applications and allow ready interpretation of the effects of different types of misspecifications in the mean, variance, and covariance structure of the ensemble forecasts on the performance of the post-processing methods. Overall, we find that the Schaake shuffle provides a compelling benchmark that is difficult to outperform, whereas the forecast quality of parametric copula approaches and variants of ensemble copula coupling strongly depend on the misspecifications at hand.

Download Full-text

Using Machine Learning to Generate Storm-Scale Probabilistic Guidance of Severe Weather Hazards in the Warn-on-Forecast System

Monthly Weather Review ◽

10.1175/mwr-d-20-0194.1 ◽

2021 ◽

Author(s):

Montgomery L. Flora ◽

Corey K. Potvin ◽

Patrick S. Skinner ◽

Shawn Handler ◽

Amy McGovern

Keyword(s):

Machine Learning ◽

Storm Track ◽

Severe Weather ◽

Storm Tracks ◽

State Variables ◽

Post Processing ◽

Short Term ◽

Ensemble Forecasts ◽

Weather Forecasts ◽

Environment Variables

AbstractA primary goal of the National Oceanic and Atmospheric Administration Warn-on-Forecast (WoF) project is to provide rapidly updating probabilistic guidance to human forecasters for short-term (e.g., 0-3 h) severe weather forecasts. Post-processing is required to maximize the usefulness of probabilistic guidance from an ensemble of convection-allowing model forecasts. Machine learning (ML) models have become popular methods for post-processing severe weather guidance since they can leverage numerous variables to discover useful patterns in complex datasets. In this study, we develop and evaluate a series of ML models to produce calibrated, probabilistic severe weather guidance from WoF System (WoFS) output.Our dataset includes WoFS ensemble forecasts available every 5 minutes out to 150 min of lead time from the 2017-2019 NOAA Hazardous Weather Testbed Spring Forecasting Experiments (81 dates). Using a novel ensemble storm track identification method, we extracted three sets of predictors from the WoFS forecasts: intra-storm state variables, near-storm environment variables, and morphological attributes of the ensemble storm tracks. We then trained random forests, gradient-boosted trees, and logistic regression algorithms to predict which WoFS 30-min ensemble storm tracks will overlap a tornado, severe hail, and/or severe wind report. To provide rigorous baselines against which to evaluate the skill of the ML models, we extracted the ensemble probabilities of hazard-relevant WoFS variables exceeding tuned thresholds from each ensemble storm track. The three ML algorithms discriminated well for all three hazards and produced more reliable probabilities than the baseline predictions. Overall, the results suggest that ML-based post-processing of dynamical ensemble output can improve short term, storm-scale severe weather probabilistic guidance.

Download Full-text

Simulation-based comparison of multivariate ensemble post-processing methods

10.5194/npg-2019-62 ◽

2020 ◽

Author(s):

Sebastian Lerch ◽

Sándor Baran ◽

Annette Möller ◽

Jürgen Groß ◽

Roman Schefzik ◽

...

Keyword(s):

Covariance Structure ◽

Post Processing ◽

Copula Functions ◽

Ensemble Forecasts ◽

Processing Methods ◽

Practical Applications ◽

Weather Forecasts ◽

Simulation Based ◽

Mean Variance

Abstract. Many practical applications of statistical post-processing methods for ensemble weather forecasts require to accurately model spatial, temporal and inter-variable dependencies. Over the past years, a variety of approaches has been proposed to address this need. We provide a comprehensive review and comparison of state of the art methods for multivariate ensemble post-processing. We focus on generally applicable two-step approaches where ensemble predictions are first post-processed separately in each margin, and multivariate dependencies are restored via copula functions in a second step. The comparisons are based on simulation studies tailored to mimic challenges occurring in practical applications and allow to readily interpret the effects of different types of misspecifications in the mean, variance and covariance structure of the ensemble forecasts on the performance of the post-processing methods. Overall, we find that the Schaake shuffle provides a compelling benchmark that is difficult to outperform, whereas the forecast quality of parametric copula approaches and variants of ensemble copula coupling strongly depend on the misspecifications at hand.

Download Full-text

Statistical post-processing of ensemble forecasts at the Belgian met service

10.5194/egusphere-egu21-2495 ◽

2021 ◽

Author(s):

Jonathan Demaeyer ◽

Bert Van schaeybroeck ◽

Stéphane Vannitsem

Keyword(s):

Maximum Temperature ◽

Post Processing ◽

Ensemble Forecasts ◽

Weather Forecasts ◽

Operational Application ◽

Uncertainty Estimates ◽

The Mean ◽

Model Output Statistics ◽

Ensemble Weather Forecasts ◽

Meteorological Institute

Statistical post-processing of ensemble weather forecasts has become an essential step in the forecasting chain as it enables the correction of biases and reliable uncertainty estimates of ensembles (Gneiting, 2014).&#160;&#160;One algorithm recently proposed to perform the correction of ensemble weather forecasts is a linear member-by-member (MBM) Model Output Statistics (MOS) system, post-processing each member of the ECMWF ensemble (Van Schaeybroeck & Vannitsem, 2015). This method consists in correcting the mean and variability of the ensemble members in line with the observed climatology. At the same time, it calibrates the ensemble spread such as to match, on average, the mean square error of the ensemble mean. The MBM method calibrates the ensemble forecasts based on the station observations by minimizing the continuous ranked probability score (CRPS).Using this method, the Royal Meteorological Institute of Belgium has started in 2020 its new postprocessing program by developing an operational application to perform the calibration of the ECMWF ensemble forecasts at the stations points for the minimum and maximum temperature, and for wind gusts.&#160;In this report, we will first describe briefly the postprocessing methods being used and the architecture of the application. We will then present the results over the first few months of operation. Finally, we will discuss the future developments of this application and of the program. Gneiting, T., 2014: Calibration of medium-range weather forecasts. ECMWF Technical Memorandum No. 719 Van Schaeybroeck, B. & Vannitsem, S., 2015: Ensemble post-processing using member-by-member approaches: theoretical aspects. Quarterly Journal of the Royal Meteorological Society, 141, 807&#8211;818.

Download Full-text

Recent Advances in Real-Time Pluvial Flash Flood Forecasting

Water ◽

10.3390/w12020570 ◽

2020 ◽

Vol 12 (2) ◽

pp. 570 ◽

Cited By ~ 5

Author(s):

Andre Zanchetta ◽

Paulin Coulibaly

Keyword(s):

Flash Flood ◽

Low Cost ◽

Extreme Rainfall ◽

Early Warning Systems ◽

Flood Forecasting ◽

Flash Floods ◽

Atmospheric Conditions ◽

Extreme Rainfall Events ◽

Recent Advances ◽

Flash Flood Forecasting

Recent years have witnessed considerable developments in multiple fields with the potential to enhance our capability of forecasting pluvial flash floods, one of the most costly environmental hazards in terms of both property damage and loss of life. This work provides a summary and description of recent advances related to insights on atmospheric conditions that precede extreme rainfall events, to the development of monitoring systems of relevant hydrometeorological parameters, and to the operational adoption of weather and hydrological models towards the prediction of flash floods. With the exponential increase of available data and computational power, most of the efforts are being directed towards the improvement of multi-source data blending and assimilation techniques, as well as assembling approaches for uncertainty estimation. For urban environments, in which the need for high-resolution simulations demands computationally expensive systems, query-based approaches have been explored for the timely retrieval of pre-simulated flood inundation forecasts. Within the concept of the Internet of Things, the extensive deployment of low-cost sensors opens opportunities from the perspective of denser monitoring capabilities. However, different environmental conditions and uneven distribution of data and resources usually leads to the adoption of site-specific solutions for flash flood forecasting in the context of early warning systems.

Download Full-text

Autoclaved and Extruded Legumes as a Source of Bioactive Phytochemicals: A Review

Foods ◽

10.3390/foods10020379 ◽

2021 ◽

Vol 10 (2) ◽

pp. 379

Author(s):

Mercedes M. Pedrosa ◽

Eva Guillamón ◽

Claudia Arribas

Keyword(s):

Inositol Phosphates ◽

Low Cost ◽

Bioactive Compound ◽

Well Being ◽

Processing Technique ◽

Processing Method ◽

Industrial Processing ◽

Pulse Type ◽

Bioactive Phytochemicals ◽

Traditional Processing

Legumes have been consumed since ancient times all over the world due to their easy cultivation and availability as a low-cost food. Nowadays, it is well known that pulses are also a good source of bioactive phytochemicals that play an important role in the health and well-being of humans. Pulses are mainly consumed after processing to soften cotyledons and to improve their nutritive and sensorial characteristics. However, processing affects not only their nutritive constituents, but also their bioactive compounds. The final content of phytochemicals depends on the pulse type and variety, the processing method and their parameters (mainly temperature and time), the food matrix structure and the chemical nature of each phytochemical. This review focuses on the changes produced in the bioactive-compound content of pulses processed by a traditional processing method like cooking (with or without pressure) or by an industrial processing technique like extrusion, which is widely used in the food industry to develop new food products with pulse flours as ingredients. In particular, the effect of processing methods on inositol phosphates, galactosides, protease inhibitors and phenolic-compound content is highlighted in order to ascertain their content in processed pulses or pulse-based products as a source of healthy phytochemicals.

Download Full-text

Post-Processing Ensemble Weather Forecasts for Introducing Multisite and Multivariable Correlations using Rank Shuffle and Copula Theory

Monthly Weather Review ◽

10.1175/mwr-d-21-0100.1 ◽

2021 ◽

Keyword(s):

Latent Variables ◽

Gaussian Copula ◽

Post Processing ◽

Ensemble Forecasts ◽

Copula Theory ◽

Weather Forecasts ◽

Empirical Copula ◽

Post Process ◽

Reconstruction Methods ◽

Ensemble Weather Forecasts

Abstract Statistical methods have been widely used to post-process ensemble weather forecasts for hydrological predictions. However, most of the statistical post-processing methods apply to a single weather variable at a single location, thus neglecting the inter-site and inter-variable dependence structures of forecast variables. This study synthesized a multisite and multivariate (MSMV) post-processing framework that extends the univariate method to the MSMV version by directly rearranging the post-processed ensemble members (post-reordering strategy) or by rearranging the latent variables used in univariate method (pre-reordering strategy). Based on the univariate Generator-based Post-Processing (GPP) method, the two reordering strategies and three dependence reconstruction methods (Rank shuffle (RS), Gaussian Copula (GC), and Empirical Copula (EC)) totaling 6 MSMV methods (RS-Pre, GC-Pre, EC-Pre, RS-Post, GC-Post, and EC-Post) were evaluated in post-processing ensemble precipitation and temperature forecasts for the Xiangjiang Basin in China using the 11-member ensemble forecasts from the Global Ensemble Forecasting System (GEFS). The results showed that raw GEFS forecasts tend to be biased for both the forecast ensembles and the inter-site and inter-variable dependencies. Univariate method can improve the univariate performance of ensemble mean and spread but misrepresent the inter-site and inter-variable dependence among the forecast variables. The MSMV framework can well utilize the advantages of the univariate method and also reconstruct the inter-site and inter-variable dependencies. Among the six methods, RS-Pre, RS-Post, GC-Post, and EC-Post perform better than the others with respect to reproducing the univariate statistics and multivariable dependences. The post-reordering strategy is recommended to combine the univariate method (i.e. GPP) and reconstruction methods.

Download Full-text

The MISTRAL Project provides a new tool for Flash Flood Forecasting in Italy

10.5194/ems2021-87 ◽

2021 ◽

Author(s):

Estíbaliz Gascón ◽

Andrea Montani ◽

Tim Hewson

Keyword(s):

Heavy Rainfall ◽

Positive Impact ◽

Flash Flood ◽

Flash Floods ◽

Processing Technique ◽

Added Value ◽

Limited Area ◽

The European Union ◽

Post Processing ◽

The World

Localized heavy rainfall, which can be associated with flash floods, is difficult to predict accurately: both the predicted location and the intensity can exhibit large errors. Moreover, weather forecasts should be provided for points and not for the large regions represented by global model grid boxes. This mismatch can in principle be addressed using high-resolution limited-area models, or by applying some post-processing to global forecast models, as used in &#8220;ecPoint-rainfall&#8221;, a new ECMWF probabilistic post-processing technique to improve precipitation forecasts. One novel premise of ecPoint, which has a major positive impact on the calibration, is that the forecast-versus-point-observation relationship depends on &#8220;gridbox weather types&#8221; that could potentially occur in many parts of the world.The MISTRAL (Meteo Italian SupercompuTing PoRtAL) project, funded under the Connecting Europe Facility (CEF) &#8211; Telecommunication Sector Programme of the European Union came to its end in January 2021. The main project goal was to facilitate and foster the re-use of datasets by weather-dependent communities, to provide added value services using HPC resources. ECMWF participated in the project with the goal of improving probabilistic 6-h rainfall forecast products, to improve the prediction of flash floods in Italy and nearby Mediterranean regions. One of the objectives was to exploit the CINECA supercomputer facilities in Bologna to extract maximum benefit from ecPoint-Rainfall and from a 2.2km resolution COSMO limited area ensemble. To address that, we applied a new and innovative scale-selective neighbourhood post-processing technique to the COSMOS output, which, on the one hand, identifies and preserves the most reliable heavy rainfall signals and, on the other, spreads out those signals which are less consistently handled. Then, it is blended with a new 6h ecPoint-Rainfall product in order to leverage the most skilful aspects of the two systems. The 6-h ecPoint Rainfall forecasts were also developed during the project, building on the pre-existing ecPoint-Rainfall 12h product (already delivered to ECMWF customers in real-time). The final blended product includes, for lead times of 1-10 days, 6-h accumulated rainfall for each COSMO gridbox in percentiles (1, 2,..99) and probabilities of exceeding certain thresholds.The main objective of this work was to improve forecasts and support weather-alert decisions for flash flood prediction. As a legacy of the project, we are now providing forecast data for Italy and nearby regions with a higher level of quality and resolution than has hitherto been possible,&#160; and we are also delivering a robust gateway to products for the European community within the MISTRAL portal (https://meteohub.hpc.cineca.it/app/maps/flashflood). The principles could also be usefully applied in other parts of Europe, or indeed the world, where limited area ensembles are running operationally.In this presentation, we will introduce the methodologies, the verification results and will illustrate with forecast examples.

Download Full-text