scholarly journals Calibration of wind speed ensemble forecasts for power generation

Időjárás ◽  
2021 ◽  
Vol 125 (4) ◽  
pp. 609-624
Author(s):  
Sándor Baran ◽  
Ágnes Baran
Keyword(s):  
Machine Learning ◽  
Wind Speed ◽  
Wind Power ◽  
Weather Prediction ◽  
Predictive Performance ◽  
Ensemble Prediction ◽  
Post Processing ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
Truncated Normal

In the last decades, wind power became the second largest energy source in the EU covering 16% of its electricity demand. However, due to its volatility, accurate short range wind power predictions are required for successful integration of wind energy into the electrical grid. Accurate predictions of wind power require accurate hub height wind speed forecasts, where the state-of-the-art method is the probabilistic approach based on ensemble forecasts obtained from multiple runs of numerical weather prediction models. Nonetheless, ensemble forecasts are often uncalibrated and might also be biased, thus require some form of post-processing to improve their predictive performance. We propose a novel flexible machine learning approach for calibrating wind speed ensemble forecasts, which results in a truncated normal predictive distribution. In a case study based on 100m wind speed forecasts produced by the operational ensemble prediction system of the Hungarian Meteorological Service, the forecast skill of this method is compared with the predictive performance of three different ensemble model output statistics approaches and the raw ensemble forecasts. We show that compared with the raw ensemble, post-processing always improves the calibration of probabilistic and accuracy of point forecasts, and from the four competing methods, the novel machine learning based approach results in the best overall performance.

scholarly journals From research to applications – examples of operational ensemble post-processing in France using machine learning

2020 ◽  
Vol 27 (2) ◽  
pp. 329-347 ◽  
Author(s):  
Maxime Taillardat ◽  
Olivier Mestre
Keyword(s):  
Machine Learning ◽  
Heat Waves ◽  
Computation Time ◽  
Ensemble Prediction ◽  
Post Processing ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
Practical Applications ◽  
Medium Resolution ◽  
Hourly Rainfall

Abstract. Statistical post-processing of ensemble forecasts, from simple linear regressions to more sophisticated techniques, is now a well-known procedure for correcting biased and poorly dispersed ensemble weather predictions. However, practical applications in national weather services are still in their infancy compared to deterministic post-processing. This paper presents two different applications of ensemble post-processing using machine learning at an industrial scale. The first is a station-based post-processing of surface temperature and subsequent interpolation to a grid in a medium-resolution ensemble system. The second is a gridded post-processing of hourly rainfall amounts in a high-resolution ensemble prediction system. The techniques used rely on quantile regression forests (QRFs) and ensemble copula coupling (ECC), chosen for their robustness and simplicity of training regardless of the variable subject to calibration. Moreover, some variants of classical techniques used, such as QRF and ECC, were developed in order to adjust to operational constraints. A forecast anomaly-based QRF is used for temperature for a better prediction of cold and heat waves. A variant of ECC for hourly rainfall was built, accounting for more realistic longer rainfall accumulations. We show that both forecast quality and forecast value are improved compared to the raw ensemble. Finally, comments about model size and computation time are made.

scholarly journals From research to applications – Examples of operational ensemble post-processing in France using machine learning

2020 ◽  
Author(s):  
Maxime Taillardat ◽  
Olivier Mestre
Keyword(s):  
Machine Learning ◽  
Heat Waves ◽  
Computation Time ◽  
Ensemble Prediction ◽  
Post Processing ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
Practical Applications ◽  
Medium Resolution ◽  
Hourly Rainfall

Abstract. Statistical post-processing of ensemble forecasts, from simple linear regressions to more sophisticated techniques, is now a well-known procedure in order to correct biased and misdispersed ensemble weather predictions. However, practical applications in National Weather Services is still in its infancy compared to deterministic post-processing. This paper presents two different applications of ensemble post-processing using machine learning at an industrial scale. The first is a station-based post-processing of surface temperature in a medium resolution ensemble system. The second is a gridded post-processing of hourly rainfall amounts in a high resolution ensemble prediction system. The techniques used rely on quantile regression forests (QRF) and ensemble copula coupling (ECC), chosen for their robustness and simplicity of training whatever the variable subject to calibration. Moreover, some variants of classical techniques used such as QRF or ECC have been developed in order to adjust to operational constraints. A forecast anomaly-based QRF is used for temperature for a better prediction of cold and heat waves. A variant of ECC for hourly rainfall is built, accounting for more realistic longer rainfall accumulations. It is shown that forecast quality as well as forecast value is improved compared to the raw ensemble. At last, comments about model size and computation time are made.

scholarly journals From research to applications – Examples of operational ensemble post-processing in France using machine learning

2020 ◽  
Author(s):  
Maxime Taillardat ◽  
Olivier Mestre
Keyword(s):  
Machine Learning ◽  
Heat Waves ◽  
Computation Time ◽  
Ensemble Prediction ◽  
Post Processing ◽  
Ensemble Forecasts ◽  
Ensemble Prediction System ◽  
Practical Applications ◽  
Medium Resolution ◽  
Hourly Rainfall

<p>Statistical post-processing of ensemble forecasts, from simple linear regressions to more sophisticated techniques, is now a well-known procedure in order to correct biased and misdispersed ensemble weather predictions. However, practical applications in National Weather Services is still in its infancy compared to deterministic post-processing. This paper presents two different applications of ensemble post-processing using machine learning at an industrial scale. The first is a station-based post-processing of surface temperature in a medium resolution ensemble system. The second is a gridded post-processing of hourly rainfall amounts in a high resolution ensemble prediction system. The techniques used rely on quantile regression forests (QRF) and ensemble copula coupling (ECC), chosen for their robustness and simplicity of training whatever the variable subject to calibration.</p><p>Moreover, some variants of classical techniques used such as QRF or ECC have been developed in order to adjust to operational constraints. A forecast anomaly-based QRF is used for temperature for a better prediction of cold and heat waves. A variant of ECC for hourly rainfall is built, accounting for more realistic longer rainfall accumulations. It is shown that forecast quality as well as forecast value is improved compared to the raw ensemble. At last, comments about model size and computation time are made.</p>

scholarly journals Generating wind power scenarios for probabilistic ramp event prediction using multivariate statistical post-processing

2018 ◽  
Author(s):  
Rochelle P. Worsnop ◽  
Michael Scheuerer ◽  
Thomas M. Hamill ◽  
Julie K. Lundquist
Keyword(s):  
Wind Speed ◽  
Wind Power ◽  
Wind Farm ◽  
Weather Prediction ◽  
Model Performance ◽  
The United States ◽  
Numerical Weather Prediction Model ◽  
Post Processing ◽  
Skill Scores ◽  
Ramp Events

Abstract. Wind power forecasting is gaining international significance as more regions promote policies to increase the use of renewable energy. Wind ramps, large variations in wind power production during a period of minutes to hours, challenge utilities and electrical balancing authorities. A sudden decrease in wind energy production must be balanced by other power generators to meet energy demands, while a sharp increase in unexpected production results in excess power that may not be used in the power grid, leading to a loss of potential profits. In this study, we compare different methods to generate probabilistic ramp forecasts from the High Resolution Rapid Refresh (HRRR) numerical weather prediction model with up to twelve hours of lead time at two tall-tower locations in the United States. We validate model performance using 21 months of 80-m wind speed observations from towers in Boulder, Colorado and near the Columbia River Gorge in eastern Oregon. We employ four statistical post-processing methods, three of which are not currently used in the literature for wind forecasting. These procedures correct biases in the model and generate short-term wind speed scenarios which are then converted to power scenarios. This probabilistic enhancement of HRRR point forecasts provides valuable uncertainty information of ramp events and improves the skill of predicting ramp events over the raw forecasts. We compute Brier skill scores for each method at predicting up- and down-ramps to determine which method provides the best prediction. We find that the Standard Schaake Shuffle method yields the highest skill at predicting ramp events for these data sets, especially for up-ramp events at the Oregon site. Increased skill for ramp prediction is limited at the Boulder, CO site using any of the multivariate methods, because of the poor initial forecasts in this area of complex terrain. These statistical methods can be implemented by wind farm operators to generate a range of possible wind speed and power scenarios to aid and optimize decisions before ramp events occur.

scholarly journals Statistical post-processing of wind speed forecasts using convolutional neural networks

2021 ◽  
Author(s):  
Simon Veldkamp ◽  
Kirien Whan ◽  
Sjoerd Dirksen ◽  
Maurice Schmeits
Keyword(s):  
Neural Networks ◽  
Wind Speed ◽  
Convolutional Neural Networks ◽  
Weather Prediction ◽  
Estimation Methods ◽  
Post Processing ◽  
Skill Scores ◽  
Probabilistic Forecasts ◽  
Truncated Normal ◽  
Nwp Model

AbstractCurrent statistical post-processing methods for probabilistic weather forecasting are not capable of using full spatial patterns from the numerical weather prediction (NWP) model. In this paper we incorporate spatial wind speed information by using convolutional neural networks (CNNs) and obtain probabilistic wind speed forecasts in the Netherlands for 48 hours ahead, based on KNMI’s deterministic Harmonie-Arome NWP model. The probabilistic forecasts from the CNNs are shown to have higher Brier skill scores for medium to higher wind speeds, as well as a better continuous ranked probability score (CRPS) and logarithmic score, than the forecasts from fully connected neural networks and quantile regression forests. As a secondary result, we have compared the CNNs using 3 different density estimation methods (quantized softmax (QS), kernel mixture networks, and fitting a truncated normal distribution), and found the probabilistic forecasts based on the QS method to be best.

scholarly journals Test of a Poor Man’s Ensemble Prediction System for Short-Range Probability Forecasting

2005 ◽  
Vol 133 (7) ◽  
pp. 1825-1839 ◽  
Author(s):  
A. Arribas ◽  
K. B. Robertson ◽  
K. R. Mylne
Keyword(s):  
Short Range ◽  
Weather Prediction ◽  
Ensemble Prediction ◽  
Small Subset ◽  
Prediction System ◽  
Ensemble Forecasts ◽  
Sea Level Pressure ◽  
Ensemble Prediction System ◽  
To Come ◽  
Prediction Systems

Abstract Current operational ensemble prediction systems (EPSs) are designed specifically for medium-range forecasting, but there is also considerable interest in predictability in the short range, particularly for potential severe-weather developments. A possible option is to use a poor man’s ensemble prediction system (PEPS) comprising output from different numerical weather prediction (NWP) centers. By making use of a range of different models and independent analyses, a PEPS provides essentially a random sampling of both the initial condition and model evolution errors. In this paper the authors investigate the ability of a PEPS using up to 14 models from nine operational NWP centers. The ensemble forecasts are verified for a 101-day period and five variables: mean sea level pressure, 500-hPa geopotential height, temperature at 850 hPa, 2-m temperature, and 10-m wind speed. Results are compared with the operational ECMWF EPS, using the ECMWF analysis as the verifying “truth.” It is shown that, despite its smaller size, PEPS is an efficient way of producing ensemble forecasts and can provide competitive performance in the short range. The best relative performance is found to come from hybrid configurations combining output from a small subset of the ECMWF EPS with other different NWP models.

Post-processing for NWP Outputs Based on Machine Learning for 2022 Winter Olympics Games over Complex Terrain

2020 ◽  
Author(s):  
Kang Yanyan ◽  
Li Haochen ◽  
Xia Jiangjiang ◽  
Zhang Yingxin
Keyword(s):  
Machine Learning ◽  
Wind Speed ◽  
Complex Terrain ◽  
Weather Prediction ◽  
Grid Point ◽  
Forecast Accuracy ◽  
Post Processing ◽  
Winter Olympics ◽  
Forecast Time ◽  
Station Point

<p>    Weather forecasts play an important role in the Olympic game,especially the mountain snow projects, which will help to find a "window period" for the game. The winter Olympics track is located on very complex terrain, and a detailed weather forecast is needed. A Post-processing method based on machine learning is used for the future-10-days weather prediction with 1-km spatial resolution and 1-hour temporal resolution, which can greatly improve accuracy and refinement of numerical weather prediction(NWP). The ECWMF/RMAPS model data and the automatic weather station data(AWS) from 2015-2018 are prepared for the training data and test data, included 48 features and 4 labels (the observed 2m temperature, relative humidity , 10m wind speed and wind direction ). The model data are grid point, while the AWS data are station point. We take the nearest 9 model point to predict the station point, instead of making an interpolation between the grid point and station point. Then the feature number will be 48*9 in dataset. The interpolation error from grid point to station is eliminated,and the spatial distribution is considered to some extent. Machine leaning method we used are SVM, Random Forest, Gradient Boosting Decision Tree(GBDT) and XGBoost. We find that XGBoost method performs best, slightly better than GBDT and Random Forest. It is noted that we did some feature engineering work before training, and we found that it’s not that the more features, the better the model, while 10 features are enough. Also there is an interesting thing that the features that closely related the labels values becomes less important as the forecast time increases,such as the model outputed 2m temperature, 10m wind speed and wind direction. While some features that forecasters don’t pay attention to become more important in the 6-10 days prediction, such as latent heat flux, snow depth and so on. So it’s necessary to train the model based on dynamic weight parameters for different forecast time. Through the post-processing based on the machine learning method, the forecast accuracy has been greatly improved compared with EC model. The averaged forecast accuracy of 0-10 days for 2m relative humidity, 10m wind speed and direction has been increased by almost 15%, and the temperature accuracy has been increased by 20%~40% ( 40% for 0-3 days, and the accuracy decreased with the forecast time ). </p>

scholarly journals Generating wind power scenarios for probabilistic ramp event prediction using multivariate statistical post-processing

2018 ◽  
Vol 3 (1) ◽  
pp. 371-393 ◽  
Author(s):  
Rochelle P. Worsnop ◽  
Michael Scheuerer ◽  
Thomas M. Hamill ◽  
Julie K. Lundquist
Keyword(s):  
Wind Speed ◽  
Wind Power ◽  
Wind Farm ◽  
Weather Prediction ◽  
Model Performance ◽  
The United States ◽  
Numerical Weather Prediction Model ◽  
Post Processing ◽  
Skill Scores ◽  
Ramp Events

Abstract. Wind power forecasting is gaining international significance as more regions promote policies to increase the use of renewable energy. Wind ramps, large variations in wind power production during a period of minutes to hours, challenge utilities and electrical balancing authorities. A sudden decrease in wind-energy production must be balanced by other power generators to meet energy demands, while a sharp increase in unexpected production results in excess power that may not be used in the power grid, leading to a loss of potential profits. In this study, we compare different methods to generate probabilistic ramp forecasts from the High Resolution Rapid Refresh (HRRR) numerical weather prediction model with up to 12 h of lead time at two tall-tower locations in the United States. We validate model performance using 21 months of 80 m wind speed observations from towers in Boulder, Colorado, and near the Columbia River gorge in eastern Oregon. We employ four statistical post-processing methods, three of which are not currently used in the literature for wind forecasting. These procedures correct biases in the model and generate short-term wind speed scenarios which are then converted to power scenarios. This probabilistic enhancement of HRRR point forecasts provides valuable uncertainty information of ramp events and improves the skill of predicting ramp events over the raw forecasts. We compute Brier skill scores for each method with regard to predicting up- and down-ramps to determine which method provides the best prediction. We find that the Standard Schaake shuffle method yields the highest skill at predicting ramp events for these datasets, especially for up-ramp events at the Oregon site. Increased skill for ramp prediction is limited at the Boulder, CO, site using any of the multivariate methods because of the poor initial forecasts in this area of complex terrain. These statistical methods can be implemented by wind farm operators to generate a range of possible wind speed and power scenarios to aid and optimize decisions before ramp events occur.

scholarly journals On the Orthogonalization of Bred Vectors

2010 ◽  
Vol 25 (4) ◽  
pp. 1219-1234 ◽  
Author(s):  
Jan D. Keller ◽  
Andreas Hense ◽  
Luis Kornblueh ◽  
Andreas Rhodin
Keyword(s):  
Weather Prediction ◽  
Numerical Weather Prediction Model ◽  
Ensemble Prediction ◽  
Ensemble Forecasts ◽  
Forecast Performance ◽  
Ensemble Prediction System ◽  
Study Results ◽  
Bred Vectors ◽  
Forecasting System ◽  
Value Decomposition

Abstract The key to the improvement of the quality of ensemble forecasts assessing the inherent flow uncertainties is the choice of the initial ensemble perturbations. To generate such perturbations, the breeding of growing modes approach has been used for the past two decades. Here, the fastest-growing error modes of the initial model state are estimated. However, the resulting bred vectors (BVs) mainly point in the phase space direction of the leading Lyapunov vector and therefore favor one direction of growing errors. To overcome this characteristic and obtain growing modes pointing to Lyapunov vectors different from the leading one, an orthogonalization implemented as a singular value decomposition based on the similarity between the BVs is applied. This transformation is similar to that used in the ensemble transform technique currently in operational use at NCEP but with certain differences in the metric used and in the implementation. In this study, results of this approach using BVs generated in the Ensemble Forecasting System (EFS) based on the global numerical weather prediction model GME of the German Meteorological Service are presented. The gain in forecast performance achieved with the orthogonalized BV initialization is shown by using different probabilistic forecast scores evaluating ensemble reliability, variance, and resolution. For a 3-month period in summer 2007, the results are compared to forecasts generated with simple BV initializations of the same ensemble prediction system as well as operational ensemble forecasts from ECMWF and NCEP. The orthogonalization vastly improves the GME–EFS scores and makes them competitive with the two other centers.

scholarly journals Impact of Surface Parameter Uncertainties within the Canadian Regional Ensemble Prediction System

2013 ◽  
Vol 141 (5) ◽  
pp. 1506-1526 ◽  
Author(s):  
Christophe Lavaysse ◽  
Marco Carrera ◽  
Stéphane Bélair ◽  
Normand Gagnon ◽  
Ronald Frenette ◽  
...  
Keyword(s):  
Wind Speed ◽  
Weather Prediction ◽  
Initial Time ◽  
Ensemble Prediction ◽  
Prediction System ◽  
Surface Parameter ◽  
Ensemble Prediction System ◽  
Surface Parameters ◽  
Screen Level ◽  
The Impact

Abstract The aim of this study is to assess the impact of uncertainties in surface parameter and initial conditions on numerical prediction with the Canadian Regional Ensemble Prediction System (REPS). As part of this study, the Canadian version of the Interactions between Soil–Biosphere–Atmosphere (ISBA) land surface scheme has been coupled to Environment Canada’s numerical weather prediction model within the REPS. For 20 summer periods in 2009, stochastic perturbations of surface parameters have been generated in several experiments. Each experiment corresponds to 20 simulations differing by the perturbations at the initial time of one or several surface parameters or prognostic variables. The sensitivity to these perturbations is quantified especially for 2-m temperature, 10-m wind speed, cloud fraction, and precipitation up to 48-h lead time. Spatial variability of these sensitivities over the North American continent shows that soil moisture, albedo, leaf area index, and SST have the largest impacts on the screen-level variables. The temporal evolution of these sensitivities appears to be closely linked to the diurnal cycle of the boundary layer. The surface perturbations are shown to increase the ensemble spread of the REPS for all screen-level variables especially for 2-m temperature and 10-m wind speed during daytime. A preliminary study of the impact on the ensemble forecast has shown that the inclusion of the surface perturbations tends to significantly increase the 2-m temperature and 10-m wind speed skill.

Sign in / Sign up

Export Citation Format

Share Document