scholarly journals Seamless postprocessing of multi-model NWP surface wind forecasts with deep learning

2021 ◽  
Author(s):  
Daniele Nerini ◽  
Jonas Bhend ◽  
Christoph Spirig ◽  
Lionel Moret ◽  
Mark Liniger
Keyword(s):  
Deep Learning ◽  
Prediction Models ◽  
Weather Prediction ◽  
Surface Wind ◽  
Ensemble Forecasting ◽  
Added Value ◽  
Random Errors ◽  
Ensemble Forecasts ◽  
Late Model ◽  
Set Up

<p>Hourly wind forecasts from numerical weather prediction models suffer from a range of systematic and random errors that are to a great extent related to limitations in the model grid resolution. To correct for such biases, statistical postprocessing and downscaling procedures are commonly applied so to leverage the information provided by automatic wind measurements at the surface. More recently, such techniques have been reformulated in a machine learning framework so to profit from the increased availability of data and computational resources. The results reported in the literature are promising and call for a serious evaluation of their potential for operational forecasting.</p><p>However, there remain several scientific and more applied challenges that need to be addressed before such methods can transition to real-world applications. One such challenge relates to the availability of multiple ensemble forecasts for the same point in time and space, which raises the question of how the information can be efficiently and optimally handled during postprocessing, so to provide added value to the end-user without adding technical debt to the operational system.</p><p>We propose an approach where a single deep learning model is trained to postprocess a combination of three ensemble forecasting systems, namely the high-resolution regional COSMO model with two configurations, and the ECMWF IFS ENS global ensemble forecasting system. We will show how the training is set up to provide a robust postprocessing model that can account for real time scenarios that include missing data and late model runs, while the quality of the forecasts remains comparable to a single-model approach. We found that the flexibility of the deep learning architecture translates into a robust automatic postprocessing solution that limits the maintenance burden and improves the system’s reliability.</p>

scholarly journals A Transformed Lagged Ensemble Forecasting Technique for Increasing Ensemble Size

2007 ◽  
Vol 135 (4) ◽  
pp. 1424-1438 ◽  
Author(s):  
Andrew R. Lawrence ◽  
James A. Hansen
Keyword(s):  
Prediction Models ◽  
Weather Prediction ◽  
Ensemble Forecast ◽  
Ensemble Forecasting ◽  
Limited Area ◽  
Ensemble Size ◽  
Probabilistic Forecasting ◽  
Ensemble Forecasts ◽  
Chaotic Model ◽  
The Impact

Abstract An ensemble-based data assimilation approach is used to transform old ensemble forecast perturbations with more recent observations for the purpose of inexpensively increasing ensemble size. The impact of the transformations are propagated forward in time over the ensemble’s forecast period without rerunning any models, and these transformed ensemble forecast perturbations can be combined with the most recent ensemble forecast to sensibly increase forecast ensemble sizes. Because the transform takes place in perturbation space, the transformed perturbations must be centered on the ensemble mean from the most recent forecasts. Thus, the benefit of the approach is in terms of improved ensemble statistics rather than improvements in the mean. Larger ensemble forecasts can be used for numerous purposes, including probabilistic forecasting, targeted observations, and to provide boundary conditions to limited-area models. This transformed lagged ensemble forecasting approach is explored and is shown to give positive results in the context of a simple chaotic model. By incorporating a suitable perturbation inflation factor, the technique was found to generate forecast ensembles whose skill were statistically comparable to those produced by adding nonlinear model integrations. Implications for ensemble forecasts generated by numerical weather prediction models are briefly discussed, including multimodel ensemble forecasting.

scholarly journals Understanding the relationship between clouds and surface downward radiation forecast errors with Unsupervised Deep Learning

2021 ◽  
Author(s):  
Matthias Zech ◽  
Lueder von Bremen
Keyword(s):  
Deep Learning ◽  
Unsupervised Learning ◽  
Prediction Models ◽  
Weather Prediction ◽  
Reanalysis Data ◽  
Cloud Formation ◽  
Trade Wind ◽  
Forecast Errors ◽  
Model Learning ◽  
Unsupervised Deep Learning

<p>Cloudiness is a difficult parameter to forecast and has improved relatively little over the last decade in numerical weather prediction models as the EMCWF IFS. However, surface downward solar radiation forecast (ssrd) errors are becoming more important with higher penetration of photovoltaics in Europe as forecasts errors induce power imbalances that might lead to high balancing costs. This study continues recent approaches to better understand clouds using satellite images with Deep Learning. Unlike other studies which focus on shallow trade wind cumulus clouds over the ocean, this study investigates the European land area. To better understand the clouds, we use the daily MODIS optical cloud thickness product which shows both water and ice phase of the cloud. This allows to consider both cloud structure and cloud formation during learning. It is also much easier to distinguish between snow and cloud in contrast to using visible bands. Methodologically, it uses the Unsupervised Learning approach <em>tile2vec</em> to derive a lower dimensional representation of the clouds. Three cloud regions with two similar neighboring tiles and one tile from a different time and location are sampled to learn lower-rank embeddings. In contrast to the initial <em>tile2vec</em> implementation, this study does not sample arbitrarily distant tiles but uses the fractal dimension of the clouds in a pseudo-random sampling fashion to improve model learning.</p><p>The usefulness of the cloud segments is shown by applying them in a case study to investigate statistical properties of ssrd forecast errors over Europe which are derived from hourly ECMWF IFS forecasts and ERA5 reanalysis data. This study shows how Unsupervised Learning has high potential despite its relatively low usage compared to Supervised Learning in academia. It further shows, how the generated land cloud product can be used to better characterize ssrd forecast errors over Europe.</p>

scholarly journals Ensemble Hail Prediction for the Storms of 10 May 2010 in South-Central Oklahoma Using Single- and Double-Moment Microphysical Schemes

2017 ◽  
Vol 145 (12) ◽  
pp. 4911-4936 ◽  
Author(s):  
Jonathan Labriola ◽  
Nathan Snook ◽  
Youngsun Jung ◽  
Bryan Putnam ◽  
Ming Xue
Keyword(s):  
Prediction Models ◽  
Weather Prediction ◽  
Radar Data ◽  
Ensemble Forecasts ◽  
South Central ◽  
Single Moment ◽  
Double Moment ◽  
Horizontal Grid ◽  
Severe Weather Event ◽  
Numerical Weather Prediction Models

Explicit prediction of hail using numerical weather prediction models remains a significant challenge; microphysical uncertainties and errors are a significant contributor to this challenge. This study assesses the ability of storm-scale ensemble forecasts using single-moment Lin or double-moment Milbrandt and Yau microphysical schemes in predicting hail during a severe weather event over south-central Oklahoma on 10 May 2010. Radar and surface observations are assimilated using an ensemble Kalman filter (EnKF) at 5-min intervals. Three sets of ensemble forecasts, launched at 15-min intervals, are then produced from EnKF analyses at times ranging from 30 min prior to the first observed hail to the time of the first observed hail. Forty ensemble members are run at 500-m horizontal grid spacing in both EnKF assimilation cycles and subsequent forecasts. Hail forecasts are verified using radar-derived products including information from single- and dual-polarization radar data: maximum estimated size of hail (MESH), hydrometeor classification algorithm (HCA) output, and hail size discrimination algorithm (HSDA) output. Resulting hail forecasts show at most marginal skill, with the level of skill dependent on the forecast initialization time and microphysical scheme used. Forecasts using the double-moment scheme predict many small hailstones aloft, while the single-moment members predict larger hailstones. Near the surface, double-moment members predict larger hailstone sizes than their single-member counterparts. Hail in the forecasts is found to melt too quickly near the surface for members using either of the microphysics schemes examined. Analysis of microphysical budgets in both schemes indicates that both schemes suboptimally represent hail processes, adversely impacting the skill of surface hail forecasts.

Temporal Changes in Wind as Objects for Evaluating Mesoscale Numerical Weather Prediction

2009 ◽  
Vol 24 (5) ◽  
pp. 1374-1389 ◽  
Author(s):  
Daran L. Rife ◽  
Christopher A. Davis ◽  
Jason C. Knievel
Keyword(s):  
Numerical Weather Prediction ◽  
Prediction Models ◽  
Mesoscale Model ◽  
Weather Prediction ◽  
Grid Point ◽  
Surface Wind ◽  
Temporal Changes ◽  
High Temporal Resolution ◽  
Numerical Weather ◽  
Wind Events

Abstract The study describes a method of evaluating numerical weather prediction models by comparing the characteristics of temporal changes in simulated and observed 10-m (AGL) winds. The method is demonstrated on a 1-yr collection of 1-day simulations by the fifth-generation Pennsylvania State University–National Center for Atmospheric Research Mesoscale Model (MM5) over southern New Mexico. Temporal objects, or wind events, are defined at the observation locations and at each grid point in the model domain as vector wind changes over 2 h. Changes above the uppermost quartile of the distributions in the observations and simulations are empirically classified as significant; their attributes are analyzed and interpreted. It is demonstrated that the model can discriminate between large and modest wind changes on a pointwise basis, suggesting that many forecast events have an observational counterpart. Spatial clusters of significant wind events are highly continuous in space and time. Such continuity suggests that displaying maps of surface wind changes with high temporal resolution can alert forecasters to the occurrence of important phenomena. Documented systematic errors in the amplitude, direction, and timing of wind events will allow forecasters to mentally adjust for biases in features forecast by the model.

Towards operational postprocessing of cloud cover at MeteoSwiss

2020 ◽  
Author(s):  
Stephan Hemri ◽  
Christoph Spirig ◽  
Jonas Bhend ◽  
Lionel Moret ◽  
Mark Liniger
Keyword(s):  
Cloud Cover ◽  
Prediction Models ◽  
Weather Prediction ◽  
Weather Forecast ◽  
Forecast Errors ◽  
Observation Data ◽  
Model Output ◽  
Ensemble Forecasts ◽  
Direct Model ◽  
Ensemble Model Output Statistics

<p>Over the last decades ensemble approaches have become state-of-the-art for the quantification of weather forecast uncertainty. Despite ongoing improvements, ensemble forecasts issued by numerical weather prediction models (NWPs) still tend to be biased and underdispersed. Statistical postprocessing has proven to be an appropriate tool to correct biases and underdispersion, and hence to improve forecast skill. Here we focus on multi-model postprocessing of cloud cover forecasts in Switzerland. In order to issue postprocessed forecasts at any point in space, ensemble model output statistics (EMOS) models are trained and verified against EUMETSAT CM SAF satellite data with a spatial resolution of around 2 km over Switzerland. Training with a minimal record length of the past 45 days of forecast and observation data already produced an EMOS model improving direct model output (DMO). Training on a 3 years record of the corresponding season further improved the performance. We evaluate how well postprocessing corrects the most severe forecast errors, like missing fog and low level stratus in winter. For such conditions, postprocessing of cloud cover benefits strongly from incorporating additional predictors into the postprocessing suite. A quasi-operational prototype has been set up and was used to explore meteogram-like visualizations of probabilistic cloud cover forecasts.</p>

scholarly journals A Similarity-Based Implementation of the Schaake Shuffle

2016 ◽  
Vol 144 (5) ◽  
pp. 1909-1921 ◽  
Author(s):  
Roman Schefzik
Keyword(s):  
Prediction Models ◽  
Initial Conditions ◽  
Weather Prediction ◽  
Similarity Criterion ◽  
Ensemble Prediction ◽  
Ensemble Forecasts ◽  
Weather Forecasts ◽  
Prediction Horizon ◽  
Prediction Systems ◽  
Specific Implementation

Contemporary weather forecasts are typically based on ensemble prediction systems, which consist of multiple runs of numerical weather prediction models that vary with respect to the initial conditions and/or the parameterization of the atmosphere. Ensemble forecasts are frequently biased and show dispersion errors and thus need to be statistically postprocessed. However, current postprocessing approaches are often univariate and apply to a single weather quantity at a single location and for a single prediction horizon only, thereby failing to account for potentially crucial dependence structures. Nonparametric multivariate postprocessing methods based on empirical copulas, such as ensemble copula coupling or the Schaake shuffle, can address this shortcoming. A specific implementation of the Schaake shuffle, called the SimSchaake approach, is introduced. The SimSchaake method aggregates univariately postprocessed ensemble forecasts using dependence patterns from past observations. Specifically, the observations are taken from historical dates at which the ensemble forecasts resembled the current ensemble prediction with respect to a specific similarity criterion. The SimSchaake ensemble outperforms all reference ensembles in an application to ensemble forecasts for 2-m temperature from the European Centre for Medium-Range Weather Forecasts.

scholarly journals Object-Based Analog Forecasts for Surface Wind Speed

2017 ◽  
Vol 145 (12) ◽  
pp. 5083-5102 ◽  
Author(s):  
Maria E. B. Frediani ◽  
Thomas M. Hopson ◽  
Joshua P. Hacker ◽  
Emmanouil N. Anagnostou ◽  
Luca Delle Monache ◽  
...  
Keyword(s):  
Wind Speed ◽  
Surface Wind ◽  
Surface Wind Speed ◽  
Selection Combining ◽  
Random Errors ◽  
Training Set ◽  
Ensemble Forecasts ◽  
Object Based ◽  
Object Area ◽  
Spatial Dimensionality

Analogs are used as a forecast postprocessing technique, in which a statistical forecast is derived from past prognostic states. This study proposes a method to identify analogs through spatial objects, which are then used to create forecast ensembles. The object-analog technique preserves the field’s spatial relationships, reduces spatial dimensionality, and consequently facilitates the use of artificial intelligence algorithms to improve analog selection. Forecast objects are created with a three-step object selection, combining standard image processing algorithms. The resulting objects are used to find similar forecasts in a training set with a similarity measure based on object area intersection and magnitude. Storm-induced power outages in the Northeast United States motivated the method’s validation for 10-m AGL wind speed forecasts. The training set comprises reforecasts and reanalyses of events that caused damages to the utility infrastructure. The corresponding reanalyses of the best reforecast analogs are used to produce the object-analog ensemble forecasts. The forecasts are compared with other analog forecast methods. Analogs representing lower and upper predictability limits provide references to distinguish the method’s ability (to find good analogs) from the training set’s ability (to provide good analogs) to generate skillful ensemble forecasts. The object-analog forecasts are competitively skillful compared to simpler analog techniques with an advantage of lower spatial dimensionality, while generating reliable ensemble forecasts, with reduced systematic and random errors, maintaining correlation, and improving Brier scores.

scholarly journals Ensemble Forecasting Frame Based on Deep Learning and Multi-Objective Optimization for Planning Solar Energy Management: A Case Study

2021 ◽  
Vol 9 ◽  
Author(s):  
Yongjiu Liu ◽  
Li Li ◽  
Shenglin Zhou
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Learning Strategies ◽  
High Efficiency ◽  
Prediction Models ◽  
Ensemble Forecasting ◽  
Multi Objective Optimization ◽  
Multi Objective ◽  
Photovoltaic Power ◽  
Power Forecasting

There are many prediction models that have been adopted to predict uncertain and non-linear photovoltaic power time series. Nonetheless, most models neglected the validity of data preprocessing and ensemble learning strategies, which leads to low forecasting precision and low stability of photovoltaic power. To effectively enhance photovoltaic power forecasting accuracy and stability, an ensemble forecasting frame based on the data pretreatment technology, multi-objective optimization algorithm, statistical method, and deep learning methods is developed. The proposed forecasting frame successfully integrates the advantages of multiple algorithms and validly depict the linear and nonlinear characteristic of photovoltaic power time series, which is conductive to achieving accurate and stable photovoltaic power forecasting results. Three datasets of 15-min photovoltaic power output data obtained from different time periods in Belgium were employed to verify the validity of the proposed system. The simulation results prove that the proposed forecasting frame positively surpasses all comparative hybrid models, ensemble models, and classical models in terms of prediction accuracy and stabilization. For one-, two-, and three-step predictions, the MAPE values obtained from the proposed frame were less than 2, 3, and 5%, respectively. Discussion results also verify that the proposed forecasting frame is obviously different from other comparative models, and is more stable and high-efficiency. Thus, the proposed frame is highly serviceable in elevating photovoltaic power forecasting performance and can be used as an efficient instrument for intelligent grid programming.

Sign in / Sign up

Export Citation Format

Share Document