Numerical Weather Forecast Post-processing with Ensemble Learning and Transfer Learning

2020 ◽  
Author(s):  
Yuwen Chen ◽  
Xiaomeng Huang
Keyword(s):  
Machine Learning ◽  
Transfer Learning ◽  
Ensemble Learning ◽  
Weather Forecasting ◽  
Weather Forecast ◽  
Processing Temperature ◽  
Forecast Errors ◽  
Post Processing ◽  
Numerical Weather ◽  
One Year

<p>Statistical approaches have been used for decades to augment and interpret numerical weather forecasts. The emergence of artificial intelligence algorithms has provided new perspectives in this field, but the extension of algorithms developed for station networks with rich historical records to include newly-built stations remains a challenge. To address this, we design a framework that combines two machine learning methods: temperature prediction based on ensemble of multiple machine learning models and transfer learning for newly-built stations. We then evaluate this framework by post-processing temperature forecasts provided by a leading weather forecast center and observations from 301 weather stations in China. Station clustering reduces forecast errors by 24.4% averagely, while transfer learning improves predictions by 13.4% for recently-built sites with only one year of data available. This work demonstrates how ensemble learning and transfer learning can be used to supplement weather forecasting.</p><p></p>

Improving machine learning-based weather forecast post-processing with clustering and transfer learning

2020 ◽  
Author(s):  
Xiaomeng Huang ◽  
Yuwen Chen ◽  
Yi Li ◽  
Yue Chen ◽  
Chi Yan Tsui ◽  
...  
Keyword(s):  
Machine Learning ◽  
Transfer Learning ◽  
Weather Forecast ◽  
Post Processing

scholarly journals Improving machine learning-based weather forecast 1 post-processing with clustering and transfer learning

2020 ◽  
Author(s):  
Xiaomeng Huang ◽  
Yuwen Chen ◽  
Yi Li ◽  
Yue Chen ◽  
Chi Yan Tsui ◽  
...  
Keyword(s):  
Machine Learning ◽  
Transfer Learning ◽  
Weather Forecast ◽  
Post Processing

Exploring the potential of vegetation information for improving weather forecast performance

2021 ◽  
Author(s):  
Melissa Ruiz ◽  
Sungmin Oh ◽  
Rene Orth ◽  
Gianpaolo Balsamo
Keyword(s):  
Land Surface ◽  
Vegetation Index ◽  
Weather Forecasting ◽  
Forecast Accuracy ◽  
Weather Forecast ◽  
Spatial And Temporal Variation ◽  
Forecast Errors ◽  
Forecast Performance ◽  
Forecast Models ◽  
Moderate Resolution Imaging Spectroradiometer

<p>The quality of weather forecasts is continuously improving for decades. However, increases in forecast skills have slowed down in recent years. This highlights the importance of exploring new avenues towards future forecast system improvements. Until now, (near) real-time information on vegetation anomalies is not used in most forecasting models. Addressing this gap, we explore the potential of the vegetation state for explaining the spatial and temporal variation in forecast accuracy globally across climate regions, seasons, and vegetation types. For this purpose, we employ re-forecasts from the European Centre of Medium-Range Weather Forecasting (ECMWF) and infer the vegetation status through the Enhanced Vegetation Index derived from the Moderate-Resolution Imaging Spectroradiometer (MODIS) satellite observations during the 2000-2019 period. In particular, we focus on land surface variables such as evaporation and temperature to study the relationship between forecast errors and vegetation anomalies.</p><p>The results show a stronger correlation between forecast errors and vegetation anomalies in semi-arid and sub-humid regions during the growing season, which highlights that vegetation information has the potential to help advance weather forecast performance. To put these results into perspective, we will further perform a multivariate analysis to determine the relative roles of vegetation, hydrology and climate in explaining weather forecast errors. Thereby, our results can inform the future development of weather forecast models and underlying data assimilation schemes.</p>

One Year of Operational Numerical Weather Prediction, Part II *

1957 ◽  
Vol 38 (6) ◽  
pp. 315-328 ◽  
Keyword(s):  
Numerical Weather Prediction ◽  
Weather Forecasting ◽  
Weather Prediction ◽  
Numerical Weather ◽  
Level Data ◽  
Analysis Center ◽  
Numerical Weather Forecasting ◽  
One Year ◽  
Near Future

This is the second of two brief reports on the activities and results of the Joint Numerical Weather Prediction Unit since May 1955, and is concerned primarily with the accuracy and characteristic errors of the numerical forecasts described in the previous report. The quality of the barotropic and 3-level forecasts has been measured by several statistical indices of error, and compared with that of the subjective forecasts issued by the National Weather Analysis Center. A breakdown of these statistics shows the dependence of forecasting accuracy on length of forecast period, level, data coverage, and proximity of lateral boundaries. Various sources of systematic error are discussed with reference to the JNWP Unit's efforts to isolate and remedy them. After almost a year of experimentation and operational numerical weather forecasting, it is concluded that the quality of the numerical 500 millibar forecasts is not significantly different from that of the best subjective forecasts prepared by methods in current use. Recent results indicate that a significant improvement can be expected in the near future. The numerical 1000 mb forecasts are worse, but recent changes of model show promise of matching the performance of subjective methods. Finally, the most glaring systematic errors of the present numerical forecasts have adequate explanation in existing theory, and can be (or have already been) corrected by generalization of the models.

scholarly journals Data-Parallel Numerical Weather Forecasting

1995 ◽  
Vol 4 (3) ◽  
pp. 141-153 ◽  
Author(s):  
Lex Wolters ◽  
Gerard Cats ◽  
Nils Gustafsson
Keyword(s):  
Message Passing ◽  
Weather Forecasting ◽  
Weather Forecast ◽  
Forecast Model ◽  
Parallel Computer ◽  
Numerical Weather ◽  
Data Parallel ◽  
Production Code ◽  
Numerical Solution Methods ◽  
Solution Methods

In this article we describe the implementation of a numerical weather forecast model on a massively parallel computer system. This model is a production code used for routine weather forecasting at the meteorological institutes of several European countries. The modifications needed to achieve a data-parallel version of this model without explicit message passing are outlined. The achieved performance of different numerical solution methods within this model is presented and compared.

scholarly journals Artificial Learning Dispatch Planning with Probabilistic Forecasts: Using Uncertainties as an Asset

Energies ◽  
2020 ◽  
Vol 13 (3) ◽  
pp. 616
Author(s):  
Ana Carolina do Amaral Burghi ◽  
Tobias Hirsch ◽  
Robert Pitz-Paal
Keyword(s):  
Machine Learning ◽  
Learning Algorithm ◽  
Weather Forecast ◽  
Machine Learning Algorithm ◽  
Post Processing ◽  
Weather Forecasts ◽  
Additional Information ◽  
Model Following ◽  
Probabilistic Forecasts ◽  
Artificial Learning

Weather forecast uncertainty is a key element for energy market volatility. By intelligently considering uncertainties on the schedule development, renewable energy systems with storage could improve dispatching accuracy, and therefore, effectively participate in electricity wholesale markets. Deterministic forecasts have been traditionally used to support dispatch planning, representing reduced or no uncertainty information about the future weather. Aiming at better representing the uncertainties involved, probabilistic forecasts have been developed to increase forecasting accuracy. For the dispatch planning, this can highly influence the development of a more precise schedule. This work extends a dispatch planning method to the use of probabilistic weather forecasts. The underlying method used a schedule optimizer coupled to a post-processing machine learning algorithm. This machine learning algorithm was adapted to include probabilistic forecasts, considering their additional information on uncertainties. This post-processing applied a calibration of the planned schedule considering the knowledge about uncertainties obtained from similar past situations. Simulations performed with a concentrated solar power plant model following the proposed strategy demonstrated promising financial improvement and relevant potential in dealing with uncertainties. Results especially show that information included in probabilistic forecasts can increase financial revenues up to 15% (in comparison to a persistence solar driven approach) if processed in a suitable way.

scholarly journals Machine learning emulation of gravity wave drag in numerical weather forecasting

2021 ◽  
Author(s):  
Matthew Chantry ◽  
Sam Hatfield ◽  
Peter Duben ◽  
Inna Polichtchouk ◽  
Tim Palmer
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Data Assimilation ◽  
Gravity Wave ◽  
Weather Forecasting ◽  
Wave Drag ◽  
Seasonal Forecasting ◽  
Numerical Weather ◽  
Forecasting System ◽  
Numerical Weather Forecasting

<p>We assess the value of machine learning as an accelerator for a kernel of an operational weather forecasting system, specifically the parameterisation of non-orographic gravity wave drag. Emulators of this scheme can be trained that produce stable and accurate results up to seasonal forecasting timescales. By training on an increased complexity version of the parameterisation scheme we build emulators that produce more accurate forecasts than the existing parameterisation scheme. Leveraging the differentiability of neural networks we generate tangent linear and adjoint versions of our parameterisation, key components in 4D-var data-assimilation. We test our tangent linear and adjoint codes within an operational-like 4D-var setup and find no degradation in skill vs hand-written tangent-linear and adjoint codes.</p>

scholarly journals A Model Output Deep Learning Method for Grid Temperature Forecasts in Tianjin Area

2020 ◽  
Vol 10 (17) ◽  
pp. 5808 ◽  
Author(s):  
Keran Chen ◽  
Ping Wang ◽  
Xiaojun Yang ◽  
Nan Zhang ◽  
Di Wang
Keyword(s):  
Temperature Field ◽  
Deep Learning ◽  
Numerical Weather Prediction ◽  
Weather Forecasting ◽  
Weather Prediction ◽  
Physical Models ◽  
Model Output ◽  
Post Processing ◽  
Numerical Weather ◽  
Convolution Model

In weather forecasting, numerical weather prediction (NWP) that is based on physical models requires proper post-processing before it can be applied to actual operations. Therefore, research on intelligent post-processing algorithms has always been an important topic in this field. This paper proposes a model output deep learning (MODL) method for post-processing, which can improve the forecast effect of numerical weather prediction. MODL is an end-to-end post-processing method based on deep convolutional neural network, which directly learns the mapping relationship between the forecast fields output by numerical model and the observation temperature field in order to obtain more accurate temperature forecasts. MODL modifies the existing deep convolution model according to the post-processing problem’s characteristics, thereby improving the performance of the weather forecast. This paper uses The International Grand Global Ensemble (TIGGE) dataset from European Centre for Medium-Range Weather Forecasts (ECMWF) and the observed air temperature of 2 m obtained from Tianjin meteorological station in order to test the post-processing performance of MODL. The MODL method applied to temperature in post-processing is compared with the ECMWF forecast, Model Output Statistics (MOS) methods, and Model Output Machine Learning (MOML) methods. The Root Mean Square Error (RMSE) of the temperature field predicted by MODL and the observed temperature field is smaller than the other models and the accuracy of the temperature difference of 2 °C (Acc) is higher, especially where the prediction time is in the first three days. The lightweight nature of MODL also makes it suitable for most operations.

scholarly journals A framework for probabilistic weather forecast post-processing across models and lead times using machine learning

2021 ◽  
Vol 379 (2194) ◽  
pp. 20200099 ◽  
Author(s):  
Charlie Kirkwood ◽  
Theo Economou ◽  
Henry Odbert ◽  
Nicolas Pugeault
Keyword(s):  
Machine Learning ◽  
Decision Support ◽  
Probability Distributions ◽  
Weather Prediction ◽  
Weather Forecast ◽  
Predictive Distribution ◽  
Lead Times ◽  
Climate Modelling ◽  
Post Processing ◽  
Probabilistic Forecasts

Forecasting the weather is an increasingly data-intensive exercise. Numerical weather prediction (NWP) models are becoming more complex, with higher resolutions, and there are increasing numbers of different models in operation. While the forecasting skill of NWP models continues to improve, the number and complexity of these models poses a new challenge for the operational meteorologist: how should the information from all available models, each with their own unique biases and limitations, be combined in order to provide stakeholders with well-calibrated probabilistic forecasts to use in decision making? In this paper, we use a road surface temperature example to demonstrate a three-stage framework that uses machine learning to bridge the gap between sets of separate forecasts from NWP models and the ‘ideal’ forecast for decision support: probabilities of future weather outcomes. First, we use quantile regression forests to learn the error profile of each numerical model, and use these to apply empirically derived probability distributions to forecasts. Second, we combine these probabilistic forecasts using quantile averaging. Third, we interpolate between the aggregate quantiles in order to generate a full predictive distribution, which we demonstrate has properties suitable for decision support. Our results suggest that this approach provides an effective and operationally viable framework for the cohesive post-processing of weather forecasts across multiple models and lead times to produce a well-calibrated probabilistic output. This article is part of the theme issue ‘Machine learning for weather and climate modelling’.

Sign in / Sign up

Export Citation Format

Share Document