Sub‐seasonal forecasting with a large ensemble of deep‐learning weather prediction models

Cloudiness is a difficult parameter to forecast and has improved relatively little over the last decade in numerical weather prediction models as the EMCWF IFS. However, surface downward solar radiation forecast (ssrd) errors are becoming more important with higher penetration of photovoltaics in Europe as forecasts errors induce power imbalances that might lead to high balancing costs. This study continues recent approaches to better understand clouds using satellite images with Deep Learning. Unlike other studies which focus on shallow trade wind cumulus clouds over the ocean, this study investigates the European land area. To better understand the clouds, we use the daily MODIS optical cloud thickness product which shows both water and ice phase of the cloud. This allows to consider both cloud structure and cloud formation during learning. It is also much easier to distinguish between snow and cloud in contrast to using visible bands. Methodologically, it uses the Unsupervised Learning approach tile2vec to derive a lower dimensional representation of the clouds. Three cloud regions with two similar neighboring tiles and one tile from a different time and location are sampled to learn lower-rank embeddings. In contrast to the initial tile2vec&#160;implementation, this study does not sample arbitrarily distant tiles but uses the fractal dimension of the clouds in a pseudo-random sampling fashion to improve model learning.The usefulness of the cloud segments is shown by applying them in a case study to investigate statistical properties of ssrd forecast errors over Europe which are derived from hourly ECMWF IFS forecasts and ERA5 reanalysis data. This study shows how Unsupervised Learning has high potential despite its relatively low usage compared to Supervised Learning in academia. It further shows, how the generated land cloud product can be used to better characterize ssrd forecast errors over Europe.

Download Full-text

A Deep Learning Method for Bias Correction of ECMWF 24–240 h Forecasts

Advances in Atmospheric Sciences ◽

10.1007/s00376-021-0215-y ◽

2021 ◽

Author(s):

Lei Han ◽

Mingxuan Chen ◽

Kangkai Chen ◽

Haonan Chen ◽

Yanbiao Zhang ◽

...

Keyword(s):

Deep Learning ◽

Relative Humidity ◽

Wind Direction ◽

Prediction Models ◽

Weather Prediction ◽

Ground Truth ◽

Learning Method ◽

Weather Variables ◽

Medium Range Weather Forecast ◽

Forecast Lead Time

AbstractCorrecting the forecast bias of numerical weather prediction models is important for severe weather warnings. The refined grid forecast requires direct correction on gridded forecast products, as opposed to correcting forecast data only at individual weather stations. In this study, a deep learning method called CU-net is proposed to correct the gridded forecasts of four weather variables from the European Centre for Medium-Range Weather Forecast Integrated Forecasting System global model (ECMWF-IFS): 2-m temperature, 2-m relative humidity, 10-m wind speed, and 10-m wind direction, with a forecast lead time of 24 h to 240 h in North China. First, the forecast correction problem is transformed into an image-to-image translation problem in deep learning under the CU-net architecture, which is based on convolutional neural networks. Second, the ECMWF-IFS forecasts and ECMWF reanalysis data (ERA5) from 2005 to 2018 are used as training, validation, and testing datasets. The predictors and labels (ground truth) of the model are created using the ECMWF-IFS and ERA5, respectively. Finally, the correction performance of CU-net is compared with a conventional method, anomaly numerical correction with observations (ANO). Results show that forecasts from CU-net have lower root mean square error, bias, mean absolute error, and higher correlation coefficient than those from ANO for all forecast lead times from 24 h to 240 h. CU-net improves upon the ECMWF-IFS forecast for all four weather variables in terms of the above evaluation metrics, whereas ANO improves upon ECMWF-IFS performance only for 2-m temperature and relative humidity. For the correction of the 10-m wind direction forecast, which is often difficult to achieve, CU-net also improves the correction performance.

Download Full-text

RN-Net: A Deep Learning Approach to 0–2 h Rainfall Nowcasting Based on Radar and Automatic Weather Station Data

Sensors ◽

10.3390/s21061981 ◽

2021 ◽

Vol 21 (6) ◽

pp. 1981

Author(s):

Fuhan Zhang ◽

Xiaodong Wang ◽

Jiping Guan ◽

Meihan Wu ◽

Lina Guo

Keyword(s):

Deep Learning ◽

Prediction Models ◽

Doppler Radar ◽

Weather Prediction ◽

Rainfall Data ◽

Weather Research And Forecasting ◽

Large Area ◽

Spatiotemporal Resolution ◽

Weather Stations ◽

Numerical Weather Prediction Models

Precipitation has an important impact on people’s daily life and disaster prevention and mitigation. However, it is difficult to provide more accurate results for rainfall nowcasting due to spin-up problems in numerical weather prediction models. Furthermore, existing rainfall nowcasting methods based on machine learning and deep learning cannot provide large-area rainfall nowcasting with high spatiotemporal resolution. This paper proposes a dual-input dual-encoder recurrent neural network, namely Rainfall Nowcasting Network (RN-Net), to solve this problem. It takes the past grid rainfall data interpolated by automatic weather stations and doppler radar mosaic data as input data, and then forecasts the grid rainfall data for the next 2 h. We conduct experiments on the Southeastern China dataset. With a threshold of 0.25 mm, the RN-Net’s rainfall nowcasting threat scores have reached 0.523, 0.503, and 0.435 within 0.5 h, 1 h, and 2 h. Compared with the Weather Research and Forecasting model rainfall nowcasting, the threat scores have been increased by nearly four times, three times, and three times, respectively.

Download Full-text

Forecasting of Meteorologically Driven “Extremes” in Wind and Solar Power: Can We Tackle and Improve Selected Cases of Non-forecasted Extreme Events using Deep Learning?

10.5194/ems2021-164 ◽

2021 ◽

Author(s):

Petrina Papazek ◽

Irene Schicker ◽

Rosmarie de Wit

Keyword(s):

Renewable Energy ◽

Deep Learning ◽

Extreme Events ◽

Energy Production ◽

Solar Power ◽

Prediction Models ◽

Weather Prediction ◽

Power Production ◽

Saharan Dust ◽

Learning Framework

With the rapid transition towards an increased usage of renewable energy accurate predictions of the expected power production are needed to ensure grid stability, energy trading, and (re)scheduling of maintenance and energy transfer. In the last decades, both numerical weather prediction models as well as post-processing methods have significantly improved their prediction skills when applied to renewable energy. There are, however, events in renewable energy production which can be considered as extreme events but are not necessarily extremes in terms of meteorological conditions. The MEDEA project, funded by the Austrian Climate Research Program, aims at improving the definition and detection of extreme events relevant for renewable energies and to use these findings to improve both weather and climate predictions of such extreme events. In this MEDEA case study, we investigate selected (extremes) cases in renewable energy which were not properly reproduced by the models. We will have a deeper dive into two Saharan dust events in 2021 where none of the models was able to properly reproduce the amount of aerosols in Central Europe and solar power production was off by more than 5 GW in contrast to the predictions. &#160;Here, several NWP models have failed to properly recognize its impact and, thus, impaired results based on raw model output. To tackle such events and improve the predictability, a deep learning framework consisting of an auto-encoder LSTM (long short-term memory; type of an artificial neural network) and random forest will be used and adopted for day-ahead predictions of these events. Relevant features for the learning algorithms are extracted from different NWP models, satellite data, and observations. Similarly, for wind energy production we demonstrate the methods in two selected case studies of extreme events. Results obtained by the deep learning framework yield, in general, high forecast-skills where we elaborate on interesting cases studies from a meteorological point of view. Different combinations of inputs and processing-steps are part of the analysis. We compare results to traditional forecast methods in order to validate the performance of our methods.

Download Full-text

Seamless postprocessing of multi-model NWP surface wind forecasts with deep learning

10.5194/ems2021-277 ◽

2021 ◽

Author(s):

Daniele Nerini ◽

Jonas Bhend ◽

Christoph Spirig ◽

Lionel Moret ◽

Mark Liniger

Keyword(s):

Deep Learning ◽

Prediction Models ◽

Weather Prediction ◽

Surface Wind ◽

Ensemble Forecasting ◽

Added Value ◽

Random Errors ◽

Ensemble Forecasts ◽

Late Model ◽

Set Up

Hourly wind forecasts from numerical weather prediction models suffer from a range of systematic and random errors that are to a great extent related to limitations in the model grid resolution. To correct for such biases, statistical postprocessing and downscaling procedures are commonly applied so to leverage the information provided by automatic wind measurements at the surface. More recently, such techniques have been reformulated in a machine learning framework so to profit from the increased availability of data and computational resources. The results reported in the literature are promising and call for a serious evaluation of their potential for operational forecasting.However, there remain several scientific and more applied challenges that need to be addressed before such methods can transition to real-world applications. One such challenge relates to the availability of multiple ensemble forecasts for the same point in time and space, which raises the question of how the information can be efficiently and optimally handled during postprocessing, so to provide added value to the end-user without adding technical debt to the operational system.We propose an approach where a single deep learning model is trained to postprocess a combination of three ensemble forecasting systems, namely the high-resolution regional COSMO model with two configurations, and the ECMWF IFS ENS global ensemble forecasting system. We will show how the training is set up to provide a robust postprocessing model that can account for real time scenarios that include missing data and late model runs, while the quality of the forecasts remains comparable to a single-model approach. We found that the flexibility of the deep learning architecture translates into a robust automatic postprocessing solution that limits the maintenance burden and improves the system&#8217;s reliability.

Download Full-text

Operational seasonal forecasting of crop performance

Philosophical Transactions of the Royal Society B Biological Sciences ◽

10.1098/rstb.2005.1753 ◽

2005 ◽

Vol 360 (1463) ◽

pp. 2109-2124 ◽

Cited By ~ 49

Author(s):

Roger C Stone ◽

Holger Meinke

Keyword(s):

Decision Making ◽

Agricultural Production ◽

Crop Production ◽

Prediction Models ◽

Weather Prediction ◽

Seasonal Forecast ◽

Seasonal Forecasting ◽

Operational Management ◽

Crop Performance

Integrated, interdisciplinary crop performance forecasting systems, linked with appropriate decision and discussion support tools, could substantially improve operational decision making in agricultural management. Recent developments in connecting numerical weather prediction models and general circulation models with quantitative crop growth models offer the potential for development of integrated systems that incorporate components of long-term climate change. However, operational seasonal forecasting systems have little or no value unless they are able to change key management decisions. Changed decision making through incorporation of seasonal forecasting ultimately has to demonstrate improved long-term performance of the cropping enterprise. Simulation analyses conducted on specific production scenarios are especially useful in improving decisions, particularly if this is done in conjunction with development of decision-support systems and associated facilitated discussion groups. Improved management of the overall crop production system requires an interdisciplinary approach, where climate scientists, agricultural scientists and extension specialists are intimately linked with crop production managers in the development of targeted seasonal forecast systems. The same principle applies in developing improved operational management systems for commodity trading organizations, milling companies and agricultural marketing organizations. Application of seasonal forecast systems across the whole value chain in agricultural production offers considerable benefits in improving overall operational management of agricultural production.

Download Full-text

Radar Echo Spatiotemporal Sequence Prediction Using an Improved ConvGRU Deep Learning Model

Atmosphere ◽

10.3390/atmos13010088 ◽

2022 ◽

Vol 13 (1) ◽

pp. 88

Author(s):

Wei He ◽

Taisong Xiong ◽

Hao Wang ◽

Jianxin He ◽

Xinyue Ren ◽

...

Keyword(s):

Deep Learning ◽

Input Data ◽

Prediction Models ◽

Mean Squared Error ◽

Mean Absolute Error ◽

Weather Prediction ◽

Absolute Error ◽

Sequence Prediction ◽

Squared Error ◽

Radar Echo

Precipitation nowcasting is extremely important in disaster prevention and mitigation, and can improve the quality of meteorological forecasts. In recent years, deep learning-based spatiotemporal sequence prediction models have been widely used in precipitation nowcasting, obtaining better prediction results than numerical weather prediction models and traditional radar echo extrapolation results. Because existing deep learning models rarely consider the inherent interactions between the model input data and the previous output, model prediction results do not sufficiently meet the actual forecast requirement. We propose a Modified Convolutional Gated Recurrent Unit (M-ConvGRU) model that performs convolution operations on the input data and previous output of a GRU network. Moreover, this adopts an encoder–forecaster structure to better capture the characteristics of spatiotemporal correlation in radar echo maps. The results of multiple experiments demonstrate the effectiveness of the proposed model. The balanced mean absolute error (B-MAE) and balanced mean squared error (B-MSE) of M-ConvGRU are slightly lower than Convolutional Long Short-Term Memory (ConvLSTM), but the mean absolute error (MAE) and mean squared error (MSE) of M-ConvGRU are 6.29% and 10.25% lower than ConvLSTM, and the prediction accuracy and prediction performance for strong echo regions were also improved.

Download Full-text

The Latest Update of JMA Numerical Weather Prediction Models and its Solar Power Forecasting Errors

IEEJ Transactions on Power and Energy ◽

10.1541/ieejpes.138.881 ◽

2018 ◽

Vol 138 (11) ◽

pp. 881-892 ◽

Cited By ~ 1

Author(s):

Hideaki Ohtake ◽

Fumichika Uno ◽

Takashi Oozeki ◽

Yoshinori Yamada

Keyword(s):

Numerical Weather Prediction ◽

Solar Power ◽

Prediction Models ◽

Weather Prediction ◽

Numerical Weather ◽

Solar Power Forecasting ◽

Forecasting Errors ◽

Numerical Weather Prediction Models ◽

Power Forecasting

Download Full-text

Applying Deep Learning to Audit Procedures: An Illustrative Framework

Accounting Horizons ◽

10.2308/acch-52455 ◽

2019 ◽

Vol 33 (3) ◽

pp. 89-109 ◽

Cited By ~ 9

Author(s):

Ting (Sophia) Sun

Keyword(s):

Decision Making ◽

Deep Learning ◽

Data Warehouse ◽

Visual Recognition ◽

Prediction Models ◽

Historical Data ◽

Structured Data ◽

Text Understanding ◽

Audit Data ◽

Learning Functions

SYNOPSIS This paper aims to promote the application of deep learning to audit procedures by illustrating how the capabilities of deep learning for text understanding, speech recognition, visual recognition, and structured data analysis fit into the audit environment. Based on these four capabilities, deep learning serves two major functions in supporting audit decision making: information identification and judgment support. The paper proposes a framework for applying these two deep learning functions to a variety of audit procedures in different audit phases. An audit data warehouse of historical data can be used to construct prediction models, providing suggested actions for various audit procedures. The data warehouse will be updated and enriched with new data instances through the application of deep learning and a human auditor's corrections. Finally, the paper discusses the challenges faced by the accounting profession, regulators, and educators when it comes to applying deep learning.

Download Full-text