Complex hybrid models combining deterministic and machine learning components for numerical climate modeling and weather prediction

2006 ◽  
Vol 19 (2) ◽  
pp. 122-134 ◽  
Author(s):  
Vladimir M. Krasnopolsky ◽  
Michael S. Fox-Rabinovitz
Keyword(s):  
Machine Learning ◽  
Climate Modeling ◽  
Weather Prediction ◽  
Hybrid Models ◽  
Complex Hybrid

scholarly journals Hybrid Machine Learning for Solar Radiation Prediction in Reduced Feature Spaces

Energies ◽  
2021 ◽  
Vol 14 (23) ◽  
pp. 7970
Author(s):  
Abdel-Rahman Hedar ◽  
Majid Almaraashi ◽  
Alaa E. Abdel-Hakim ◽  
Mahmoud Abdulrahim
Keyword(s):  
Machine Learning ◽  
Solar Radiation ◽  
Root Mean Square ◽  
Prediction Models ◽  
Weather Prediction ◽  
Hybrid Models ◽  
Mean Square ◽  
Learning Models ◽  
Mean Square Errors ◽  
Machine Learning Models

Solar radiation prediction is an important process in ensuring optimal exploitation of solar energy power. Numerous models have been applied to this problem, such as numerical weather prediction models and artificial intelligence models. However, well-designed hybridization approaches that combine numerical models with artificial intelligence models to yield a more powerful model can provide a significant improvement in prediction accuracy. In this paper, novel hybrid machine learning approaches that exploit auxiliary numerical data are proposed. The proposed hybrid methods invoke different machine learning paradigms, including feature selection, classification, and regression. Additionally, numerical weather prediction (NWP) models are used in the proposed hybrid models. Feature selection is used for feature space dimension reduction to reduce the large number of recorded parameters that affect estimation and prediction processes. The rough set theory is applied for attribute reduction and the dependency degree is used as a fitness function. The effect of the attribute reduction process is investigated using thirty different classification and prediction models in addition to the proposed hybrid model. Then, different machine learning models are constructed based on classification and regression techniques to predict solar radiation. Moreover, other hybrid prediction models are formulated to use the output of the numerical model of Weather Research and Forecasting (WRF) as learning elements in order to improve the prediction accuracy. The proposed methodologies are evaluated using a data set that is collected from different regions in Saudi Arabia. The feature-reduction has achieved higher classification rates up to 8.5% for the best classifiers and up to 15% for other classifiers, for the different data collection regions. Additionally, in the regression, it achieved improvements of average root mean square error up to 5.6% and in mean absolute error values up to 8.3%. The hybrid models could reduce the root mean square errors by 70.2% and 4.3% than the numerical and machine learning models, respectively, when these models are applied to some dataset. For some reduced feature data, the hybrid models could reduce the root mean square errors by 47.3% and 14.4% than the numerical and machine learning models, respectively.

scholarly journals Numerical Weather Prediction and Artificial Neural Network Coupling for Wind Energy Forecast

Energies ◽  
2021 ◽  
Vol 14 (2) ◽  
pp. 338
Author(s):  
Lorenzo Donadio ◽  
Jiannong Fang ◽  
Fernando Porté-Agel
Keyword(s):  
Machine Learning ◽  
Wind Energy ◽  
Wind Power ◽  
Numerical Weather Prediction ◽  
Wind Farm ◽  
Weather Prediction ◽  
Wind Power Forecasting ◽  
Numerical Weather ◽  
Artificial Neural ◽  
Power Forecasting

In the past two decades, wind energy has been under fast development worldwide. The dramatic increase of wind power penetration in electricity production has posed a big challenge to grid integration due to the high uncertainty of wind power. Accurate real-time forecasts of wind farm power outputs can help to mitigate the problem. Among the various techniques developed for wind power forecasting, the hybridization of numerical weather prediction (NWP) and machine learning (ML) techniques such as artificial neural networks (ANNs) are attracting many researchers world-wide nowadays, because it has the potential to yield more accurate forecasts. In this paper, two hybrid NWP and ANN models for wind power forecasting over a highly complex terrain are proposed. The developed models have a fine temporal resolution and a sufficiently large prediction horizon (>6 h ahead). Model 1 directly forecasts the energy production of each wind turbine. Model 2 forecasts first the wind speed, then converts it to the power using a fitted power curve. Effects of various modeling options (selection of inputs, network structures, etc.) on the model performance are investigated. Performances of different models are evaluated based on four normalized error measures. Statistical results of model predictions are presented with discussions. Python was utilized for task automation and machine learning. The end result is a fully working library for wind power predictions and a set of tools for running the models in forecast mode. It is shown that the proposed models are able to yield accurate wind farm power forecasts at a site with high terrain and flow complexities. Especially, for Model 2, the normalized Mean Absolute Error and Root Mean Squared Error are obtained as 8.76% and 13.03%, respectively, lower than the errors reported by other models in the same category.

scholarly journals ARPEGE cloud cover forecast post-processing with convolutional neural network

2020 ◽  
Author(s):  
Florian Dupuy ◽  
Olivier Mestre ◽  
Léo Pfitzner
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Convolutional Neural Network ◽  
Random Forests ◽  
Cloud Cover ◽  
Spatial Information ◽  
Weather Prediction ◽  
Excellent Result ◽  
Machine Learning Algorithms ◽  
Post Processing

<p>Cloud cover is a crucial information for many applications such as planning land observation missions from space. However, cloud cover remains a challenging variable to forecast, and Numerical Weather Prediction (NWP) models suffer from significant biases, hence justifying the use of statistical post-processing techniques. In our application, the ground truth is a gridded cloud cover product derived from satellite observations over Europe, and predictors are spatial fields of various variables produced by ARPEGE (Météo-France global NWP) at the corresponding lead time.</p><p>In this study, ARPEGE cloud cover is post-processed using a convolutional neural network (CNN). CNN is the most popular machine learning tool to deal with images. In our case, CNN allows to integrate spatial information contained in NWP outputs. We show that a simple U-Net architecture produces significant improvements over Europe. Compared to the raw ARPEGE forecasts, MAE drops from 25.1 % to 17.8 % and RMSE decreases from 37.0 % to 31.6 %. Considering specific needs for earth observation, special interest was put on forecasts with low cloud cover conditions (< 10 %). For this particular nebulosity class, we show that hit rate jumps from 40.6 to 70.7 (which is the order of magnitude of what can be achieved using classical machine learning algorithms such as random forests) while false alarm decreases from 38.2 to 29.9. This is an excellent result, since improving hit rates by means of random forests usually also results in a slight increase of false alarms.</p>

scholarly journals Using machine learning to correct model error in data assimilation and forecast applications

2021 ◽  
Author(s):  
Alban Farchi ◽  
Patrick Laloyaux ◽  
Massimo Bonavita ◽  
Marc Bocquet
Keyword(s):  
Machine Learning ◽  
Data Assimilation ◽  
Data Science ◽  
Weather Prediction ◽  
Realistic Model ◽  
Model Error ◽  
Underlying Assumption ◽  
Correct Model ◽  
Recent Developments ◽  
Spatiotemporal Processes

<p>Recent developments in machine learning (ML) have demonstrated impressive skills in reproducing complex spatiotemporal processes. However, contrary to data assimilation (DA), the underlying assumption behind ML methods is that the system is fully observed and without noise, which is rarely the case in numerical weather prediction. In order to circumvent this issue, it is possible to embed the ML problem into a DA formalism characterised by a cost function similar to that of the weak-constraint 4D-Var (Bocquet et al., 2019; Bocquet et al., 2020). In practice ML and DA are combined to solve the problem: DA is used to estimate the state of the system while ML is used to estimate the full model. </p><p>In realistic systems, the model dynamics can be very complex and it may not be possible to reconstruct it from scratch. An alternative could be to learn the model error of an already existent model using the same approach combining DA and ML. In this presentation, we test the feasibility of this method using a quasi geostrophic (QG) model. After a brief description of the QG model model, we introduce a realistic model error to be learnt. We then asses the potential of ML methods to reconstruct this model error, first with perfect (full and noiseless) observation and then with sparse and noisy observations. We show in either case to what extent the trained ML models correct the mid-term forecasts. Finally, we show how the trained ML models can be used in a DA system and to what extent they correct the analysis.</p><p>Bocquet, M., Brajard, J., Carrassi, A., and Bertino, L.: Data assimilation as a learning tool to infer ordinary differential equation representations of dynamical models, Nonlin. Processes Geophys., 26, 143–162, 2019</p><p>Bocquet, M., Brajard, J., Carrassi, A., and Bertino, L.: Bayesian inference of chaotic dynamics by merging data assimilation, machine learning and expectation-maximization, Foundations of Data Science, 2 (1), 55-80, 2020</p><p>Farchi, A., Laloyaux, P., Bonavita, M., and Bocquet, M.: Using machine learning to correct model error in data assimilation and forecast applications, arxiv:2010.12605, submitted. </p>

scholarly journals Machine Learning for Internet of Things

2019 ◽  
Vol 8 (07) ◽  
pp. 24680-24782
Author(s):  
Manisha Bagri ◽  
Neha Aggarwal
Keyword(s):  
Machine Learning ◽  
Internet Of Things ◽  
Learning Algorithm ◽  
Weather Prediction ◽  
Support Vector ◽  
The Internet ◽  
Machine Learning Algorithm ◽  
Learning Patterns ◽  
New Development ◽  
Share Data

By 2020 around 25-50 billion devices are likely to be connected to the internet. Due to this new development, it gives rise to something called Internet of Things (IoT). The interconnected devices can generate and share data over a network. Machine Learning plays a key role in IoT to handle the vast amount of data. It gives IoT and devices a brain to think, which is often called as intelligence. The data can be feed to machines for learning patterns, based on training the machines can identify to predict for the future. This paper gives a brief explanation of IoT. This paper gives a crisp explanation of machine learning algorithm and its types. However, Support Vector Machine (SVM) is explained in details along with its merits and demerits. An algorithm is also proposed for weather prediction using SVM for IoT.

scholarly journals Kabul River Flow Prediction Using Automated ARIMA Forecasting: A Machine Learning Approach

2021 ◽  
Vol 13 (19) ◽  
pp. 10720
Author(s):  
Muhammad Ali Musarat ◽  
Wesam Salah Alaloul ◽  
Muhammad Babar Ali Rabbani ◽  
Mujahid Ali ◽  
Muhammad Altaf ◽  
...  
Keyword(s):  
Machine Learning ◽  
Water Level ◽  
River Flow ◽  
Early Stage ◽  
Moving Average ◽  
Weather Prediction ◽  
Maximum Level ◽  
Information Criterion ◽  
Water Levels ◽  
Kabul River

The water level in a river defines the nature of flow and is fundamental to flood analysis. Extreme fluctuation in water levels in rivers, such as floods and droughts, are catastrophic in every manner; therefore, forecasting at an early stage would prevent possible disasters and relief efforts could be set up on time. This study aims to digitally model the water level in the Kabul River to prevent and alleviate the effects of any change in water level in this river downstream. This study used a machine learning tool known as the automatic autoregressive integrated moving average for statistical methodological analysis for forecasting the river flow. Based on the hydrological data collected from the water level of Kabul River in Swat, the water levels from 2011–2030 were forecasted, which were based on the lowest value of Akaike Information Criterion as 9.216. It was concluded that the water flow started to increase from the year 2011 till it reached its peak value in the year 2019–2020, and then the water level will maintain its maximum level to 250 cumecs and minimum level to 10 cumecs till 2030. The need for this research is justified as it could prove helpful in establishing guidelines for hydrological designers, the planning and management of water, hydropower engineering projects, as an indicator for weather prediction, and for the people who are greatly dependent on the Kabul River for their survival.

Sign in / Sign up

Export Citation Format

Share Document