scholarly journals Improving machine learning-based weather forecast 1 post-processing with clustering and transfer learning

2020 ◽  
Author(s):  
Xiaomeng Huang ◽  
Yuwen Chen ◽  
Yi Li ◽  
Yue Chen ◽  
Chi Yan Tsui ◽  
...  
Keyword(s):  
Machine Learning ◽  
Transfer Learning ◽  
Weather Forecast ◽  
Post Processing

Numerical Weather Forecast Post-processing with Ensemble Learning and Transfer Learning

2020 ◽  
Author(s):  
Yuwen Chen ◽  
Xiaomeng Huang
Keyword(s):  
Machine Learning ◽  
Transfer Learning ◽  
Ensemble Learning ◽  
Weather Forecasting ◽  
Weather Forecast ◽  
Processing Temperature ◽  
Forecast Errors ◽  
Post Processing ◽  
Numerical Weather ◽  
One Year

<p>Statistical approaches have been used for decades to augment and interpret numerical weather forecasts. The emergence of artificial intelligence algorithms has provided new perspectives in this field, but the extension of algorithms developed for station networks with rich historical records to include newly-built stations remains a challenge. To address this, we design a framework that combines two machine learning methods: temperature prediction based on ensemble of multiple machine learning models and transfer learning for newly-built stations. We then evaluate this framework by post-processing temperature forecasts provided by a leading weather forecast center and observations from 301 weather stations in China. Station clustering reduces forecast errors by 24.4% averagely, while transfer learning improves predictions by 13.4% for recently-built sites with only one year of data available. This work demonstrates how ensemble learning and transfer learning can be used to supplement weather forecasting.</p><p></p>

Improving machine learning-based weather forecast post-processing with clustering and transfer learning

2020 ◽  
Author(s):  
Xiaomeng Huang ◽  
Yuwen Chen ◽  
Yi Li ◽  
Yue Chen ◽  
Chi Yan Tsui ◽  
...  
Keyword(s):  
Machine Learning ◽  
Transfer Learning ◽  
Weather Forecast ◽  
Post Processing

scholarly journals Artificial Learning Dispatch Planning with Probabilistic Forecasts: Using Uncertainties as an Asset

Energies ◽  
2020 ◽  
Vol 13 (3) ◽  
pp. 616
Author(s):  
Ana Carolina do Amaral Burghi ◽  
Tobias Hirsch ◽  
Robert Pitz-Paal
Keyword(s):  
Machine Learning ◽  
Learning Algorithm ◽  
Weather Forecast ◽  
Machine Learning Algorithm ◽  
Post Processing ◽  
Weather Forecasts ◽  
Additional Information ◽  
Model Following ◽  
Probabilistic Forecasts ◽  
Artificial Learning

Weather forecast uncertainty is a key element for energy market volatility. By intelligently considering uncertainties on the schedule development, renewable energy systems with storage could improve dispatching accuracy, and therefore, effectively participate in electricity wholesale markets. Deterministic forecasts have been traditionally used to support dispatch planning, representing reduced or no uncertainty information about the future weather. Aiming at better representing the uncertainties involved, probabilistic forecasts have been developed to increase forecasting accuracy. For the dispatch planning, this can highly influence the development of a more precise schedule. This work extends a dispatch planning method to the use of probabilistic weather forecasts. The underlying method used a schedule optimizer coupled to a post-processing machine learning algorithm. This machine learning algorithm was adapted to include probabilistic forecasts, considering their additional information on uncertainties. This post-processing applied a calibration of the planned schedule considering the knowledge about uncertainties obtained from similar past situations. Simulations performed with a concentrated solar power plant model following the proposed strategy demonstrated promising financial improvement and relevant potential in dealing with uncertainties. Results especially show that information included in probabilistic forecasts can increase financial revenues up to 15% (in comparison to a persistence solar driven approach) if processed in a suitable way.

scholarly journals A framework for probabilistic weather forecast post-processing across models and lead times using machine learning

2021 ◽  
Vol 379 (2194) ◽  
pp. 20200099 ◽  
Author(s):  
Charlie Kirkwood ◽  
Theo Economou ◽  
Henry Odbert ◽  
Nicolas Pugeault
Keyword(s):  
Machine Learning ◽  
Decision Support ◽  
Probability Distributions ◽  
Weather Prediction ◽  
Weather Forecast ◽  
Predictive Distribution ◽  
Lead Times ◽  
Climate Modelling ◽  
Post Processing ◽  
Probabilistic Forecasts

Forecasting the weather is an increasingly data-intensive exercise. Numerical weather prediction (NWP) models are becoming more complex, with higher resolutions, and there are increasing numbers of different models in operation. While the forecasting skill of NWP models continues to improve, the number and complexity of these models poses a new challenge for the operational meteorologist: how should the information from all available models, each with their own unique biases and limitations, be combined in order to provide stakeholders with well-calibrated probabilistic forecasts to use in decision making? In this paper, we use a road surface temperature example to demonstrate a three-stage framework that uses machine learning to bridge the gap between sets of separate forecasts from NWP models and the ‘ideal’ forecast for decision support: probabilities of future weather outcomes. First, we use quantile regression forests to learn the error profile of each numerical model, and use these to apply empirically derived probability distributions to forecasts. Second, we combine these probabilistic forecasts using quantile averaging. Third, we interpolate between the aggregate quantiles in order to generate a full predictive distribution, which we demonstrate has properties suitable for decision support. Our results suggest that this approach provides an effective and operationally viable framework for the cohesive post-processing of weather forecasts across multiple models and lead times to produce a well-calibrated probabilistic output. This article is part of the theme issue ‘Machine learning for weather and climate modelling’.

Deep Learning for text in limted data settings

2020 ◽  
Author(s):  
Pathikkumar Patel ◽  
Bhargav Lad ◽  
Jinan Fiaidhi
Keyword(s):  
Machine Learning ◽  
Time Series ◽  
Deep Learning ◽  
Sentiment Analysis ◽  
Transfer Learning ◽  
Text Classification ◽  
State Of The Art ◽  
Time Series Forecasting ◽  
Text Data ◽  
Performance Levels

During the last few years, RNN models have been extensively used and they have proven to be better for sequence and text data. RNNs have achieved state-of-the-art performance levels in several applications such as text classification, sequence to sequence modelling and time series forecasting. In this article we will review different Machine Learning and Deep Learning based approaches for text data and look at the results obtained from these methods. This work also explores the use of transfer learning in NLP and how it affects the performance of models on a specific application of sentiment analysis.

scholarly journals Deep Transfer Learning Based Intrusion Detection System for Electric Vehicular Networks

Sensors ◽  
2021 ◽  
Vol 21 (14) ◽  
pp. 4736
Author(s):  
Sk. Tanzir Mehedi ◽  
Adnan Anwar ◽  
Ziaur Rahman ◽  
Kawsar Ahmed
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Intrusion Detection ◽  
Real Time ◽  
Transfer Learning ◽  
Security Requirements ◽  
Detection Accuracy ◽  
Area Network ◽  
Complex Data ◽  
Network Intrusion

The Controller Area Network (CAN) bus works as an important protocol in the real-time In-Vehicle Network (IVN) systems for its simple, suitable, and robust architecture. The risk of IVN devices has still been insecure and vulnerable due to the complex data-intensive architectures which greatly increase the accessibility to unauthorized networks and the possibility of various types of cyberattacks. Therefore, the detection of cyberattacks in IVN devices has become a growing interest. With the rapid development of IVNs and evolving threat types, the traditional machine learning-based IDS has to update to cope with the security requirements of the current environment. Nowadays, the progression of deep learning, deep transfer learning, and its impactful outcome in several areas has guided as an effective solution for network intrusion detection. This manuscript proposes a deep transfer learning-based IDS model for IVN along with improved performance in comparison to several other existing models. The unique contributions include effective attribute selection which is best suited to identify malicious CAN messages and accurately detect the normal and abnormal activities, designing a deep transfer learning-based LeNet model, and evaluating considering real-world data. To this end, an extensive experimental performance evaluation has been conducted. The architecture along with empirical analyses shows that the proposed IDS greatly improves the detection accuracy over the mainstream machine learning, deep learning, and benchmark deep transfer learning models and has demonstrated better performance for real-time IVN security.

Transfer learning based convolution neural net for authentication and classification of emotions from natural and stimulated speech signals

2021 ◽  
pp. 1-12
Author(s):  
Mukul Kumar ◽  
Nipun Katyal ◽  
Nersisson Ruban ◽  
Elena Lyakso ◽  
A. Mary Mekala ◽  
...  
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Transfer Learning ◽  
Research Work ◽  
Oral Communication ◽  
Neural Net ◽  
Emotion Classification ◽  
Style Transfer ◽  
Conventional Machine ◽  
Neural Network Algorithm

Over the years the need for differentiating various emotions from oral communication plays an important role in emotion based studies. There have been different algorithms to classify the kinds of emotion. Although there is no measure of fidelity of the emotion under consideration, which is primarily due to the reason that most of the readily available datasets that are annotated are produced by actors and not generated in real-world scenarios. Therefore, the predicted emotion lacks an important aspect called authenticity, which is whether an emotion is actual or stimulated. In this research work, we have developed a transfer learning and style transfer based hybrid convolutional neural network algorithm to classify the emotion as well as the fidelity of the emotion. The model is trained on features extracted from a dataset that contains stimulated as well as actual utterances. We have compared the developed algorithm with conventional machine learning and deep learning techniques by few metrics like accuracy, Precision, Recall and F1 score. The developed model performs much better than the conventional machine learning and deep learning models. The research aims to dive deeper into human emotion and make a model that understands it like humans do with precision, recall, F1 score values of 0.994, 0.996, 0.995 for speech authenticity and 0.992, 0.989, 0.99 for speech emotion classification respectively.

Machine Learning as a Silent Observer of Advanced Geoscience Interpretation

2021 ◽  
Author(s):  
Muhammad Sajid
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Transfer Learning ◽  
Learning Performance ◽  
Learning Approach ◽  
Seismic Fault ◽  
Facies Classification ◽  
The World ◽  
Learning Machine ◽  
Fault Interpretation

Abstract Machine learning is proving its successes in all fields of life including medical, automotive, planning, engineering, etc. In the world of geoscience, ML showed impressive results in seismic fault interpretation, advance seismic attributes analysis, facies classification, and geobodies extraction such as channels, carbonates, and salt, etc. One of the challenges faced in geoscience is the availability of label data which is one of the most time-consuming requirements in supervised deep learning. In this paper, an advanced learning approach is proposed for geoscience where the machine observes the seismic interpretation activities and learns simultaneously as the interpretation progresses. Initial testing showed that through the proposed method along with transfer learning, machine learning performance is highly effective, and the machine accurately predicts features requiring minor post prediction filtering to be accepted as the optimal interpretation.

Sign in / Sign up

Export Citation Format

Share Document