An Incremental Learning approach using Long Short-Term Memory Neural Networks

2020 ◽  
Author(s):  
Álvaro C. Lemos Neto ◽  
Rodrigo A. Coelho ◽  
Cristiano L. de Castro
Keyword(s):  
Data Streams ◽  
Incremental Learning ◽  
Memory Management ◽  
Short Term Memory ◽  
Concept Drift ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Noise Data ◽  
Evolving Data ◽  
Robust To Noise

Due to Big Data and the Internet of Things, Machine Learning algorithms targeted specifically to model evolving data streams had gained attention from both academia and industry. Many Incremental Learning models had been successful in doing so, but most of them have one thing in common: they are complex variants of batch learning algorithms, which is a problem since, in a streaming setting, less complexity and more performance is desired. This paper proposes the Incremental LSTM model, which is a variant of the original LSTM with minor changes, that can tackle evolving data streams problems such as concept drift and the elasticity-plasticity dilemma without neither needing a dedicated drift detector nor a memory management system. It obtained great results that show it reacts fast to concept drifts and that is also robust to noise data.

scholarly journals Adaptive online incremental learning for evolving data streams

2021 ◽  
Vol 105 ◽  
pp. 107255
Author(s):  
Si-si Zhang ◽  
Jian-wei Liu ◽  
Xin Zuo
Keyword(s):  
Data Streams ◽  
Incremental Learning ◽  
Evolving Data ◽  
Online Incremental Learning

scholarly journals Development and Evaluation of the Combined Machine Learning Models for the Prediction of Dam Inflow

Water ◽  
2020 ◽  
Vol 12 (10) ◽  
pp. 2927
Author(s):  
Jiyeong Hong ◽  
Seoro Lee ◽  
Joo Hyun Bae ◽  
Jimin Lee ◽  
Woon Ji Park ◽  
...  
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Random Forest ◽  
Multilayer Perceptron ◽  
Short Term Memory ◽  
Learning Algorithms ◽  
Absolute Error ◽  
Machine Learning Algorithms ◽  
Gradient Boosting ◽  
Dam Inflow

Predicting dam inflow is necessary for effective water management. This study created machine learning algorithms to predict the amount of inflow into the Soyang River Dam in South Korea, using weather and dam inflow data for 40 years. A total of six algorithms were used, as follows: decision tree (DT), multilayer perceptron (MLP), random forest (RF), gradient boosting (GB), recurrent neural network–long short-term memory (RNN–LSTM), and convolutional neural network–LSTM (CNN–LSTM). Among these models, the multilayer perceptron model showed the best results in predicting dam inflow, with the Nash–Sutcliffe efficiency (NSE) value of 0.812, root mean squared errors (RMSE) of 77.218 m3/s, mean absolute error (MAE) of 29.034 m3/s, correlation coefficient (R) of 0.924, and determination coefficient (R2) of 0.817. However, when the amount of dam inflow is below 100 m3/s, the ensemble models (random forest and gradient boosting models) performed better than MLP for the prediction of dam inflow. Therefore, two combined machine learning (CombML) models (RF_MLP and GB_MLP) were developed for the prediction of the dam inflow using the ensemble methods (RF and GB) at precipitation below 16 mm, and the MLP at precipitation above 16 mm. The precipitation of 16 mm is the average daily precipitation at the inflow of 100 m3/s or more. The results show the accuracy verification results of NSE 0.857, RMSE 68.417 m3/s, MAE 18.063 m3/s, R 0.927, and R2 0.859 in RF_MLP, and NSE 0.829, RMSE 73.918 m3/s, MAE 18.093 m3/s, R 0.912, and R2 0.831 in GB_MLP, which infers that the combination of the models predicts the dam inflow the most accurately. CombML algorithms showed that it is possible to predict inflow through inflow learning, considering flow characteristics such as flow regimes, by combining several machine learning algorithms.

scholarly journals An ensemble based on neural networks with random weights for online data stream regression

2019 ◽  
Vol 24 (13) ◽  
pp. 9835-9855 ◽  
Author(s):  
Ricardo de Almeida ◽  
Yee Mey Goh ◽  
Radmehr Monfared ◽  
Maria Teresinha Arns Steiner ◽  
Andrew West
Keyword(s):  
Data Stream ◽  
Prediction Accuracy ◽  
Concept Drift ◽  
Learning Algorithms ◽  
Data Distribution ◽  
Machine Learning Algorithms ◽  
Computational Time ◽  
Online Data ◽  
Data Prediction ◽  
Random Weights

Abstract Most information sources in the current technological world are generating data sequentially and rapidly, in the form of data streams. The evolving nature of processes may often cause changes in data distribution, also known as concept drift, which is difficult to detect and causes loss of accuracy in supervised learning algorithms. As a consequence, online machine learning algorithms that are able to update actively according to possible changes in the data distribution are required. Although many strategies have been developed to tackle this problem, most of them are designed for classification problems. Therefore, in the domain of regression problems, there is a need for the development of accurate algorithms with dynamic updating mechanisms that can operate in a computational time compatible with today’s demanding market. In this article, the authors propose a new bagging ensemble approach based on neural network with random weights for online data stream regression. The proposed method improves the data prediction accuracy as well as minimises the required computational time compared to a recent algorithm for online data stream regression from literature. The experiments are carried out using four synthetic datasets to evaluate the algorithm’s response to concept drift, along with four benchmark datasets from different industries. The results indicate improvement in data prediction accuracy, effectiveness in handling concept drift, and much faster updating times compared to the existing available approach. Additionally, the use of design of experiments as an effective tool for hyperparameter tuning is demonstrated.

Online Machine Learning Algorithms over Data Streams

2018 ◽  
pp. 1-9
Author(s):  
András A. Benczúr ◽  
Levente Kocsis ◽  
Róbert Pálovics
Keyword(s):  
Machine Learning ◽  
Data Streams ◽  
Learning Algorithms ◽  
Machine Learning Algorithms

scholarly journals SEO Ministration : Content Related Tag Suggestion Using One vs Rest

2021 ◽  
pp. 245-253
Author(s):  
Dyapa Sravan Reddy ◽  
Lakshmi Prasanna Reddy ◽  
Kandibanda Sai Santhosh ◽  
Virrat Devaser
Keyword(s):  
Machine Learning ◽  
Short Term Memory ◽  
Learning Algorithms ◽  
Real Life ◽  
Machine Learning Algorithms ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

SEO Analyst pays a lot of time finding relevant tags for their articles and in some cases, they are unaware of the content topics. The current proposed ML model will recommend content-related tags so that the Content writers/SEO analyst will be having an overview regarding the content and minimizes their time spent on unknown articles. Machine Learning algorithms have a plethora of applications and the extent of their real-life implementations cannot be estimated. Using algorithms like One vs Rest (OVR), Long Short-Term Memory (LSTM), this study has analyzed how Machine Learning can be useful for tag suggestions for a topic. The training of the model with One vs Rest turned out to deliver more accurate results than others. This Study certainly answers how One vs Rest is used for tag suggestions that are needed to promote a website and further studies are required to suggest keywords required.

Synchronous and Open, Real World, Vehicle, ADAS, and Infrastructure Data Streams for Automotive Machine Learning Algorithms Research

2020 ◽  
Author(s):  
Aaron I. Rabinowitz ◽  
Tushar Gaikwad ◽  
Samantha White ◽  
Thomas Bradley ◽  
Zachary Asher
Keyword(s):  
Machine Learning ◽  
Real World ◽  
Data Streams ◽  
Learning Algorithms ◽  
Machine Learning Algorithms

Incremental Learning on Non-stationary Data Stream using Ensemble Approach

2016 ◽  
Vol 6 (4) ◽  
pp. 1811 ◽  
Author(s):  
Meenakshi Anurag Thalor ◽  
Shrishailapa Patil
Keyword(s):  
Machine Learning ◽  
Incremental Learning ◽  
Data Stream ◽  
Recommendation System ◽  
Concept Drift ◽  
Joint Probability ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Joint Probability Distribution ◽  
Changes Over Time

<span lang="EN-US">Incremental Learning on non stationary distribution has been shown to be a very challenging problem in machine learning and data mining, because the joint probability distribution between the data and classes changes over time. Many real time problems suffer concept drift as they changes with time. For example, an advertisement recommendation system, in which customer’s behavior may change depending on the season of the year, on the inflation and on new products made available. An extra challenge arises when the classes to be learned are not represented equally in the training data i.e. classes are imbalanced, as most machine learning algorithms work well only when the training data  is balanced. The objective of this paper is to develop an ensemble based classification algorithm for non-stationary data stream (ENSDS) with focus on two-class problems. In addition, we are presenting here an exhaustive comparison of purposed algorithms with state-of-the-art classification approaches using different evaluation measures like recall, f-measure and g-mean</span>

scholarly journals Employing traditional machine learning algorithms for big data streams analysis: The case of object trajectory prediction

2017 ◽  
Vol 127 ◽  
pp. 249-257 ◽  
Author(s):  
Angelos Valsamis ◽  
Konstantinos Tserpes ◽  
Dimitrios Zissis ◽  
Dimosthenis Anagnostopoulos ◽  
Theodora Varvarigou
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Data Streams ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Trajectory Prediction ◽  
Big Data Streams

Discussion and review on evolving data streams and concept drift adapting

2016 ◽  
Vol 9 (1) ◽  
pp. 1-23 ◽  
Author(s):  
Imen Khamassi ◽  
Moamar Sayed-Mouchaweh ◽  
Moez Hammami ◽  
Khaled Ghédira
Keyword(s):  
Data Streams ◽  
Concept Drift ◽  
Evolving Data
Sign in / Sign up

Export Citation Format

Share Document