Automatically Optimized Gradient Boosting Trees for Classifying Large Volume High Cardinality Data Streams Under Concept Drift

2019 ◽  
pp. 317-335
Author(s):  
Jobin Wilson ◽  
Amit Kumar Meher ◽  
Bivin Vinodkumar Bindu ◽  
Santanu Chaudhury ◽  
Brejesh Lall ◽  
...  
Keyword(s):  
Large Volume ◽  
Data Streams ◽  
Concept Drift ◽  
Gradient Boosting

scholarly journals Concept Drift Adaptation Techniques in Distributed Environment for Real-World Data Streams

Smart Cities ◽  
2021 ◽  
Vol 4 (1) ◽  
pp. 349-371
Author(s):  
Hassan Mehmood ◽  
Panos Kostakos ◽  
Marta Cortes ◽  
Theodoros Anagnostopoulos ◽  
Susanna Pirttikangas ◽  
...  
Keyword(s):  
Real World ◽  
Data Streams ◽  
Smart City ◽  
Smart Cities ◽  
Concept Drift ◽  
Distributed Environment ◽  
Real World Data ◽  
Unique Challenge ◽  
World Data ◽  
Concept Drift Detection

Real-world data streams pose a unique challenge to the implementation of machine learning (ML) models and data analysis. A notable problem that has been introduced by the growth of Internet of Things (IoT) deployments across the smart city ecosystem is that the statistical properties of data streams can change over time, resulting in poor prediction performance and ineffective decisions. While concept drift detection methods aim to patch this problem, emerging communication and sensing technologies are generating a massive amount of data, requiring distributed environments to perform computation tasks across smart city administrative domains. In this article, we implement and test a number of state-of-the-art active concept drift detection algorithms for time series analysis within a distributed environment. We use real-world data streams and provide critical analysis of results retrieved. The challenges of implementing concept drift adaptation algorithms, along with their applications in smart cities, are also discussed.

scholarly journals Measuring the Effectiveness of Adaptive Random Forest for Handling Concept Drift in Big Data Streams

Entropy ◽  
2021 ◽  
Vol 23 (7) ◽  
pp. 859
Author(s):  
Abdulaziz O. AlQabbany ◽  
Aqil M. Azmi
Keyword(s):  
Big Data ◽  
Random Forest ◽  
Real Time ◽  
Data Streams ◽  
Learning Algorithm ◽  
Concept Drift ◽  
The United States ◽  
Careful Consideration ◽  
Data Sets ◽  
Stream Data

We are living in the age of big data, a majority of which is stream data. The real-time processing of this data requires careful consideration from different perspectives. Concept drift is a change in the data’s underlying distribution, a significant issue, especially when learning from data streams. It requires learners to be adaptive to dynamic changes. Random forest is an ensemble approach that is widely used in classical non-streaming settings of machine learning applications. At the same time, the Adaptive Random Forest (ARF) is a stream learning algorithm that showed promising results in terms of its accuracy and ability to deal with various types of drift. The incoming instances’ continuity allows for their binomial distribution to be approximated to a Poisson(1) distribution. In this study, we propose a mechanism to increase such streaming algorithms’ efficiency by focusing on resampling. Our measure, resampling effectiveness (ρ), fuses the two most essential aspects in online learning; accuracy and execution time. We use six different synthetic data sets, each having a different type of drift, to empirically select the parameter λ of the Poisson distribution that yields the best value for ρ. By comparing the standard ARF with its tuned variations, we show that ARF performance can be enhanced by tackling this important aspect. Finally, we present three case studies from different contexts to test our proposed enhancement method and demonstrate its effectiveness in processing large data sets: (a) Amazon customer reviews (written in English), (b) hotel reviews (in Arabic), and (c) real-time aspect-based sentiment analysis of COVID-19-related tweets in the United States during April 2020. Results indicate that our proposed method of enhancement exhibited considerable improvement in most of the situations.

Online Clustering for Novelty Detection and Concept Drift in Data Streams

2019 ◽  
pp. 448-459
Author(s):  
Kemilly Dearo Garcia ◽  
Mannes Poel ◽  
Joost N. Kok ◽  
André C. P. L. F. de Carvalho
Keyword(s):  
Data Streams ◽  
Concept Drift ◽  
Novelty Detection ◽  
Online Clustering

Knowledge Discovery From Evolving Data Streams

2019 ◽  
pp. 19-39
Author(s):  
Prasanna Lakshmi Kompalli
Keyword(s):  
Real Time ◽  
Data Streams ◽  
Data Stream ◽  
Concept Drift ◽  
Data Stream Mining ◽  
Time Data ◽  
Stream Mining ◽  
New Challenges ◽  
Mining Data Streams ◽  
Different Sources

Data coming from different sources is referred to as data streams. Data stream mining is an online learning technique where each data point must be processed as the data arrives and discarded as the processing is completed. Progress of technologies has resulted in the monitoring these data streams in real time. Data streams has created many new challenges to the researchers in real time. The main features of this type of data are they are fast flowing, large amounts of data which are continuous and growing in nature, and characteristics of data might change in course of time which is termed as concept drift. This chapter addresses the problems in mining data streams with concept drift. Due to which, isolating the correct literature would be a grueling task for researchers and practitioners. This chapter tries to provide a solution as it would be an amalgamation of all techniques used for data stream mining with concept drift.

On learning guarantees to unsupervised concept drift detection on data streams

2019 ◽  
Vol 117 ◽  
pp. 90-102 ◽  
Author(s):  
Rodrigo F. de Mello ◽  
Yule Vaz ◽  
Carlos H. Grossi ◽  
Albert Bifet
Keyword(s):  
Data Streams ◽  
Concept Drift ◽  
Concept Drift Detection

scholarly journals Mining Massive E-Health Data Streams for IoMT Enabled Healthcare Systems

Sensors ◽  
2020 ◽  
Vol 20 (7) ◽  
pp. 2131 ◽  
Author(s):  
Affan Ahmed Toor ◽  
Muhammad Usman ◽  
Farah Younas ◽  
Alvis Cheuk M. Fong ◽  
Sajid Ali Khan ◽  
...  
Keyword(s):  
Data Streams ◽  
Detection Method ◽  
Concept Drift ◽  
Class Imbalance ◽  
Health Data ◽  
Smart Devices ◽  
Detection Delay ◽  
Medical Sensors ◽  
Synthetic Datasets ◽  
Almost All

With the increasing popularity of the Internet-of-Medical-Things (IoMT) and smart devices, huge volumes of data streams have been generated. This study aims to address the concept drift, which is a major challenge in the processing of voluminous data streams. Concept drift refers to overtime change in data distribution. It may occur in the medical domain, for example the medical sensors measuring for general healthcare or rehabilitation, which may switch their roles for ICU emergency operations when required. Detecting concept drifts becomes trickier when the class distributions in data are skewed, which is often true for medical sensors e-health data. Reactive Drift Detection Method (RDDM) is an efficient method for detecting long concepts. However, RDDM has a high error rate, and it does not handle class imbalance. We propose an Enhanced Reactive Drift Detection Method (ERDDM), which systematically generates strategies to handle concept drift with class imbalance in data streams. We conducted experiments to compare ERDDM with three contemporary techniques in terms of prediction error, drift detection delay, latency, and ability to handle data imbalance. The experimentation was done in Massive Online Analysis (MOA) on 48 synthetic datasets customized to possess the capabilities of data streams. ERDDM can handle abrupt and gradual drifts and performs better than all benchmarks in almost all experiments.

A novel concept drift detection method in data streams using ensemble classifiers

2016 ◽  
Vol 20 (6) ◽  
pp. 1329-1350 ◽  
Author(s):  
Mahdie Dehghan ◽  
Hamid Beigy ◽  
Poorya ZareMoodi
Keyword(s):  
Data Streams ◽  
Detection Method ◽  
Concept Drift ◽  
Ensemble Classifiers ◽  
Concept Drift Detection ◽  
Novel Concept
Sign in / Sign up

Export Citation Format

Share Document