ADINOF: adaptive density summarizing incremental natural outlier detection in data stream

2021 ◽  
Author(s):  
Manmohan Singh ◽  
Rajendra Pamula
Keyword(s):  
Outlier Detection ◽  
Data Stream

scholarly journals FAAD: A Self-Optimizing Algorithm for Anomaly Detection

2019 ◽  
Vol 17 (2) ◽  
pp. 272-280
Author(s):  
Adeel Hashmi ◽  
Tanvir Ahmad
Keyword(s):  
Big Data ◽  
Anomaly Detection ◽  
Unsupervised Learning ◽  
Outlier Detection ◽  
Data Stream ◽  
Firefly Algorithm ◽  
Learning Approach ◽  
Detection Algorithms ◽  
Data Points ◽  
Optimizing Algorithm

Anomaly/Outlier detection is the process of finding abnormal data points in a dataset or data stream. Most of the anomaly detection algorithms require setting of some parameters which significantly affect the performance of the algorithm. These parameters are generally set by hit-and-trial; hence performance is compromised with default or random values. In this paper, the authors propose a self-optimizing algorithm for anomaly detection based on firefly meta-heuristic, and named as Firefly Algorithm for Anomaly Detection (FAAD). The proposed solution is a non-clustering unsupervised learning approach for anomaly detection. The algorithm is implemented on Apache Spark for scalability and hence the solution can handle big data as well. Experiments were conducted on various datasets, and the results show that the proposed solution is much accurate than the standard algorithms of anomaly detection.

scholarly journals Survey on Outlier Detection in Data Stream

2016 ◽  
Vol 136 (2) ◽  
pp. 13-16 ◽  
Author(s):  
Pooja Thakkar ◽  
Jay Vala ◽  
Vishal Prajapati
Keyword(s):  
Outlier Detection ◽  
Data Stream

An efficient outlier detection approach on weighted data stream based on minimal rare pattern mining

2019 ◽  
Vol 16 (10) ◽  
pp. 83-99 ◽  
Author(s):  
Saihua Cai ◽  
Ruizhi Sun ◽  
Shangbo Hao ◽  
Sicong Li ◽  
Gang Yuan
Keyword(s):  
Outlier Detection ◽  
Data Stream ◽  
Pattern Mining ◽  
Detection Approach ◽  
Weighted Data

scholarly journals TADILOF: Time Aware Density-Based Incremental Local Outlier Detection in Data Streams

Sensors ◽  
2020 ◽  
Vol 20 (20) ◽  
pp. 5829 ◽  
Author(s):  
Jen-Wei Huang ◽  
Meng-Xun Zhong ◽  
Bijay Prasad Jaysawal
Keyword(s):  
Outlier Detection ◽  
Data Streams ◽  
Data Stream ◽  
State Of The Art ◽  
Streaming Data ◽  
Current State ◽  
Data Points ◽  
Local Outlier ◽  
Time Aware ◽  
Over Time

Outlier detection in data streams is crucial to successful data mining. However, this task is made increasingly difficult by the enormous growth in the quantity of data generated by the expansion of Internet of Things (IoT). Recent advances in outlier detection based on the density-based local outlier factor (LOF) algorithms do not consider variations in data that change over time. For example, there may appear a new cluster of data points over time in the data stream. Therefore, we present a novel algorithm for streaming data, referred to as time-aware density-based incremental local outlier detection (TADILOF) to overcome this issue. In addition, we have developed a means for estimating the LOF score, termed "approximate LOF," based on historical information following the removal of outdated data. The results of experiments demonstrate that TADILOF outperforms current state-of-the-art methods in terms of AUC while achieving similar performance in terms of execution time. Moreover, we present an application of the proposed scheme to the development of an air-quality monitoring system.

scholarly journals Research on Outlier Detection Algorithm for Evaluation of Battery System Safety

2014 ◽  
Vol 6 ◽  
pp. 830402 ◽  
Author(s):  
Changhao Piao ◽  
Zhi Huang ◽  
Ling Su ◽  
Sheng Lu
Keyword(s):  
Outlier Detection ◽  
Data Stream ◽  
Concept Drift ◽  
Detection Algorithm ◽  
High Dimensional ◽  
Small Scale ◽  
Data Sets ◽  
Angle Distribution ◽  
System Safety ◽  
Battery System

Battery system is the key part of the electric vehicle. To realize outlier detection in the running process of battery system effectively, a new high-dimensional data stream outlier detection algorithm (DSOD) based on angle distribution is proposed. First, in order to improve the algorithm stability in high-dimensional space, the method of angle distribution-based outlier detection algorithm is employed. Second, to reduce the computational complexity, a small-scale calculation set of data stream is established, which is composed of normal set and border set. For the purpose of solving the problem of concept drift, an update mechanism for the normal set and border set is developed in this paper. By this way, these hidden abnormal points will be rapidly detected. The experimental results on real data sets and battery system simulation data sets demonstrate that DSOD is more efficient than Simple variance of angles (Simple VOA) and angle-based outlier detection (ABOD) and is very suitable for the evaluation of battery system safety.

Sign in / Sign up

Export Citation Format

Share Document