scholarly journals MEOD: Memory-Efficient Outlier Detection on Streaming Data

Symmetry ◽  
2021 ◽  
Vol 13 (3) ◽  
pp. 458
Author(s):  
Ankita Karale ◽  
Milena Lazarova ◽  
Pavlina Koleva ◽  
Vladimir Poulkov
Keyword(s):  
Outlier Detection ◽  
Optimization Problem ◽  
Streaming Data ◽  
Local Correlation ◽  
Detection Problem ◽  
Swarm Optimization ◽  
Correlation Integral ◽  
Memory Time ◽  
Local Outlier ◽  
Memory Efficient

In this paper, a memory-efficient outlier detection (MEOD) approach for streaming data is proposed. The approach uses a local correlation integral (LOCI) algorithm for outlier detection, finding the outlier based on the density of neighboring points defined by a given radius. The radius value detection problem is converted into an optimization problem. The radius value is determined using a particle swarm optimization (PSO)-based approach. The results of the MEOD technique application are compared with existing approaches in terms of memory, time, and accuracy, such as the memory-efficient incremental local outlier factor (MiLOF) detection technique. The MEOD technique finds outlier points similar to MiLOF with nearly equal accuracy but requires less memory for processing.

scholarly journals TADILOF: Time Aware Density-Based Incremental Local Outlier Detection in Data Streams

Sensors ◽  
2020 ◽  
Vol 20 (20) ◽  
pp. 5829 ◽  
Author(s):  
Jen-Wei Huang ◽  
Meng-Xun Zhong ◽  
Bijay Prasad Jaysawal
Keyword(s):  
Outlier Detection ◽  
Data Streams ◽  
Data Stream ◽  
State Of The Art ◽  
Streaming Data ◽  
Current State ◽  
Data Points ◽  
Local Outlier ◽  
Time Aware ◽  
Over Time

Outlier detection in data streams is crucial to successful data mining. However, this task is made increasingly difficult by the enormous growth in the quantity of data generated by the expansion of Internet of Things (IoT). Recent advances in outlier detection based on the density-based local outlier factor (LOF) algorithms do not consider variations in data that change over time. For example, there may appear a new cluster of data points over time in the data stream. Therefore, we present a novel algorithm for streaming data, referred to as time-aware density-based incremental local outlier detection (TADILOF) to overcome this issue. In addition, we have developed a means for estimating the LOF score, termed "approximate LOF," based on historical information following the removal of outdated data. The results of experiments demonstrate that TADILOF outperforms current state-of-the-art methods in terms of AUC while achieving similar performance in terms of execution time. Moreover, we present an application of the proposed scheme to the development of an air-quality monitoring system.

Fast Memory Efficient Local Outlier Detection in Data Streams (Extended Abstract)

2017 ◽  
Author(s):  
Mahsa Salehi ◽  
Christopher Leckie ◽  
James C. Bezdek ◽  
Tharshan Vaithianathan ◽  
Xuyun Zhang
Keyword(s):  
Outlier Detection ◽  
Data Streams ◽  
Fast Memory ◽  
Local Outlier ◽  
Memory Efficient

scholarly journals Genetic-based Summarization for Local Outlier Detection in Data Stream

2021 ◽  
Vol 13 (1) ◽  
pp. 58-68
Author(s):  
Mohamed Sakr ◽  
◽  
Walid Atwa ◽  
Arabi Keshk
Keyword(s):  
Outlier Detection ◽  
Data Streams ◽  
Approximate Solutions ◽  
Streaming Data ◽  
Detection Algorithms ◽  
Processing Power ◽  
Static Data ◽  
Large Memory ◽  
Two Phases ◽  
Local Outlier

Outlier detection is one of the important tasks in data mining. Detecting outliers over streaming data has become an important task in many applications, such as network analysis, fraud detections, and environment monitoring. One of the well-known outlier detection algorithms called Local Outlier Factor (LOF). However, the original LOF has many drawbacks that can’t be used with data streams: 1- it needs a lot of processing power (CPU) and large memory to detect the outliers. 2- it deals with static data which mean that in any change in data the LOF recalculates the outliers from the beginning on the whole data. These drawbacks make big challenges for existing outlier detection algorithms in terms of their accuracies when they are implemented in the streaming environment. In this paper, we propose a new algorithm called GSILOF that focuses on detecting outliers from data streams using genetics. GSILOF solve the problem of large memory needed as it has fixed memory bound. GSILOF has two phases. First, the summarization phase that tries to summarize the past data arrived. Second, the detection phase detects the outliers from the new arriving data. The summarization phase uses a genetic algorithm to try to find the subset of points that can represent the whole original set. our experiments have been done over real datasets. Our experiments confirming the effectiveness of the proposed approach and the high quality of approximate solutions in a set of real-world streaming data.

Fast Memory Efficient Local Outlier Detection in Data Streams

2016 ◽  
Vol 28 (12) ◽  
pp. 3246-3260 ◽  
Author(s):  
Mahsa Salehi ◽  
Christopher Leckie ◽  
James C. Bezdek ◽  
Tharshan Vaithianathan ◽  
Xuyun Zhang
Keyword(s):  
Outlier Detection ◽  
Data Streams ◽  
Fast Memory ◽  
Local Outlier ◽  
Memory Efficient

scholarly journals Robust CNN Compression Framework for Security-Sensitive Embedded Systems

2021 ◽  
Vol 11 (3) ◽  
pp. 1093
Author(s):  
Jeonghyun Lee ◽  
Sangkyun Lee
Keyword(s):  
Embedded Systems ◽  
Optimization Problem ◽  
State Of The Art ◽  
Classification Problems ◽  
Proximal Gradient Method ◽  
Knowledge Distillation ◽  
New Type ◽  
Adversarial Examples ◽  
Adversarial Training ◽  
Memory Efficient

Convolutional neural networks (CNNs) have achieved tremendous success in solving complex classification problems. Motivated by this success, there have been proposed various compression methods for downsizing the CNNs to deploy them on resource-constrained embedded systems. However, a new type of vulnerability of compressed CNNs known as the adversarial examples has been discovered recently, which is critical for security-sensitive systems because the adversarial examples can cause malfunction of CNNs and can be crafted easily in many cases. In this paper, we proposed a compression framework to produce compressed CNNs robust against such adversarial examples. To achieve the goal, our framework uses both pruning and knowledge distillation with adversarial training. We formulate our framework as an optimization problem and provide a solution algorithm based on the proximal gradient method, which is more memory-efficient than the popular ADMM-based compression approaches. In experiments, we show that our framework can improve the trade-off between adversarial robustness and compression rate compared to the existing state-of-the-art adversarial pruning approach.

Sign in / Sign up

Export Citation Format

Share Document