scholarly journals Unsupervised Anomaly Detection with Distillated Teacher-Student Network Ensemble

Entropy ◽  
2021 ◽  
Vol 23 (2) ◽  
pp. 201
Author(s):  
Qinfeng Xiao ◽  
Jing Wang ◽  
Youfang Lin ◽  
Wenbo Gongsa ◽  
Ganghui Hu ◽  
...  
Keyword(s):  
Anomaly Detection ◽  
Multivariate Data ◽  
Failure Detection ◽  
Superior Performance ◽  
Detection Algorithms ◽  
Teacher Student ◽  
Model Complex ◽  
Unsupervised Anomaly Detection ◽  
Real World Datasets ◽  
Complex Features

We address the problem of unsupervised anomaly detection for multivariate data. Traditional machine learning based anomaly detection algorithms rely on specific assumptions of normal patterns and fail to model complex feature interactions and relations. Recently, existing deep learning based methods are promising for extracting representations from complex features. These methods train an auxiliary task, e.g., reconstruction and prediction, on normal samples. They further assume that anomalies fail to perform well on the auxiliary task since they are never trained during the model optimization. However, the assumption does not always hold in practice. Deep models may also perform the auxiliary task well on anomalous samples, leading to the failure detection of anomalies. To effectively detect anomalies for multivariate data, this paper introduces a teacher-student distillation based framework Distillated Teacher-Student Network Ensemble (DTSNE). The paradigm of the teacher-student distillation is able to deal with high-dimensional complex features. In addition, an ensemble of student networks provides a better capability to avoid generalizing the auxiliary task performance on anomalous samples. To validate the effectiveness of our model, we conduct extensive experiments on real-world datasets. Experimental results show superior performance of DTSNE over competing methods. Analysis and discussion towards the behavior of our model are also provided in the experiment section.

scholarly journals Multivariate anomaly detection for Earth observations: a comparison of algorithms and feature extraction techniques

2017 ◽  
Vol 8 (3) ◽  
pp. 677-696 ◽  
Author(s):  
Milan Flach ◽  
Fabian Gans ◽  
Alexander Brenning ◽  
Joachim Denzler ◽  
Markus Reichstein ◽  
...  
Keyword(s):  
Feature Extraction ◽  
Anomaly Detection ◽  
Data Streams ◽  
Multivariate Data ◽  
Earth System ◽  
Earth System Science ◽  
Sample Mean ◽  
System Science ◽  
Detection Algorithms ◽  
Earth Observations

Abstract. Today, many processes at the Earth's surface are constantly monitored by multiple data streams. These observations have become central to advancing our understanding of vegetation dynamics in response to climate or land use change. Another set of important applications is monitoring effects of extreme climatic events, other disturbances such as fires, or abrupt land transitions. One important methodological question is how to reliably detect anomalies in an automated and generic way within multivariate data streams, which typically vary seasonally and are interconnected across variables. Although many algorithms have been proposed for detecting anomalies in multivariate data, only a few have been investigated in the context of Earth system science applications. In this study, we systematically combine and compare feature extraction and anomaly detection algorithms for detecting anomalous events. Our aim is to identify suitable workflows for automatically detecting anomalous patterns in multivariate Earth system data streams. We rely on artificial data that mimic typical properties and anomalies in multivariate spatiotemporal Earth observations like sudden changes in basic characteristics of time series such as the sample mean, the variance, changes in the cycle amplitude, and trends. This artificial experiment is needed as there is no gold standard for the identification of anomalies in real Earth observations. Our results show that a well-chosen feature extraction step (e.g., subtracting seasonal cycles, or dimensionality reduction) is more important than the choice of a particular anomaly detection algorithm. Nevertheless, we identify three detection algorithms (k-nearest neighbors mean distance, kernel density estimation, a recurrence approach) and their combinations (ensembles) that outperform other multivariate approaches as well as univariate extreme-event detection methods. Our results therefore provide an effective workflow to automatically detect anomalies in Earth system science data.

scholarly journals Multivariate Anomaly Detection for Earth Observations: A Comparison of Algorithms and Feature Extraction Techniques

2016 ◽  
Author(s):  
Milan Flach ◽  
Fabian Gans ◽  
Alexander Brenning ◽  
Joachim Denzler ◽  
Markus Reichstein ◽  
...  
Keyword(s):  
Feature Extraction ◽  
Anomaly Detection ◽  
Data Streams ◽  
Multivariate Data ◽  
Detection Methods ◽  
Earth System ◽  
Earth System Science ◽  
System Science ◽  
Detection Algorithms ◽  
Earth Observations

Abstract. Today, many processes at the Earth's surface are constantly monitored by multiple data streams. These observations have become central to advance our understanding of e.g. vegetation dynamics in response to climate or land use change. Another set of important applications is monitoring effects of climatic extreme events, other disturbances such as fires, or abrupt land transitions. One important methodological question is how to reliably detect anomalies in an automated and generic way within multivariate data streams, which typically vary seasonally and are interconnected across variables. Although many algorithms have been proposed for detecting anomalies in multivariate data, only few have been investigated in the context of Earth system science applications. In this study, we systematically combine and compare feature extraction and anomaly detection algorithms for detecting anomalous events. Our aim is to identify suitable workflows for automatically detecting anomalous patterns in multivariate Earth system data streams. We rely on artificial data that mimic typical properties and anomalies in multivariate spatiotemporal Earth observations. This artificial experiment is needed as there is no 'gold standard' for the identification of anomalies in real Earth observations. Our results show that a well chosen feature extraction step (e.g. subtracting seasonal cycles, or dimensionality reduction) is more important than the choice of a particular anomaly detection algorithm. Nevertheless, we identify 3 detection algorithms (k-nearest neighbours mean distance, kernel density estimation, a recurrence approach) and their combinations (ensembles) that outperform other multivariate approaches as well as univariate extreme event detection methods. Our results therefore provide an effective workflow to automatically detect anomalies in Earth system science data.

scholarly journals Add-On Anomaly Threshold Technique for Improving Unsupervised Intrusion Detection on SCADA Data

Electronics ◽  
2020 ◽  
Vol 9 (6) ◽  
pp. 1017 ◽  
Author(s):  
Abdulmohsen Almalawi ◽  
Adil Fahad ◽  
Zahir Tari ◽  
Asif Irshad Khan ◽  
Nouf Alzahrani ◽  
...  
Keyword(s):  
Anomaly Detection ◽  
Supervisory Control ◽  
State Of The Art ◽  
Current Approach ◽  
Industrial Processes ◽  
Parameter Choice ◽  
Detection Algorithms ◽  
Detection Approach ◽  
Unsupervised Anomaly Detection ◽  
Threshold Technique

Supervisory control and data acquisition (SCADA) systems monitor and supervise our daily infrastructure systems and industrial processes. Hence, the security of the information systems of critical infrastructures cannot be overstated. The effectiveness of unsupervised anomaly detection approaches is sensitive to parameter choices, especially when the boundaries between normal and abnormal behaviours are not clearly distinguishable. Therefore, the current approach in detecting anomaly for SCADA is based on the assumptions by which anomalies are defined; these assumptions are controlled by a parameter choice. This paper proposes an add-on anomaly threshold technique to identify the observations whose anomaly scores are extreme and significantly deviate from others, and then such observations are assumed to be ”abnormal”. The observations whose anomaly scores are significantly distant from ”abnormal” ones will be assumed as ”normal”. Then, the ensemble-based supervised learning is proposed to find a global and efficient anomaly threshold using the information of both ”normal”/”abnormal” behaviours. The proposed technique can be used for any unsupervised anomaly detection approach to mitigate the sensitivity of such parameters and improve the performance of the SCADA unsupervised anomaly detection approaches. Experimental results confirm that the proposed technique achieved a significant improvement compared to the state-of-the-art of two unsupervised anomaly detection algorithms.

scholarly journals Evaluation of Unsupervised Anomaly Detection Methods in Sentiment Mining

2019 ◽  
Vol 8 (9) ◽  
pp. 1080-1085
Keyword(s):  
Anomaly Detection ◽  
Computation Time ◽  
Vital Role ◽  
Detection Methods ◽  
Research Focus ◽  
Comparative Performance ◽  
Detection Techniques ◽  
Detection Algorithms ◽  
Sentiment Mining ◽  
Unsupervised Anomaly Detection

Anomaly detection has vital role in data preprocessing and also in the mining of outstanding points for marketing, network sensors, fraud detection, intrusion detection, stock market analysis. Recent studies have been found to concentrate more on outlier detection for real time datasets. Anomaly detection study is at present focuses on the expansion of innovative machine learning methods and on enhancing the computation time. Sentiment mining is the process to discover how people feel about a particular topic. Though many anomaly detection techniques have been proposed, it is also notable that the research focus lacks a comparative performance evaluation in sentiment mining datasets. In this study, three popular unsupervised anomaly detection algorithms such as density based, statistical based and cluster based anomaly detection methods are evaluated on movie review sentiment mining dataset. This paper will set a base for anomaly detection methods in sentiment mining research. The results show that density based (LOF) anomaly detection method suits best for the movie review sentiment dataset.

An Ensemble Approach for Detecting Anomalous User Behaviors

2018 ◽  
Vol 28 (11n12) ◽  
pp. 1637-1656
Author(s):  
Xiangyu Xi ◽  
Tong Zhang ◽  
Wei Ye ◽  
Zhao Wen ◽  
Shikun Zhang ◽  
...  
Keyword(s):  
Anomaly Detection ◽  
User Behavior ◽  
Behavior Patterns ◽  
Training Set ◽  
Sensitive Data ◽  
False Negatives ◽  
Detection Algorithms ◽  
User Behaviors ◽  
Unsupervised Anomaly Detection ◽  
Isolation Forest

An intruder of a company’s network may use stolen login credentials to silently collect sensitive data. Such malicious user behavior is difficult to detect as long as it does not trigger access violation or data leak alert. In this paper, we propose to use an ensemble of three unsupervised anomaly detection algorithms, namely OCSVM, RNN and Isolation Forest, to detect abnormal user behavior patterns. Besides, an User Behavior Analytics (UBA) Platform is proposed to collect logs, extract features and conduct experiments. The experiment results indicate that our algorithm outperforms each individual algorithm with recall of 96.55% and precision of 91.24% on average, while both OCSVM and RNN suffer from anomalies in the training set, and [Formula: see text] produces more false positives and false negatives in prediction.

scholarly journals A Novel Active Anomaly Discovery Method and its Applications in Additive Manufacturing

2021 ◽  
Author(s):  
Bo Shen ◽  
Zhenyu Kong
Keyword(s):  
Additive Manufacturing ◽  
Anomaly Detection ◽  
State Of The Art ◽  
Detection Algorithm ◽  
The State ◽  
Detection Accuracy ◽  
Detection Algorithms ◽  
Detection Analysis ◽  
Unsupervised Anomaly Detection ◽  
Discovery Method

Anomaly detection aims to identify the true anomalies from a given set of data instances. Unsupervised anomaly detection algorithms are applied to an unlabeled dataset by producing a ranked list based on anomaly scores. Unfortunately, due to the inherent limitations, many of the top-ranked instances by unsupervised algorithms are not anomalies or not interesting from an application perspective, which leads to high false-positive rates. Active anomaly discovery (AAD) is proposed to overcome this deficiency, which sequentially selects instances to get the labeling information and incorporate it into the anomaly detection algorithm to improve the detection accuracy iteratively. However, labeling is often costly. Therefore, the way to balance detection accuracy and labeling cost is essential. Along this line, this paper proposes a novel AAD method to achieve the goal. Our approach is based on the state-of-the-art unsupervised anomaly detection algorithm, namely, Isolation Forest, to extract features. Thereafter, the sparsity of the extracted features is utilized to improve its anomaly detection performance. To enforce the sparsity of the features and subsequent improvement of the detection analysis, a new algorithm based on online gradient descent, namely, Sparse Approximated Linear Anomaly Discovery (SALAD), is proposed with its theoretical Regret analysis. Extensive experiments on both open-source and additive manufacturing datasets demonstrate that the proposed algorithm significantly outperforms the state-of-the-art algorithms for anomaly detection.

Sign in / Sign up

Export Citation Format

Share Document