scholarly journals Validity and efficiency of conformal anomaly detection on big distributed data

2017 ◽  
Vol 2 (3) ◽  
pp. 254-267
Author(s):  
Ilia Nouretdinov
Sensors ◽  
2018 ◽  
Vol 19 (1) ◽  
pp. 84 ◽  
Author(s):  
Yuejun Guo ◽  
Anton Bardera

To perform anomaly detection for trajectory data, we study the Sequential Hausdorff Nearest-Neighbor Conformal Anomaly Detector (SHNN-CAD) approach, and propose an enhanced version called SHNN-CAD +. SHNN-CAD was introduced based on the theory of conformal prediction dealing with the problem of online detection. Unlike most related approaches requiring several not intuitive parameters, SHNN-CAD has the advantage of being parameter-light which enables the easy reproduction of experiments. We propose to adaptively determine the anomaly threshold during the online detection procedure instead of predefining it without any prior knowledge, which makes the algorithm more usable in practical applications. We present a modified Hausdorff distance measure that takes into account the direction difference and also reduces the computational complexity. In addition, the anomaly detection is more flexible and accurate via a re-do strategy. Extensive experiments on both real-world and synthetic data show that SHNN-CAD + outperforms SHNN-CAD with regard to accuracy and running time.


2010 ◽  
Vol 09 (06) ◽  
pp. 935-957 ◽  
Author(s):  
JUNLIN ZHOU ◽  
ALEKSANDAR LAZAREVIC ◽  
KUO-WEI HSU ◽  
JAIDEEP SRIVASTAVA ◽  
YAN FU ◽  
...  

Anomaly detection has recently become an important problem in many industrial and financial applications. Very often, the databases from which anomalies have to be found are located at multiple local sites and cannot be merged due to privacy reasons or communication overhead. In this paper, a novel general framework for distributed anomaly detection is proposed. The proposed method consists of three steps: (i) building local models for distributed data sources with unsupervised anomaly detection methods and computing quality measure of local models; (ii) transforming local unsupervised local models into sharing models; and (iii) reusing sharing models for new data and combining their results by considering both quality and diversity of them to detect anomalies in a global view. In experiments performed on synthetic and real-life large data set, the proposed distributed anomaly detection method achieved prediction performance comparable or even slightly better than the global anomaly detection algorithm applied on the data set obtained when all distributed data set were merged.


Author(s):  
James Smith ◽  
Ilia Nouretdinov ◽  
Rachel Craddock ◽  
Charles Offer ◽  
Alexander Gammerman

2021 ◽  
Author(s):  
Shiraz Farouq ◽  
Stefan Byttner ◽  
Mohamed-Rafik Bouguelia ◽  
Henrik Gadd

Sign in / Sign up

Export Citation Format

Share Document