AECID: A Light-Weight Log Analysis Approach for Online Anomaly Detection

2021 ◽  
pp. 99-129
Author(s):  
Florian Skopik ◽  
Markus Wurzenberger ◽  
Max Landauer
Keyword(s):  
Anomaly Detection ◽  
Analysis Approach ◽  
Log Analysis ◽  
Light Weight ◽  
Online Anomaly Detection

Anomaly Detection via Mining Numerical Workflow Relations from Logs

2020 ◽  
Author(s):  
Bo Zhang ◽  
Hongyu Zhang ◽  
Pablo Moscato
Keyword(s):  
Distributed Systems ◽  
Anomaly Detection ◽  
Text Messages ◽  
Distributed File System ◽  
Log Data ◽  
Sliding Windows ◽  
Novel Approach ◽  
Hadoop Distributed File System ◽  
Blue Gene ◽  
Online Anomaly Detection

<div>Complex software intensive systems, especially distributed systems, generate logs for troubleshooting. The logs are text messages recording system events, which can help engineers determine the system's runtime status. This paper proposes a novel approach named ADR (stands for Anomaly Detection by workflow Relations) that employs matrix nullspace to mine numerical relations from log data. The mined relations can be used for both offline and online anomaly detection and facilitate fault diagnosis. We have evaluated ADR on log data collected from two distributed systems, HDFS (Hadoop Distributed File System) and BGL (IBM Blue Gene/L supercomputers system). ADR successfully mined 87 and 669 numerical relations from the logs and used them to detect anomalies with high precision and recall. For online anomaly detection, ADR employs PSO (Particle Swarm Optimization) to find the optimal sliding windows' size and achieves fast anomaly detection.</div><div>The experimental results confirm that ADR is effective for both offline and online anomaly detection. </div>

scholarly journals A flexible framework for anomaly Detection via dimensionality reduction

2021 ◽  
Author(s):  
Alireza Vafaei Sadr ◽  
Bruce A. Bassett ◽  
M. Kunz
Keyword(s):  
Anomaly Detection ◽  
Dimensionality Reduction ◽  
Dimensional Space ◽  
High Dimensions ◽  
Detection Algorithms ◽  
Latent Space ◽  
Wide Range ◽  
Flexible Framework ◽  
Online Anomaly Detection ◽  
Python Package

AbstractAnomaly detection is challenging, especially for large datasets in high dimensions. Here, we explore a general anomaly detection framework based on dimensionality reduction and unsupervised clustering. DRAMA is released as a general python package that implements the general framework with a wide range of built-in options. This approach identifies the primary prototypes in the data with anomalies detected by their large distances from the prototypes, either in the latent space or in the original, high-dimensional space. DRAMA is tested on a wide variety of simulated and real datasets, in up to 3000 dimensions, and is found to be robust and highly competitive with commonly used anomaly detection algorithms, especially in high dimensions. The flexibility of the DRAMA framework allows for significant optimization once some examples of anomalies are available, making it ideal for online anomaly detection, active learning, and highly unbalanced datasets. Besides, DRAMA naturally provides clustering of outliers for subsequent analysis.

Unsupervised Online Anomaly Detection to Identify Cyber-Attacks on Internet Connected Photovoltaic System Inverters

2021 ◽  
Author(s):  
C. Birk Jones ◽  
Adrian Chavez ◽  
Shamina Hossain-McKenzie ◽  
Nicholas Jacobs ◽  
Adam Summers ◽  
...  
Keyword(s):  
Anomaly Detection ◽  
Cyber Attacks ◽  
Photovoltaic System ◽  
Online Anomaly Detection

A Statistical-Based Light-Weight Anomaly Detection Framework for Wireless Body Area Networks

2021 ◽  
Author(s):  
Smrithy G S ◽  
Ramadoss Balakrishnan
Keyword(s):  
Anomaly Detection ◽  
Patient Monitoring ◽  
Body Area Networks ◽  
Wireless Body Area Networks ◽  
Physiological Parameters ◽  
Light Weight ◽  
Remote Patient Monitoring ◽  
Body Area ◽  
Statistical Measures ◽  
Remote Patient

Abstract In healthcare scenario, the major challenge in anomaly detection for remote patient monitoring is to classify true medical conditions and false alarms. This paper proposes a light-weight anomaly detection (LWAD) framework for detecting anomalies in remote patient monitoring based on wireless body area networks. The proposed framework uses distance correlation for finding correlated (both linear and non-linear) physiological parameters. It also uses a statistical-based improvised dynamic sliding window algorithm for efficient short-range prediction of physiological parameters. Finally, the proposed LWAD framework detects anomalies using anomaly detection framework based on robust statistical techniques. The validation of LWAD framework is performed using three real time datasets with various statistical measures. The proposed LWAD framework outperforms existing methods.

scholarly journals Online Anomaly Detection of Distillation Tower System Using Adaptive Resonance Theory

2017 ◽  
Vol 50 (6) ◽  
pp. 430-438 ◽  
Author(s):  
Yoshinari Hori ◽  
Hiroki Yamamoto ◽  
Tomoko Suzuki ◽  
Jun Okitsu ◽  
Tomohiro Nakamura ◽  
...  
Keyword(s):  
Anomaly Detection ◽  
Adaptive Resonance Theory ◽  
Distillation Tower ◽  
Resonance Theory ◽  
Adaptive Resonance ◽  
Online Anomaly Detection ◽  
Tower System
Sign in / Sign up

Export Citation Format

Share Document