Log Sequence Anomaly Detection Method Based on Contrastive Adversarial Training and Dual Feature Extraction

The log messages generated in the system reflect the state of the system at all times. The realization of autonomous detection of abnormalities in log messages can help operators find abnormalities in time and provide a basis for analyzing the causes of abnormalities. First, this paper proposes a log sequence anomaly detection method based on contrastive adversarial training and dual feature extraction. This method uses BERT (Bidirectional Encoder Representations from Transformers) and VAE (Variational Auto-Encoder) to extract the semantic features and statistical features of the log sequence, respectively, and the dual features are combined to perform anomaly detection on the log sequence, with a novel contrastive adversarial training method also used to train the model. In addition, this paper introduces the method of obtaining statistical features of log sequence and the method of combining semantic features with statistical features. Furthermore, the specific process of contrastive adversarial training is described. Finally, an experimental comparison is carried out, and the experimental results show that the method in this paper is better than the contrasted log sequence anomaly detection method.

Download Full-text

Experimental Comparison and Survey of Twelve Time Series Anomaly Detection Algorithms

Journal of Artificial Intelligence Research ◽

10.1613/jair.1.12698 ◽

2021 ◽

Vol 72 ◽

pp. 849-899

Author(s):

Cynthia Freeman ◽

Jonathan Merriman ◽

Ian Beaver ◽

Abdullah Mueen

Keyword(s):

Time Series ◽

Anomaly Detection ◽

Detection Method ◽

Concept Drift ◽

Area Under The Curve ◽

Detection Methods ◽

Experimental Comparison ◽

Detection Algorithms ◽

Extensive Evaluation ◽

Online Setting

The existence of an anomaly detection method that is optimal for all domains is a myth. Thus, there exists a plethora of anomaly detection methods which increases every year for a wide variety of domains. But a strength can also be a weakness; given this massive library of methods, how can one select the best method for their application? Current literature is focused on creating new anomaly detection methods or large frameworks for experimenting with multiple methods at the same time. However, and especially as the literature continues to expand, an extensive evaluation of every anomaly detection method is simply not feasible. To reduce this evaluation burden, we present guidelines to intelligently choose the optimal anomaly detection methods based on the characteristics the time series displays such as seasonality, trend, level change concept drift, and missing time steps. We provide a comprehensive experimental validation and survey of twelve anomaly detection methods over different time series characteristics to form guidelines based on several metrics: the AUC (Area Under the Curve), windowed F-score, and Numenta Anomaly Benchmark (NAB) scoring model. Applying our methodologies can save time and effort by surfacing the most promising anomaly detection methods instead of experimenting extensively with a rapidly expanding library of anomaly detection methods, especially in an online setting.

Download Full-text

PQ Event Classification Using Fuzzy Classifier

Advanced Materials Research ◽

10.4028/www.scientific.net/amr.403-408.3854 ◽

2011 ◽

Vol 403-408 ◽

pp. 3854-3858 ◽

Cited By ~ 2

Author(s):

Manish Kumar Saini ◽

Rajiv Kapoor ◽

Bharat Bhushan Sharma

Keyword(s):

Feature Extraction ◽

Classification Scheme ◽

Scaling Function ◽

Classification Performance ◽

Correct Classification ◽

Statistical Features ◽

Fuzzy Classifier ◽

Event Classification ◽

Multiwavelet Transform ◽

Better Than

The work presented here uses multiwavelet because of its inherent property to resolve the signal better than all single wavelets. Multiwavelets are based on more than one scaling function. The proposed methodology utilizes an enhanced resolving capability of multiwavelet to recognize power quality events. PQ events classification scheme is performed using multiwavelet transform for feature extraction and fuzzy classifier for classification. In proposed algorithm,statistical features (.i.e. mean, standard deviation, variation etc.) and energy of the signal at different decomposition levels have been considered as feature vectors. The performance of fuzzy classifier has been evaluated by using total 1000 PQ disturbance signals which are generated using the based model. The classification performance of different PQ events using proposed algorithm has been tested. The rate of average correct classification is about 99.95% for the different PQ disturbance signals and noisy disturbances.

Download Full-text