scholarly journals Network Anomaly Detection by Using a Time-Decay Closed Frequent Pattern

Information ◽  
2019 ◽  
Vol 10 (8) ◽  
pp. 262
Author(s):  
Ying Zhao ◽  
Junjun Chen ◽  
Di Wu ◽  
Jian Teng ◽  
Nabin Sharma ◽  
...  
Keyword(s):  
Anomaly Detection ◽  
Network Traffic ◽  
User Behavior ◽  
Frequent Pattern ◽  
Detection Methods ◽  
Frequent Patterns ◽  
Time Decay ◽  
Network Behavior ◽  
Detection Model ◽  
Network Anomaly Detection

Anomaly detection of network traffic flows is a non-trivial problem in the field of network security due to the complexity of network traffic. However, most machine learning-based detection methods focus on network anomaly detection but ignore the user anomaly behavior detection. In real scenarios, the anomaly network behavior may harm the user interests. In this paper, we propose an anomaly detection model based on time-decay closed frequent patterns to address this problem. The model mines closed frequent patterns from the network traffic of each user and uses a time-decay factor to distinguish the weight of current and historical network traffic. Because of the dynamic nature of user network behavior, a detection model update strategy is provided in the anomaly detection framework. Additionally, the closed frequent patterns can provide interpretable explanations for anomalies. Experimental results show that the proposed method can detect user behavior anomaly, and the network anomaly detection performance achieved by the proposed method is similar to the state-of-the-art methods and significantly better than the baseline methods.

scholarly journals Network Anomaly Detection System with Optimized DS Evidence Theory

2014 ◽  
Vol 2014 ◽  
pp. 1-13 ◽  
Author(s):  
Yuan Liu ◽  
Xiaofeng Wang ◽  
Kaiyu Liu
Keyword(s):  
Anomaly Detection ◽  
Detection Rate ◽  
Computer Network ◽  
Detection System ◽  
Evidence Theory ◽  
Optimization Methods ◽  
High Detection Rate ◽  
Detection Model ◽  
Network Anomaly Detection ◽  
Anomaly Detection System

Network anomaly detection has been focused on by more people with the fast development of computer network. Some researchers utilized fusion method and DS evidence theory to do network anomaly detection but with low performance, and they did not consider features of network—complicated and varied. To achieve high detection rate, we present a novel network anomaly detection system with optimized Dempster-Shafer evidence theory (ODS) and regression basic probability assignment (RBPA) function. In this model, we add weights for each senor to optimize DS evidence theory according to its previous predict accuracy. And RBPA employs sensor’s regression ability to address complex network. By four kinds of experiments, we find that our novel network anomaly detection model has a better detection rate, and RBPA as well as ODS optimization methods can improve system performance significantly.

scholarly journals ConAnomaly: Content-Based Anomaly Detection for System Logs

Sensors ◽  
2021 ◽  
Vol 21 (18) ◽  
pp. 6125
Author(s):  
Dan Lv ◽  
Nurbol Luktarhan ◽  
Yiyong Chen
Keyword(s):  
Anomaly Detection ◽  
Semantic Information ◽  
Short Term Memory ◽  
Weighted Average ◽  
Detection Methods ◽  
Detection Model ◽  
Part Of Speech Tagging ◽  
Part Of Speech ◽  
System Logs ◽  
System Maintenance

Enterprise systems typically produce a large number of logs to record runtime states and important events. Log anomaly detection is efficient for business management and system maintenance. Most existing log-based anomaly detection methods use log parser to get log event indexes or event templates and then utilize machine learning methods to detect anomalies. However, these methods cannot handle unknown log types and do not take advantage of the log semantic information. In this article, we propose ConAnomaly, a log-based anomaly detection model composed of a log sequence encoder (log2vec) and multi-layer Long Short Term Memory Network (LSTM). We designed log2vec based on the Word2vec model, which first vectorized the words in the log content, then deleted the invalid words through part of speech tagging, and finally obtained the sequence vector by the weighted average method. In this way, ConAnomaly not only captures semantic information in the log but also leverages log sequential relationships. We evaluate our proposed approach on two log datasets. Our experimental results show that ConAnomaly has good stability and can deal with unseen log types to a certain extent, and it provides better performance than most log-based anomaly detection methods.

A Hybrid Technique Using PCA and Wavelets in Network Traffic Anomaly Detection

2014 ◽  
Vol 6 (1) ◽  
pp. 17-53 ◽  
Author(s):  
Stevan Novakov ◽  
Chung-Horng Lung ◽  
Ioannis Lambadaris ◽  
Nabil Seddigh
Keyword(s):  
Spectral Analysis ◽  
Statistical Analysis ◽  
Wavelet Analysis ◽  
Anomaly Detection ◽  
Network Traffic ◽  
Hybrid Approach ◽  
Haar Wavelet ◽  
Wavelet Filtering ◽  
Analysis Technique ◽  
Network Anomaly Detection

Research into network anomaly detection has become crucial as a result of a significant increase in the number of computer attacks. Many approaches in network anomaly detection have been reported in the literature, but data or solutions typically are not freely available. Recently, a labeled network traffic flow dataset, Kyoto2006+, has been created and is publicly available. Most existing approaches using Kyoto2006+ for network anomaly detection apply various clustering techniques. This paper leverages existing well known statistical analysis and spectral analysis techniques for network anomaly detection. The first popular approach is a statistical analysis technique called Principal Component Analysis (PCA). PCA describes data in a new dimension to unlock otherwise hidden characteristics. The other well known spectral analysis technique is Haar Wavelet filtering analysis. It measures the amount and magnitude of abrupt changes in data. Both approaches have strengths and limitations. In response, this paper proposes a Hybrid PCA–Haar Wavelet Analysis. The hybrid approach first applies PCA to describe the data and then Haar Wavelet filtering for analysis. Based on prototyping and measurement, an investigation of the Hybrid PCA–Haar Wavelet Analysis technique is performed using the Kyoto2006+ dataset. The authors consider a number of parameters and present experimental results to demonstrate the effectiveness of the hybrid approach as compared to the two algorithms individually.

scholarly journals Learning Representations of Network Traffic Using Deep Neural Networks for Network Anomaly Detection: A Perspective towards Oil and Gas IT Infrastructures

Symmetry ◽  
2020 ◽  
Vol 12 (11) ◽  
pp. 1882
Author(s):  
Sheraz Naseer ◽  
Rao Faizan Ali ◽  
P.D.D Dominic ◽  
Yasir Saleem
Keyword(s):  
Anomaly Detection ◽  
Network Traffic ◽  
Oil And Gas ◽  
Resource Planning ◽  
Machine Learning Algorithms ◽  
Data Driven ◽  
Network Data ◽  
It Infrastructure ◽  
Industrial Automation ◽  
Network Anomaly Detection

Oil and Gas organizations are dependent on their IT infrastructure, which is a small part of their industrial automation infrastructure, to function effectively. The oil and gas (O&G) organizations industrial automation infrastructure landscape is complex. To perform focused and effective studies, Industrial systems infrastructure is divided into functional levels by The Instrumentation, Systems and Automation Society (ISA) Standard ANSI/ISA-95:2005. This research focuses on the ISA-95:2005 level-4 IT infrastructure to address network anomaly detection problem for ensuring the security and reliability of Oil and Gas resource planning, process planning and operations management. Anomaly detectors try to recognize patterns of anomalous behaviors from network traffic and their performance is heavily dependent on extraction time and quality of network traffic features or representations used to train the detector. Creating efficient representations from large volumes of network traffic to develop anomaly detection models is a time and resource intensive task. In this study we propose, implement and evaluate use of Deep learning to learn effective Network data representations from raw network traffic to develop data driven anomaly detection systems. Proposed methodology provides an automated and cost effective replacement of feature extraction which is otherwise a time and resource intensive task for developing data driven anomaly detectors. The ISCX-2012 dataset is used to represent ISA-95 level-4 network traffic because the O&G network traffic at this level is not much different than normal internet traffic. We trained four representation learning models using popular deep neural network architectures to extract deep representations from ISCX 2012 traffic flows. A total of sixty anomaly detectors were trained by authors using twelve conventional Machine Learning algorithms to compare the performance of aforementioned deep representations with that of a human-engineered handcrafted network data representation. The comparisons were performed using well known model evaluation parameters. Results showed that deep representations are a promising feature in engineering replacement to develop anomaly detection models for IT infrastructure security. In our future research, we intend to investigate the effectiveness of deep representations, extracted using ISA-95:2005 Level 2-3 traffic comprising of SCADA systems, for anomaly detection in critical O&G systems.

HELAD: A novel network anomaly detection model based on heterogeneous ensemble learning

2020 ◽  
Vol 169 ◽  
pp. 107049 ◽  
Author(s):  
Ying Zhong ◽  
Wenqi Chen ◽  
Zhiliang Wang ◽  
Yifan Chen ◽  
Kai Wang ◽  
...  
Keyword(s):  
Anomaly Detection ◽  
Ensemble Learning ◽  
Detection Model ◽  
Model Based ◽  
Heterogeneous Ensemble ◽  
Network Anomaly Detection

Network anomaly detection based on signal processing techniques

2013 ◽  
Vol 18 (1) ◽  
pp. 15-21
Author(s):  
Tomasz Andrysiak ◽  
Łukasz Saganowski ◽  
Mirosław Maszewski
Keyword(s):  
Signal Processing ◽  
Anomaly Detection ◽  
Network Traffic ◽  
Detection Method ◽  
Matching Pursuit ◽  
Matching Pursuit Decomposition ◽  
Network Anomaly Detection ◽  
Processing Techniques ◽  
Signal Processing Techniques

Abstract The article depicts possibility of using Matching Pursuit decomposition in order to recognize unspecified hazards in network traffic. Furthermore, the work aims to present feasible enhancements to the anomaly detection method, as well as their efficiency on the basis of a wide collection of pattern test traces.

scholarly journals A New Design of Custom Optimized Cnn-Lstm Assists to Detect Network Anomaly Using Categorical Data

2021 ◽  
Author(s):  
Kanmani R ◽  
A.Christy Jeba Malar ◽  
Roopa V ◽  
Ranjani D ◽  
Suganya R
Keyword(s):  
Feature Selection ◽  
Anomaly Detection ◽  
Categorical Data ◽  
Imbalanced Data ◽  
Experimental Result ◽  
Training Dataset ◽  
Detection Model ◽  
Effective Performance ◽  
Network Anomaly Detection ◽  
System Effectiveness

Abstract For traditional intrusion detection model, the system effectiveness is fully based on training dataset and feature selection. During feature selection, it needs more labour charge and trusted mainly on expert’s knowledge. Moreover, the training dataset contains more imbalanced data which in terms model tends to be biased. Here, an automatic approach is introduced to correct deficiency in the system. In this paper, the author proposes novel network anomaly detection (NID) build using categorical data. A model has to be designed with modified form of deep neural network primarily utilized for detecting anomaly within the network. Custom CNN-LSTM with Harris Hawks Optimization (named as custom optimized CNN-LSTM) is designed as a new classifier majorly used to detect the anomaly from word cloud to distinguish the data with effective performance. The experimental result shows that the proposed method achieves a promising output for network anomaly detection.

scholarly journals Network Traffic Anomaly Detection Based on ML-ESN for Power Metering System

2020 ◽  
Vol 2020 ◽  
pp. 1-21
Author(s):  
S. T. Zhang ◽  
X. B. Lin ◽  
L. Wu ◽  
Y. Q. Song ◽  
N. D. Liao ◽  
...  
Keyword(s):  
Anomaly Detection ◽  
Network Traffic ◽  
Network Flow ◽  
Small Sample ◽  
Detection Methods ◽  
Echo State Network ◽  
Power Network ◽  
Power Metering ◽  
Traffic Anomaly ◽  
Traffic Anomaly Detection

Due to the diversity and complexity of power network system platforms, some traditional network traffic detection methods work well for small sample datasets. However, the network data detection of complex power metering system platforms has problems of low accuracy and high false-positive rate. In this paper, through a combination of exploration and feedback, a solution for power network traffic anomaly detection based on multilayer echo state network (ML-ESN) is proposed. This method first relies on the Pearson and Gini coefficient method to calculate the statistical distribution and correlation of network flow characteristics and then uses the ML-ESN method to classify the network attacks abnormally. Because the ML-ESN method abandons the backpropagation mechanism, the nonlinear fitting ability of the model is solved. In order to verify the effectiveness of the proposed method, a simulation test was conducted on the UNSW_NB15 network security dataset. The test results show that the average accuracy of this method is more than 97%, which is significantly better than single-layer echo state network, shallow BP neural network, and some traditional machine learning methods.

scholarly journals Visualization of Anomalies using Graph-Based Anomaly Detection

2021 ◽  
Vol 34 (1) ◽  
Author(s):  
Ramesh Paudel ◽  
Lauren Tharp ◽  
Dulce Kaiser ◽  
William Eberle ◽  
Gerald Gannod
Keyword(s):  
Anomaly Detection ◽  
Network Traffic ◽  
False Positives ◽  
Visual Context ◽  
Traffic Flows ◽  
Security Analysts ◽  
Fast Analysis ◽  
Detection Systems ◽  
Potential Damage ◽  
Network Anomaly Detection

Network protocol analyzers such asWireshark are valuable for analyzing network traffic but pose a challenge in that it can be difficult to determine which behaviors are out of the ordinary due to the volume of data that must be analyzed. Network anomaly detection systems can provide vital insights to security analysts to supplement protocol analyzers, but this feedback can be difficult to interpret due to the complexity of the algorithms used and the lack of context to determine the reasoning for which an event was labeled as anomalous. We present an approach for visualizing anomalies using a graph-based anomaly detection methodology that aims to provide visual context to network traffic. We demonstrate the approach using network traffic flows as an approach for aiding in the investigation and triage of anomalous network events. The simplicity of a visual representation supports fast analysis of anomalous traffic to identify true positives from false positives and prevent further potential damage.

Sign in / Sign up

Export Citation Format

Share Document