The evaluation of Network Anomaly Detection Systems: Statistical analysis of the UNSW-NB15 data set and the comparison with the KDD99 data set

2016 ◽  
Vol 25 (1-3) ◽  
pp. 18-31 ◽  
Author(s):  
Nour Moustafa ◽  
Jill Slay
Keyword(s):  
Statistical Analysis ◽  
Anomaly Detection ◽  
Data Set ◽  
Detection Systems ◽  
Network Anomaly Detection

scholarly journals Machine Learning Based Network Anomaly Detection

2019 ◽  
Vol 8 (4) ◽  
pp. 542-548 ◽  
Keyword(s):  
Anomaly Detection ◽  
Dynamic Change ◽  
Model Performance ◽  
Accuracy Score ◽  
Traffic Data ◽  
Two Phase ◽  
Data Set ◽  
Detection Systems ◽  
Network Anomaly Detection ◽  
Tools And Techniques

Network Anomaly Detection Systems (NADSs) play prominent role in network security. Due to dynamic change of malware in network traffic data, traditional tools and techniques are failing to protect networks from attack penetration. In this paper we propose a two-phase model to detect and categorize anomalies. First, we selected Random Forest based on the highest accuracy-score out of eleven commonly used algorithms tested with the same set of data. The RF is used to detect anomalies and generate an extra feature named “attack-or-not”. Secondly we fed Neural Network with the data having “attack-or-not” feature to differentiate attack categories, which will help treating each type accordingly. The model performance was good, it scored 0.99 for both Precision and Recall in anomaly detection phase and 0.93 for Precision and 0.88 for Recall in attack categorization phase. We used UNSW-NB15 data set in our study.

A Hybrid Technique Using PCA and Wavelets in Network Traffic Anomaly Detection

2014 ◽  
Vol 6 (1) ◽  
pp. 17-53 ◽  
Author(s):  
Stevan Novakov ◽  
Chung-Horng Lung ◽  
Ioannis Lambadaris ◽  
Nabil Seddigh
Keyword(s):  
Spectral Analysis ◽  
Statistical Analysis ◽  
Wavelet Analysis ◽  
Anomaly Detection ◽  
Network Traffic ◽  
Hybrid Approach ◽  
Haar Wavelet ◽  
Wavelet Filtering ◽  
Analysis Technique ◽  
Network Anomaly Detection

Research into network anomaly detection has become crucial as a result of a significant increase in the number of computer attacks. Many approaches in network anomaly detection have been reported in the literature, but data or solutions typically are not freely available. Recently, a labeled network traffic flow dataset, Kyoto2006+, has been created and is publicly available. Most existing approaches using Kyoto2006+ for network anomaly detection apply various clustering techniques. This paper leverages existing well known statistical analysis and spectral analysis techniques for network anomaly detection. The first popular approach is a statistical analysis technique called Principal Component Analysis (PCA). PCA describes data in a new dimension to unlock otherwise hidden characteristics. The other well known spectral analysis technique is Haar Wavelet filtering analysis. It measures the amount and magnitude of abrupt changes in data. Both approaches have strengths and limitations. In response, this paper proposes a Hybrid PCA–Haar Wavelet Analysis. The hybrid approach first applies PCA to describe the data and then Haar Wavelet filtering for analysis. Based on prototyping and measurement, an investigation of the Hybrid PCA–Haar Wavelet Analysis technique is performed using the Kyoto2006+ dataset. The authors consider a number of parameters and present experimental results to demonstrate the effectiveness of the hybrid approach as compared to the two algorithms individually.

scholarly journals Visualization of Anomalies using Graph-Based Anomaly Detection

2021 ◽  
Vol 34 (1) ◽  
Author(s):  
Ramesh Paudel ◽  
Lauren Tharp ◽  
Dulce Kaiser ◽  
William Eberle ◽  
Gerald Gannod
Keyword(s):  
Anomaly Detection ◽  
Network Traffic ◽  
False Positives ◽  
Visual Context ◽  
Traffic Flows ◽  
Security Analysts ◽  
Fast Analysis ◽  
Detection Systems ◽  
Potential Damage ◽  
Network Anomaly Detection

Network protocol analyzers such asWireshark are valuable for analyzing network traffic but pose a challenge in that it can be difficult to determine which behaviors are out of the ordinary due to the volume of data that must be analyzed. Network anomaly detection systems can provide vital insights to security analysts to supplement protocol analyzers, but this feedback can be difficult to interpret due to the complexity of the algorithms used and the lack of context to determine the reasoning for which an event was labeled as anomalous. We present an approach for visualizing anomalies using a graph-based anomaly detection methodology that aims to provide visual context to network traffic. We demonstrate the approach using network traffic flows as an approach for aiding in the investigation and triage of anomalous network events. The simplicity of a visual representation supports fast analysis of anomalous traffic to identify true positives from false positives and prevent further potential damage.

Anomaly Detection in Wireless Networks

2016 ◽  
pp. 108-118
Author(s):  
Yirui Hu
Keyword(s):  
Wireless Networks ◽  
Anomaly Detection ◽  
Initial Step ◽  
Training Data ◽  
Data Set ◽  
Detection Systems ◽  
Detection Analysis ◽  
Anomaly Score ◽  
Unsupervised Algorithms ◽  
Traffic Behavior

This chapter is an introduction to multi-cluster based anomaly detection analysis. Various anomalies present different behaviors in wireless networks. Not all anomalies are known to networks. Unsupervised algorithms are desirable to automatically characterize the nature of traffic behavior and detect anomalies from normal behaviors. Essentially all anomaly detection systems first learn a model of the normal patterns in training data set, and then determine the anomaly score of a given testing data point based on the deviations from the learned patterns. The initial step of learning a good model is the most crucial part in anomaly detection. Multi-cluster based analysis are valuable because they can obtain the insights of human behaviors and learn similar patterns in temporal traffic data. The anomaly threshold can be determined by quantitative analysis based on the trained model. A novel quantitative “Donut” algorithm of anomaly detection on the basis of model log-likelihood is proposed in this chapter.

Accuracy improving guidelines for network anomaly detection systems

2009 ◽  
Vol 7 (1) ◽  
pp. 63-81 ◽  
Author(s):  
Ayesha Binte Ashfaq ◽  
Muhammad Qasim Ali ◽  
Syed Ali Khayam
Keyword(s):  
Anomaly Detection ◽  
Detection Systems ◽  
Network Anomaly Detection

A Hybrid Technique Using PCA and Wavelets in Network Traffic Anomaly Detection

2015 ◽  
pp. 758-786
Author(s):  
Stevan Novakov ◽  
Chung-Horng Lung ◽  
Ioannis Lambadaris ◽  
Nabil Seddigh
Keyword(s):  
Spectral Analysis ◽  
Statistical Analysis ◽  
Wavelet Analysis ◽  
Anomaly Detection ◽  
Network Traffic ◽  
Hybrid Approach ◽  
Haar Wavelet ◽  
Wavelet Filtering ◽  
Analysis Technique ◽  
Network Anomaly Detection

Research into network anomaly detection has become crucial as a result of a significant increase in the number of computer attacks. Many approaches in network anomaly detection have been reported in the literature, but data or solutions typically are not freely available. Recently, a labeled network traffic flow dataset, Kyoto2006+, has been created and is publicly available. Most existing approaches using Kyoto2006+ for network anomaly detection apply various clustering techniques. This paper leverages existing well known statistical analysis and spectral analysis techniques for network anomaly detection. The first popular approach is a statistical analysis technique called Principal Component Analysis (PCA). PCA describes data in a new dimension to unlock otherwise hidden characteristics. The other well known spectral analysis technique is Haar Wavelet filtering analysis. It measures the amount and magnitude of abrupt changes in data. Both approaches have strengths and limitations. In response, this paper proposes a Hybrid PCA–Haar Wavelet Analysis. The hybrid approach first applies PCA to describe the data and then Haar Wavelet filtering for analysis. Based on prototyping and measurement, an investigation of the Hybrid PCA–Haar Wavelet Analysis technique is performed using the Kyoto2006+ dataset. The authors consider a number of parameters and present experimental results to demonstrate the effectiveness of the hybrid approach as compared to the two algorithms individually.

Sign in / Sign up

Export Citation Format

Share Document