A Hybrid Technique Using PCA and Wavelets in Network Traffic Anomaly Detection

2015 ◽  
pp. 758-786
Author(s):  
Stevan Novakov ◽  
Chung-Horng Lung ◽  
Ioannis Lambadaris ◽  
Nabil Seddigh
Keyword(s):  
Spectral Analysis ◽  
Statistical Analysis ◽  
Wavelet Analysis ◽  
Anomaly Detection ◽  
Network Traffic ◽  
Hybrid Approach ◽  
Haar Wavelet ◽  
Wavelet Filtering ◽  
Analysis Technique ◽  
Network Anomaly Detection

Research into network anomaly detection has become crucial as a result of a significant increase in the number of computer attacks. Many approaches in network anomaly detection have been reported in the literature, but data or solutions typically are not freely available. Recently, a labeled network traffic flow dataset, Kyoto2006+, has been created and is publicly available. Most existing approaches using Kyoto2006+ for network anomaly detection apply various clustering techniques. This paper leverages existing well known statistical analysis and spectral analysis techniques for network anomaly detection. The first popular approach is a statistical analysis technique called Principal Component Analysis (PCA). PCA describes data in a new dimension to unlock otherwise hidden characteristics. The other well known spectral analysis technique is Haar Wavelet filtering analysis. It measures the amount and magnitude of abrupt changes in data. Both approaches have strengths and limitations. In response, this paper proposes a Hybrid PCA–Haar Wavelet Analysis. The hybrid approach first applies PCA to describe the data and then Haar Wavelet filtering for analysis. Based on prototyping and measurement, an investigation of the Hybrid PCA–Haar Wavelet Analysis technique is performed using the Kyoto2006+ dataset. The authors consider a number of parameters and present experimental results to demonstrate the effectiveness of the hybrid approach as compared to the two algorithms individually.

A Hybrid Technique Using PCA and Wavelets in Network Traffic Anomaly Detection

2014 ◽  
Vol 6 (1) ◽  
pp. 17-53 ◽  
Author(s):  
Stevan Novakov ◽  
Chung-Horng Lung ◽  
Ioannis Lambadaris ◽  
Nabil Seddigh
Keyword(s):  
Spectral Analysis ◽  
Statistical Analysis ◽  
Wavelet Analysis ◽  
Anomaly Detection ◽  
Network Traffic ◽  
Hybrid Approach ◽  
Haar Wavelet ◽  
Wavelet Filtering ◽  
Analysis Technique ◽  
Network Anomaly Detection

Research into network anomaly detection has become crucial as a result of a significant increase in the number of computer attacks. Many approaches in network anomaly detection have been reported in the literature, but data or solutions typically are not freely available. Recently, a labeled network traffic flow dataset, Kyoto2006+, has been created and is publicly available. Most existing approaches using Kyoto2006+ for network anomaly detection apply various clustering techniques. This paper leverages existing well known statistical analysis and spectral analysis techniques for network anomaly detection. The first popular approach is a statistical analysis technique called Principal Component Analysis (PCA). PCA describes data in a new dimension to unlock otherwise hidden characteristics. The other well known spectral analysis technique is Haar Wavelet filtering analysis. It measures the amount and magnitude of abrupt changes in data. Both approaches have strengths and limitations. In response, this paper proposes a Hybrid PCA–Haar Wavelet Analysis. The hybrid approach first applies PCA to describe the data and then Haar Wavelet filtering for analysis. Based on prototyping and measurement, an investigation of the Hybrid PCA–Haar Wavelet Analysis technique is performed using the Kyoto2006+ dataset. The authors consider a number of parameters and present experimental results to demonstrate the effectiveness of the hybrid approach as compared to the two algorithms individually.

scholarly journals Learning Representations of Network Traffic Using Deep Neural Networks for Network Anomaly Detection: A Perspective towards Oil and Gas IT Infrastructures

Symmetry ◽  
2020 ◽  
Vol 12 (11) ◽  
pp. 1882
Author(s):  
Sheraz Naseer ◽  
Rao Faizan Ali ◽  
P.D.D Dominic ◽  
Yasir Saleem
Keyword(s):  
Anomaly Detection ◽  
Network Traffic ◽  
Oil And Gas ◽  
Resource Planning ◽  
Machine Learning Algorithms ◽  
Data Driven ◽  
Network Data ◽  
It Infrastructure ◽  
Industrial Automation ◽  
Network Anomaly Detection

Oil and Gas organizations are dependent on their IT infrastructure, which is a small part of their industrial automation infrastructure, to function effectively. The oil and gas (O&G) organizations industrial automation infrastructure landscape is complex. To perform focused and effective studies, Industrial systems infrastructure is divided into functional levels by The Instrumentation, Systems and Automation Society (ISA) Standard ANSI/ISA-95:2005. This research focuses on the ISA-95:2005 level-4 IT infrastructure to address network anomaly detection problem for ensuring the security and reliability of Oil and Gas resource planning, process planning and operations management. Anomaly detectors try to recognize patterns of anomalous behaviors from network traffic and their performance is heavily dependent on extraction time and quality of network traffic features or representations used to train the detector. Creating efficient representations from large volumes of network traffic to develop anomaly detection models is a time and resource intensive task. In this study we propose, implement and evaluate use of Deep learning to learn effective Network data representations from raw network traffic to develop data driven anomaly detection systems. Proposed methodology provides an automated and cost effective replacement of feature extraction which is otherwise a time and resource intensive task for developing data driven anomaly detectors. The ISCX-2012 dataset is used to represent ISA-95 level-4 network traffic because the O&G network traffic at this level is not much different than normal internet traffic. We trained four representation learning models using popular deep neural network architectures to extract deep representations from ISCX 2012 traffic flows. A total of sixty anomaly detectors were trained by authors using twelve conventional Machine Learning algorithms to compare the performance of aforementioned deep representations with that of a human-engineered handcrafted network data representation. The comparisons were performed using well known model evaluation parameters. Results showed that deep representations are a promising feature in engineering replacement to develop anomaly detection models for IT infrastructure security. In our future research, we intend to investigate the effectiveness of deep representations, extracted using ISA-95:2005 Level 2-3 traffic comprising of SCADA systems, for anomaly detection in critical O&G systems.

Network anomaly detection based on signal processing techniques

2013 ◽  
Vol 18 (1) ◽  
pp. 15-21
Author(s):  
Tomasz Andrysiak ◽  
Łukasz Saganowski ◽  
Mirosław Maszewski
Keyword(s):  
Signal Processing ◽  
Anomaly Detection ◽  
Network Traffic ◽  
Detection Method ◽  
Matching Pursuit ◽  
Matching Pursuit Decomposition ◽  
Network Anomaly Detection ◽  
Processing Techniques ◽  
Signal Processing Techniques

Abstract The article depicts possibility of using Matching Pursuit decomposition in order to recognize unspecified hazards in network traffic. Furthermore, the work aims to present feasible enhancements to the anomaly detection method, as well as their efficiency on the basis of a wide collection of pattern test traces.

scholarly journals Visualization of Anomalies using Graph-Based Anomaly Detection

2021 ◽  
Vol 34 (1) ◽  
Author(s):  
Ramesh Paudel ◽  
Lauren Tharp ◽  
Dulce Kaiser ◽  
William Eberle ◽  
Gerald Gannod
Keyword(s):  
Anomaly Detection ◽  
Network Traffic ◽  
False Positives ◽  
Visual Context ◽  
Traffic Flows ◽  
Security Analysts ◽  
Fast Analysis ◽  
Detection Systems ◽  
Potential Damage ◽  
Network Anomaly Detection

Network protocol analyzers such asWireshark are valuable for analyzing network traffic but pose a challenge in that it can be difficult to determine which behaviors are out of the ordinary due to the volume of data that must be analyzed. Network anomaly detection systems can provide vital insights to security analysts to supplement protocol analyzers, but this feedback can be difficult to interpret due to the complexity of the algorithms used and the lack of context to determine the reasoning for which an event was labeled as anomalous. We present an approach for visualizing anomalies using a graph-based anomaly detection methodology that aims to provide visual context to network traffic. We demonstrate the approach using network traffic flows as an approach for aiding in the investigation and triage of anomalous network events. The simplicity of a visual representation supports fast analysis of anomalous traffic to identify true positives from false positives and prevent further potential damage.

scholarly journals Network Anomaly Detection by Using a Time-Decay Closed Frequent Pattern

Information ◽  
2019 ◽  
Vol 10 (8) ◽  
pp. 262
Author(s):  
Ying Zhao ◽  
Junjun Chen ◽  
Di Wu ◽  
Jian Teng ◽  
Nabin Sharma ◽  
...  
Keyword(s):  
Anomaly Detection ◽  
Network Traffic ◽  
User Behavior ◽  
Frequent Pattern ◽  
Detection Methods ◽  
Frequent Patterns ◽  
Time Decay ◽  
Network Behavior ◽  
Detection Model ◽  
Network Anomaly Detection

Anomaly detection of network traffic flows is a non-trivial problem in the field of network security due to the complexity of network traffic. However, most machine learning-based detection methods focus on network anomaly detection but ignore the user anomaly behavior detection. In real scenarios, the anomaly network behavior may harm the user interests. In this paper, we propose an anomaly detection model based on time-decay closed frequent patterns to address this problem. The model mines closed frequent patterns from the network traffic of each user and uses a time-decay factor to distinguish the weight of current and historical network traffic. Because of the dynamic nature of user network behavior, a detection model update strategy is provided in the anomaly detection framework. Additionally, the closed frequent patterns can provide interpretable explanations for anomalies. Experimental results show that the proposed method can detect user behavior anomaly, and the network anomaly detection performance achieved by the proposed method is similar to the state-of-the-art methods and significantly better than the baseline methods.

scholarly journals WAVE-CUSUM: Improving CUSUM performance in network anomaly detection by means of wavelet analysis

2012 ◽  
Vol 31 (5) ◽  
pp. 727-735 ◽  
Author(s):  
C. Callegari ◽  
S. Giordano ◽  
M. Pagano ◽  
T. Pepe
Keyword(s):  
Wavelet Analysis ◽  
Anomaly Detection ◽  
Network Anomaly Detection
Sign in / Sign up

Export Citation Format

Share Document