GTF: An Adaptive Network Anomaly Detection Method at the Network Edge

Network Anomaly Detection (NAD) has become the foundation for network management and security due to the rapid development and adoption of edge computing technologies. There are two main characteristics of NAD tasks: tabular input data and imbalanced classes. Tabular input data format means NAD tasks take both sparse categorical features and dense numerical features as input. In order to achieve good performance, the detection model needs to handle both types of features efficiently. Among all widely used models, Gradient Boosting Decision Tree (GBDT) and Neural Network (NN) are the two most popular ones. However, each method has its limitation: GBDT is inefficient when dealing with sparse categorical features, while NN cannot yield satisfactory performance for dense numerical features. Imbalanced classes may downgrade the classifier’s performance and cause biased results towards the majority classes, often neglected by many exiting NAD studies. Most of the existing solutions addressing imbalance suffer from poor performance, high computational consumption, or loss of vital information under such a scenario. In this paper, we propose an adaptive ensemble-based method, named GTF, which combines TabTransformer and GBDT to leverage categorical and numerical features effectively and introduces Focal Loss to mitigate the imbalance classification. Our comprehensive experiments on two public datasets demonstrate that GTF can outperform other well-known methods in both multiclass and binary cases. Our implementation also shows that GTF has limited complexity, making it be a good candidate for deployment at the network edge.

Download Full-text

Network Anomaly Detection System with Optimized DS Evidence Theory

The Scientific World JOURNAL ◽

10.1155/2014/753659 ◽

2014 ◽

Vol 2014 ◽

pp. 1-13 ◽

Cited By ~ 2

Author(s):

Yuan Liu ◽

Xiaofeng Wang ◽

Kaiyu Liu

Keyword(s):

Anomaly Detection ◽

Detection Rate ◽

Computer Network ◽

Detection System ◽

Evidence Theory ◽

Optimization Methods ◽

High Detection Rate ◽

Detection Model ◽

Network Anomaly Detection ◽

Anomaly Detection System

Network anomaly detection has been focused on by more people with the fast development of computer network. Some researchers utilized fusion method and DS evidence theory to do network anomaly detection but with low performance, and they did not consider features of network—complicated and varied. To achieve high detection rate, we present a novel network anomaly detection system with optimized Dempster-Shafer evidence theory (ODS) and regression basic probability assignment (RBPA) function. In this model, we add weights for each senor to optimize DS evidence theory according to its previous predict accuracy. And RBPA employs sensor’s regression ability to address complex network. By four kinds of experiments, we find that our novel network anomaly detection model has a better detection rate, and RBPA as well as ODS optimization methods can improve system performance significantly.

Download Full-text

HELAD: A novel network anomaly detection model based on heterogeneous ensemble learning

Computer Networks ◽

10.1016/j.comnet.2019.107049 ◽

2020 ◽

Vol 169 ◽

pp. 107049 ◽

Cited By ~ 9

Author(s):

Ying Zhong ◽

Wenqi Chen ◽

Zhiliang Wang ◽

Yifan Chen ◽

Kai Wang ◽

...

Keyword(s):

Anomaly Detection ◽

Ensemble Learning ◽

Detection Model ◽

Model Based ◽

Heterogeneous Ensemble ◽

Network Anomaly Detection

Download Full-text

A New Design of Custom Optimized Cnn-Lstm Assists to Detect Network Anomaly Using Categorical Data

10.21203/rs.3.rs-490866/v1 ◽

2021 ◽

Author(s):

Kanmani R ◽

A.Christy Jeba Malar ◽

Roopa V ◽

Ranjani D ◽

Suganya R

Keyword(s):

Feature Selection ◽

Anomaly Detection ◽

Categorical Data ◽

Imbalanced Data ◽

Experimental Result ◽

Training Dataset ◽

Detection Model ◽

Effective Performance ◽

Network Anomaly Detection ◽

System Effectiveness

Abstract For traditional intrusion detection model, the system effectiveness is fully based on training dataset and feature selection. During feature selection, it needs more labour charge and trusted mainly on expert’s knowledge. Moreover, the training dataset contains more imbalanced data which in terms model tends to be biased. Here, an automatic approach is introduced to correct deficiency in the system. In this paper, the author proposes novel network anomaly detection (NID) build using categorical data. A model has to be designed with modified form of deep neural network primarily utilized for detecting anomaly within the network. Custom CNN-LSTM with Harris Hawks Optimization (named as custom optimized CNN-LSTM) is designed as a new classifier majorly used to detect the anomaly from word cloud to distinguish the data with effective performance. The experimental result shows that the proposed method achieves a promising output for network anomaly detection.

Download Full-text

Network Anomaly Detection by Using a Time-Decay Closed Frequent Pattern

Information ◽

10.3390/info10080262 ◽

2019 ◽

Vol 10 (8) ◽

pp. 262

Author(s):

Ying Zhao ◽

Junjun Chen ◽

Di Wu ◽

Jian Teng ◽

Nabin Sharma ◽

...

Keyword(s):

Anomaly Detection ◽

Network Traffic ◽

User Behavior ◽

Frequent Pattern ◽

Detection Methods ◽

Frequent Patterns ◽

Time Decay ◽

Network Behavior ◽

Detection Model ◽

Network Anomaly Detection

Anomaly detection of network traffic flows is a non-trivial problem in the field of network security due to the complexity of network traffic. However, most machine learning-based detection methods focus on network anomaly detection but ignore the user anomaly behavior detection. In real scenarios, the anomaly network behavior may harm the user interests. In this paper, we propose an anomaly detection model based on time-decay closed frequent patterns to address this problem. The model mines closed frequent patterns from the network traffic of each user and uses a time-decay factor to distinguish the weight of current and historical network traffic. Because of the dynamic nature of user network behavior, a detection model update strategy is provided in the anomaly detection framework. Additionally, the closed frequent patterns can provide interpretable explanations for anomalies. Experimental results show that the proposed method can detect user behavior anomaly, and the network anomaly detection performance achieved by the proposed method is similar to the state-of-the-art methods and significantly better than the baseline methods.

Download Full-text

Trust-based federated learning for network anomaly detection

Web Intelligence ◽

10.3233/web-210475 ◽

2021 ◽

pp. 1-11

Author(s):

Naiyue Chen ◽

Yi Jin ◽

Yinglong Li ◽

Luxin Cai

Keyword(s):

Machine Learning ◽

Anomaly Detection ◽

Private Information ◽

Information Disclosure ◽

Rapid Development ◽

Detection Algorithm ◽

Abnormal Behavior ◽

Local Data ◽

Daily Work ◽

Network Anomaly Detection

With the rapid development of social networks and the massive popularity of intelligent mobile terminals, network anomaly detection is becoming increasingly important. In daily work and life, edge nodes store a large number of network local connection data and audit data, which can be used to analyze network abnormal behavior. With the increasingly close network communication, the amount of network connection and other related data collected by each network terminal is increasing. Machine learning has become a classification method to analyze the features of big data in the network. Face to the problems of excessive data and long response time for network anomaly detection, we propose a trust-based Federated learning anomaly detection algorithm. We use the edge nodes to train the local data model, and upload the machine learning parameters to the central node. Meanwhile, according to the performance of edge nodes training, we set different weights to match the processing capacity of each terminal which will obtain faster convergence speed and better attack classification accuracy. The user’s private information will only be processed locally and will not be uploaded to the central server, which can reduce the risk of information disclosure. Finally, we compare the basic federated learning model and TFCNN algorithm on KDD Cup 99 dataset and MNIST dataset. The experimental results show that the TFCNN algorithm can improve accuracy and communication efficiency.

Download Full-text