Feature selection by multi-objective optimisation: Application to network anomaly detection by hierarchical self-organising maps

2014 ◽  
Vol 71 ◽  
pp. 322-338 ◽  
Author(s):  
Emiro de la Hoz ◽  
Eduardo de la Hoz ◽  
Andrés Ortiz ◽  
Julio Ortega ◽  
Antonio Martínez-Álvarez
Keyword(s):  
Feature Selection ◽  
Anomaly Detection ◽  
Multi Objective ◽  
Network Anomaly Detection

scholarly journals A New Design of Custom Optimized Cnn-Lstm Assists to Detect Network Anomaly Using Categorical Data

2021 ◽  
Author(s):  
Kanmani R ◽  
A.Christy Jeba Malar ◽  
Roopa V ◽  
Ranjani D ◽  
Suganya R
Keyword(s):  
Feature Selection ◽  
Anomaly Detection ◽  
Categorical Data ◽  
Imbalanced Data ◽  
Experimental Result ◽  
Training Dataset ◽  
Detection Model ◽  
Effective Performance ◽  
Network Anomaly Detection ◽  
System Effectiveness

Abstract For traditional intrusion detection model, the system effectiveness is fully based on training dataset and feature selection. During feature selection, it needs more labour charge and trusted mainly on expert’s knowledge. Moreover, the training dataset contains more imbalanced data which in terms model tends to be biased. Here, an automatic approach is introduced to correct deficiency in the system. In this paper, the author proposes novel network anomaly detection (NID) build using categorical data. A model has to be designed with modified form of deep neural network primarily utilized for detecting anomaly within the network. Custom CNN-LSTM with Harris Hawks Optimization (named as custom optimized CNN-LSTM) is designed as a new classifier majorly used to detect the anomaly from word cloud to distinguish the data with effective performance. The experimental result shows that the proposed method achieves a promising output for network anomaly detection.

scholarly journals CBFS: A Clustering-Based Feature Selection Mechanism for Network Anomaly Detection

IEEE Access ◽  
2020 ◽  
Vol 8 ◽  
pp. 116216-116225 ◽  
Author(s):  
Jiewen Mao ◽  
Yongquan Hu ◽  
Dong Jiang ◽  
Tongquan Wei ◽  
Fuke Shen
Keyword(s):  
Feature Selection ◽  
Anomaly Detection ◽  
Selection Mechanism ◽  
Network Anomaly Detection

scholarly journals Automated Feature Selection for Anomaly Detection in Network Traffic Data

2021 ◽  
Vol 12 (3) ◽  
pp. 1-28
Author(s):  
Makiya Nakashima ◽  
Alex Sim ◽  
Youngsoo Kim ◽  
Jonghyun Kim ◽  
Jinoh Kim
Keyword(s):  
Feature Selection ◽  
Anomaly Detection ◽  
Network Traffic ◽  
Selection Process ◽  
Traffic Data ◽  
Ensemble Techniques ◽  
Building Models ◽  
Comparable Performance ◽  
Network Anomaly Detection ◽  
Feature Selection Techniques

Variable selection (also known as feature selection ) is essential to optimize the learning complexity by prioritizing features, particularly for a massive, high-dimensional dataset like network traffic data. In reality, however, it is not an easy task to effectively perform the feature selection despite the availability of the existing selection techniques. From our initial experiments, we observed that the existing selection techniques produce different sets of features even under the same condition (e.g., a static size for the resulted set). In addition, individual selection techniques perform inconsistently, sometimes showing better performance but sometimes worse than others, thereby simply relying on one of them would be risky for building models using the selected features. More critically, it is demanding to automate the selection process, since it requires laborious efforts with intensive analysis by a group of experts otherwise. In this article, we explore challenges in the automated feature selection with the application of network anomaly detection. We first present our ensemble approach that benefits from the existing feature selection techniques by incorporating them, and one of the proposed ensemble techniques based on greedy search works highly consistently showing comparable results to the existing techniques. We also address the problem of when to stop to finalize the feature elimination process and present a set of methods designed to determine the number of features for the reduced feature set. Our experimental results conducted with two recent network datasets show that the identified feature sets by the presented ensemble and stopping methods consistently yield comparable performance with a smaller number of features to conventional selection techniques.

scholarly journals Utilizing XAI Technique to Improve Autoencoder based Model for Computer Network Anomaly Detection with Shapley Additive Explanation(SHAP)

2021 ◽  
Vol 13 (6) ◽  
pp. 109-128
Author(s):  
Khushnaseeb Roshan ◽  
Aasim Zafar
Keyword(s):  
Feature Selection ◽  
Anomaly Detection ◽  
Computer Network ◽  
Feature Selection Method ◽  
Feature Selection Technique ◽  
Internal Working ◽  
Computer Network Security ◽  
Promising Area ◽  
Shapley Values ◽  
Network Anomaly Detection

Machine learning (ML) and Deep Learning (DL) methods are being adopted rapidly, especially in computer network security, such as fraud detection, network anomaly detection, intrusion detection, and much more. However, the lack of transparency of ML and DL based models is a major obstacle to their implementation and criticized due to its black-box nature, even with such tremendous results. Explainable Artificial Intelligence (XAI) is a promising area that can improve the trustworthiness of these models by giving explanations and interpreting its output. If the internal working of the ML and DL based models is understandable, then it can further help to improve its performance. The objective of this paper is to show that how XAI can be used to interpret the results of the DL model, the autoencoder in this case. And, based on the interpretation, we improved its performance for computer network anomaly detection. The kernel SHAP method, which is based on the shapley values, is used as a novel feature selection technique. This method is used to identify only those features that are actually causing the anomalous behaviour of the set of attack/anomaly instances. Later, these feature sets are used to train and validate the autoencoderbut on benign data only. Finally, the built SHAP_Model outperformed the other two models proposed based on the feature selection method. This whole experiment is conducted on the subset of the latest CICIDS2017 network dataset. The overall accuracy and AUC of SHAP_Model is 94% and 0.969, respectively.

Sign in / Sign up

Export Citation Format

Share Document