A Survey on Data-driven Network Intrusion Detection

2022 ◽  
Vol 54 (9) ◽  
pp. 1-36
Author(s):  
Dylan Chou ◽  
Meng Jiang
Keyword(s):  
Machine Learning ◽  
Intrusion Detection ◽  
Real World ◽  
Data Driven ◽  
Network Intrusion Detection ◽  
Large Network ◽  
Learning Models ◽  
Simulated Environments ◽  
Network Intrusion ◽  
Machine Learning Models

Data-driven network intrusion detection (NID) has a tendency towards minority attack classes compared to normal traffic. Many datasets are collected in simulated environments rather than real-world networks. These challenges undermine the performance of intrusion detection machine learning models by fitting machine learning models to unrepresentative “sandbox” datasets. This survey presents a taxonomy with eight main challenges and explores common datasets from 1999 to 2020. Trends are analyzed on the challenges in the past decade and future directions are proposed on expanding NID into cloud-based environments, devising scalable models for large network data, and creating labeled datasets collected in real-world networks.

scholarly journals Assessment of Machine Learning Algorithms for Network Intrusion Detection

2020 ◽  
Vol 9 (4) ◽  
pp. 1667-1671
Keyword(s):  
Machine Learning ◽  
Intrusion Detection ◽  
Performance Metrics ◽  
Detection System ◽  
Machine Learning Algorithms ◽  
Network Intrusion Detection ◽  
Learning Models ◽  
Network Intrusion ◽  
Tree Classifier ◽  
Machine Learning Models

A Network Intrusion Detection System (NIDS) is a framework to identify network interruptions as well as abuse by checking network traffic movement and classifying it as either typical or strange. Numerous Intrusion Detection Systems have been implemented using simulated datasets like KDD’99 intrusion dataset but none of them uses a real time dataset. The proposed work performs and assesses tests to overview distinctive machine learning models reliant on KDD’99 intrusion dataset and an ongoing created dataset. The machine learning models achieved to compute required performance metrics so as to assess the chosen classifiers. The emphasis was on the accuracy metric so as to improve the recognition pace of the interruption identification framework. The actualized calculations showed that the decision tree classifier accomplished the most noteworthy estimation of accuracy while the logistic regression classifier has accomplished the least estimation of exactness for both of the datasets utilized.

scholarly journals How to Effectively Collect and Process Network Data for Intrusion Detection?

Entropy ◽  
2021 ◽  
Vol 23 (11) ◽  
pp. 1532
Author(s):  
Mikołaj Komisarek ◽  
Marek Pawlicki ◽  
Rafał Kozik ◽  
Witold Hołubowicz ◽  
Michał Choraś
Keyword(s):  
Machine Learning ◽  
Intrusion Detection ◽  
Real World ◽  
Machine Learning Algorithms ◽  
Network Intrusion Detection ◽  
Minimal Amount ◽  
Security Breaches ◽  
Network Intrusion ◽  
Benchmark Datasets ◽  
In The Wild

The number of security breaches in the cyberspace is on the rise. This threat is met with intensive work in the intrusion detection research community. To keep the defensive mechanisms up to date and relevant, realistic network traffic datasets are needed. The use of flow-based data for machine-learning-based network intrusion detection is a promising direction for intrusion detection systems. However, many contemporary benchmark datasets do not contain features that are usable in the wild. The main contribution of this work is to cover the research gap related to identifying and investigating valuable features in the NetFlow schema that allow for effective, machine-learning-based network intrusion detection in the real world. To achieve this goal, several feature selection techniques have been applied on five flow-based network intrusion detection datasets, establishing an informative flow-based feature set. The authors’ experience with the deployment of this kind of system shows that to close the research-to-market gap, and to perform actual real-world application of machine-learning-based intrusion detection, a set of labeled data from the end-user has to be collected. This research aims at establishing the appropriate, minimal amount of data that is sufficient to effectively train machine learning algorithms in intrusion detection. The results show that a set of 10 features and a small amount of data is enough for the final model to perform very well.

scholarly journals Adversarial Attack on Machine Learning Models

2019 ◽  
Vol 8 (6S4) ◽  
pp. 431-434
Keyword(s):  
Machine Learning ◽  
Statistical Tests ◽  
Original Data ◽  
Network Intrusion Detection ◽  
Learning Models ◽  
Malware Classification ◽  
Network Intrusion ◽  
Adversarial Examples ◽  
Adversarial Attack ◽  
Machine Learning Models

Machine Learning (ML) models are applied in a variety of tasks such as network intrusion detection or malware classification. Yet, these models are vulnerable to a class of malicious inputs known as adversarial examples. These are slightly perturbed inputs that are classified incorrectly by the ML model. The mitigation of these adversarial inputs remains an open problem. As a step towards understanding adversarial examples, we show that they are not drawn from the same distribution than the original data, and can thus be detected using statistical tests. Using this knowledge, we introduce a complimentary approach to identify specific inputs that are adversarial. Specifically, we augment our ML model with an additional output, in which the model is trained to classify all adversarial inputs.

scholarly journals Intrusion detection by machine learning = Behatolás detektálás gépi tanulás által

2020 ◽  
Author(s):  
◽  
Csaba Brunner
Keyword(s):  
Machine Learning ◽  
Information Technology ◽  
Intrusion Detection ◽  
Process Model ◽  
Detection Performance ◽  
Learning Models ◽  
Detection Techniques ◽  
Network Intrusion ◽  
Hybrid Detection ◽  
Machine Learning Models

Since the early days of information technology, there have been many stakeholders who used the technological capabilities for their own benefit, be it legal operations, or illegal access to computational assets and sensitive information. Every year, businesses invest large amounts of effort into upgrading their IT infrastructure, yet, even today, they are unprepared to protect their most valuable assets: data and knowledge. This lack of protection was the main reason for the creation of this dissertation. During this study, intrusion detection, a field of information security, is evaluated through the use of several machine learning models performing signature and hybrid detection. This is a challenging field, mainly due to the high velocity and imbalanced nature of network traffic. To construct machine learning models capable of intrusion detection, the applied methodologies were the CRISP-DM process model designed to help data scientists with the planning, creation and integration of machine learning models into a business information infrastructure, and design science research interested in answering research questions with information technology artefacts. The two methodologies have a lot in common, which is further elaborated in the study. The goals of this dissertation were two-fold: first, to create an intrusion detector that could provide a high level of intrusion detection performance measured using accuracy and recall and second, to identify potential techniques that can increase intrusion detection performance. Out of the designed models, a hybrid autoencoder + stacking neural network model managed to achieve detection performance comparable to the best models that appeared in the related literature, with good detections on minority classes. To achieve this result, the techniques identified were synthetic sampling, advanced hyperparameter optimization, model ensembles and autoencoder networks. In addition, the dissertation set up a soft hierarchy among the different detection techniques in terms of performance and provides a brief outlook on potential future practical applications of network intrusion detection models as well.

Sign in / Sign up

Export Citation Format

Share Document