A comparative study using supervised learning for anomaly detection in network traffic

A user connects to hundreds of remote networks daily, some of which can be corrupted by malicious sources. To overcome this problem, a variety of Network Intrusion Detection systems are built, which aim to detect harmful networks before they establish a connection with the user’s local system. This paper focuses on proposing a model for Anomaly based Network Intrusion Detection systems (NIDS), by performing comparisons of various Supervised Learning Algorithms on metric of their accuracy. Two datasets were used and analysed, each having different properties in terms of the volume of data they contain and their use cases. Feature engineering was done to retrieve the most optimum features of both the datasets and only the top 25% best features were used to build the models – a smaller subset of features not only aids in decreasing the capital required to collect the data but also gets rid of redundant and noisy information. Two different splicing methods were used to train the data and each method showed different trends on the ML models.

SDN-Based Network Intrusion Detection as DDoS defense system for Virtualization Environment

Nowadays, DDoS attacks are often aimed at cloud computing environments, as more people use virtualization servers. With so many Nodes and distributed services, it will be challenging to rely solely on conventional networks to control and monitor intrusions. We design and deploy DDoS attack defense systems in virtualization environments based on Software-defined Networking (SDN) by combining signature-based Network Intrusion Detection Systems (NIDS) and sampled flow (sFlow). These techniques are practically tested and evaluated on the Proxmox production Virtualization Environment testbed, adding High Availability capabilities to the Controller. The evaluation results show that it promptly detects several types of DDoS attacks and mitigates their negative impact on network performance. Moreover, it also shows good results on Quality of Service (QoS) parameters such as average packet loss about 0 %, average latency about 0.8 ms, and average bitrate about 860 Mbit/s.

Enhanced Network Intrusion Detection System

A reasonably good network intrusion detection system generally requires a high detection rate and a low false alarm rate in order to predict anomalies more accurately. Older datasets cannot capture the schema of a set of modern attacks; therefore, modelling based on these datasets lacked sufficient generalizability. This paper operates on the UNSW-NB15 Dataset, which is currently one of the best representatives of modern attacks and suggests various models. We discuss various models and conclude our discussion with the model that performs the best using various kinds of evaluation metrics. Alongside modelling, a comprehensive data analysis on the features of the dataset itself using our understanding of correlation, variance, and similar factors for a wider picture is done for better modelling. Furthermore, hypothetical ponderings are discussed for potential network intrusion detection systems, including suggestions on prospective modelling and dataset generation as well.

BUILDING A DYNAMIC SCALABLE PARALLEL CLOUD-BASED SNORT NIDS USING CONTAINERS AND BIG DATA

Snort is one of the well-known signature-based network intrusion detection systems (NIDS). The Snort sensor placement must be in the same physical network. The defense center in the typical NIDS architecture cause limited network coverage to be monitored, especially for remote networks with restricted bandwidth and network policy. Moreover, the increasing number of sensor instances, followed by a rapid increase in log data volume, caused the existing system to face Big data challenges. This research paper aims to propose a novel design of cloud-based Snort NIDS using containers and implementing Big data in the defense center to overcome these problems. Our design consists of Docker as the sensor's platform, Apache Kafka as the distributed messaging system, and various big data technology orchestrated on lambda architecture. Experiments are conducted to measure sensor deployment, optimum message delivery from sensors to the defense center, and aggregation speed, and data processing performance efficiency on the defense center. In summary, we successfully developed a cloud-based Snort NIDS and found the optimum message delivery method from the sensor to the defense center. Our design also represents the first report on implementing the Big data architecture, namely lambda architecture, to the defense center as a part of a network security monitoring platform.