Dimension Reduction and its Effects on Clustering for Intrusion Detection

2011 ◽  
pp. 169-192
Author(s):  
Peyman Kabiri ◽  
Ali Ghorbani
Keyword(s):  
Intrusion Detection ◽  
Dimension Reduction ◽  
Clustering Algorithms ◽  
Feature Space ◽  
Information Value ◽  
Network Intrusion Detection ◽  
The Past ◽  
Network Intrusion ◽  
Network Intrusions ◽  
Network Feature

With recent advances in network based technology and the increased dependency of our every day life on this technology, assuring reliable operation of network based systems is very important. During recent years, a number of attacks on networks have dramatically increased and consequently interest in network intrusion detection has increased among the researchers. During the past few years, different approaches for collecting a dataset of network features, each with its own assumptions, have been proposed to detect network intrusions. Recently, many research works have been focused on better understanding of the network feature space so that they can come up with a better detection method. The curse of dimensionality is still a big obstacle in front of the researchers in network intrusion detection. In this chapter, DARPA’99 dataset is used for the study. Features in that dataset are analyzed with respect to their information value. Using the information value of the features, the number of dimensions in the data is reduced. Later on, using several clustering algorithms, effects of the dimension reduction on the dataset are studied and the results are reported.

Network Intrusion Detection Method Based on RS-LSSVM

2014 ◽  
Vol 602-605 ◽  
pp. 1634-1637
Author(s):  
Fang Nian Wang ◽  
Shen Shen Wang ◽  
Wan Fang Che ◽  
Yun Bai
Keyword(s):  
Intrusion Detection ◽  
Detection Method ◽  
Attribute Reduction ◽  
Feature Space ◽  
Least Square ◽  
Classification Model ◽  
Network Intrusion Detection ◽  
Support Vector ◽  
Network Intrusion ◽  
Sample Data

An intrusion detection method based on RS-LSSVM is studied in this paper. Firstly, attribute reduction algorithm based on the generalized decision table is proposed to remove the interference features and reduce the dimension of input feature space. Then the classification method based on least square support vector machine (LSSVM) is analyzed. The sample data after dimension reduction is used for LSSVM training, and the LSSVM classification model is obtained, which forms the ability of detecting unknown intrusion. Simulation results show that the proposed method can effectively remove the unnecessary features and improve the performance of network intrusion detection.

Building Recommender Systems for Network Intrusion Detection Using Intelligent Decision Technologies

2013 ◽  
pp. 49-62 ◽  
Author(s):  
Mrutyunjaya Panda ◽  
Manas Ranjan Patra ◽  
Sachidananda Dehuri
Keyword(s):  
Intrusion Detection ◽  
Recommender Systems ◽  
Machine Learning Algorithms ◽  
Network Intrusion Detection ◽  
Current Generation ◽  
Network Intrusion ◽  
Intelligent Decision ◽  
Wide Range ◽  
Network Intrusions ◽  
Decision Technologies

This chapter presents an overview of the field of recommender systems and describes the current generation of recommendation methods with their limitations and possible extensions that can improve the capabilities of the recommendations made suitable for a wide range of applications. In recent years, machine learning algorithms have been considered to be an important part of the recommendation process to take intelligent decisions. The chapter will explore the application of such techniques in the field of network intrusion detection in order to examine the vulnerabilities of different recommendation techniques. Finally, the authors outline some of the major issues in building secure recommendation systems in identifying possible network intrusions.

CLUSTERING-BASED NETWORK INTRUSION DETECTION

2007 ◽  
Vol 14 (02) ◽  
pp. 169-187 ◽  
Author(s):  
SHI ZHONG ◽  
TAGHI M. KHOSHGOFTAAR ◽  
NAEEM SELIYA
Keyword(s):  
Data Mining ◽  
Network Security ◽  
Intrusion Detection ◽  
Unsupervised Learning ◽  
Network Traffic ◽  
Clustering Algorithms ◽  
Network Intrusion Detection ◽  
Learning Methods ◽  
High Detection Rate ◽  
Network Intrusion

Recently data mining methods have gained importance in addressing network security issues, including network intrusion detection — a challenging task in network security. Intrusion detection systems aim to identify attacks with a high detection rate and a low false alarm rate. Classification-based data mining models for intrusion detection are often ineffective in dealing with dynamic changes in intrusion patterns and characteristics. Consequently, unsupervised learning methods have been given a closer look for network intrusion detection. We investigate multiple centroid-based unsupervised clustering algorithms for intrusion detection, and propose a simple yet effective self-labeling heuristic for detecting attack and normal clusters of network traffic audit data. The clustering algorithms investigated include, k-means, Mixture-Of-Spherical Gaussians, Self-Organizing Map, and Neural-Gas. The network traffic datasets provided by the DARPA 1998 offline intrusion detection project are used in our empirical investigation, which demonstrates the feasibility and promise of unsupervised learning methods for network intrusion detection. In addition, a comparative analysis shows the advantage of clustering-based methods over supervised classification techniques in identifying new or unseen attack types.

scholarly journals Network Intrusion Detection with a Hashing Based Apriori Algorithm Using Hadoop MapReduce

Computers ◽  
2019 ◽  
Vol 8 (4) ◽  
pp. 86 ◽  
Author(s):  
Azeez ◽  
Ayemobola ◽  
Misra ◽  
Maskeliūnas ◽  
Damaševičius
Keyword(s):  
Intrusion Detection ◽  
Effective Means ◽  
Network Intrusion Detection ◽  
Apriori Algorithm ◽  
Operational Mode ◽  
Mapreduce Framework ◽  
Hadoop Mapreduce ◽  
Network Intrusion ◽  
Internet Users ◽  
Network Intrusions

Ubiquitous nature of Internet services across the globe has undoubtedly expanded the strategies and operational mode being used by cybercriminals to perpetrate their unlawful activities through intrusion on various networks. Network intrusion has led to many global financial loses and privacy problems for Internet users across the globe. In order to safeguard the network and to prevent Internet users from being the regular victims of cyber-criminal activities, new solutions are needed. This research proposes solution for intrusion detection by using the improved hashing-based Apriori algorithm implemented on Hadoop MapReduce framework; capable of using association rules in mining algorithm for identifying and detecting network intrusions. We used the KDD dataset to evaluate the effectiveness and reliability of the solution. Our results obtained show that this approach provides a reliable and effective means of detecting network intrusion.

scholarly journals Network intrusion detection using machine learning

2021 ◽  
Author(s):  
Seyed Pedrum Jalali Mosallam
Keyword(s):  
Machine Learning ◽  
Intrusion Detection ◽  
Network Traffic ◽  
Machine Learning Techniques ◽  
Network Intrusion Detection ◽  
Network Intrusion ◽  
Learning Techniques ◽  
Network Intrusions ◽  
Regular Network

In this research we have studied the use of machine learning techniques in detecting network intrusions. Most research in the field has used the very outdated dataset (KDDCup99) which consists of a set handcrafted features. In our research we present models that work well on both the older dataset and on newer datasets such as ISCX2014 and ISCX2012. We also present methods for extracting features from these datasets. Another issue we found with most research in this field is that they do not study the effect of surges in regular network traffic and how that might affect the model. We put our model to test in 10x traffic and show its effectiveness under these conditions. We also study how semi-supervised models can be used in training NIDS models without directly showing them labeled data.

scholarly journals A K-means algorithm based on characteristics of density applied to network intrusion detection

2020 ◽  
Vol 17 (2) ◽  
pp. 665-687
Author(s):  
Jing Xu ◽  
Dezhi Han ◽  
Kuan-Ching Li ◽  
Hai Jiang
Keyword(s):  
Intrusion Detection ◽  
Nearest Neighbor ◽  
Clustering Algorithms ◽  
Search Space ◽  
Nearest Neighbor Search ◽  
Network Intrusion Detection ◽  
Neighbor Search ◽  
Network Intrusion ◽  
Initial Seed ◽  
Unsupervised Algorithms

K-means algorithms are a group of popular unsupervised algorithms widely used for cluster analysis. However, the results of traditional K-means clustering algorithms are greatly affected by the initial clustering center, with unstable accuracy and low speed, which makes the algorithm hard to meet the requirements for Big Data. In this paper, a modernized version of the K-means algorithm based on density to select the initial seed of clustering is proposed. Firstly, Kd-tree is used to divide the hyper-rectangle space, so those points close to each other are grouped into the same sub-tree during data pre-processing, and the generalized information is stored in the tree structure. Besides, an improved Kd-tree nearest neighbor search is used in the K-means algorithm to prune the search space and optimize the operation for speedup. The clustering results show that the clusters are stable and accurate when the numbers of clusters and iterations are constant. Experimental results in the network intrusion detection case show that the improved version of the K-means algorithms performs better in terms of detection rate and false rate.

scholarly journals Network intrusion detection using machine learning

2021 ◽  
Author(s):  
Seyed Pedrum Jalali Mosallam
Keyword(s):  
Machine Learning ◽  
Intrusion Detection ◽  
Network Traffic ◽  
Machine Learning Techniques ◽  
Network Intrusion Detection ◽  
Network Intrusion ◽  
Learning Techniques ◽  
Network Intrusions ◽  
Regular Network

In this research we have studied the use of machine learning techniques in detecting network intrusions. Most research in the field has used the very outdated dataset (KDDCup99) which consists of a set handcrafted features. In our research we present models that work well on both the older dataset and on newer datasets such as ISCX2014 and ISCX2012. We also present methods for extracting features from these datasets. Another issue we found with most research in this field is that they do not study the effect of surges in regular network traffic and how that might affect the model. We put our model to test in 10x traffic and show its effectiveness under these conditions. We also study how semi-supervised models can be used in training NIDS models without directly showing them labeled data.

scholarly journals Network Intrusion Detection with Threat Agent Profiling

2018 ◽  
Vol 2018 ◽  
pp. 1-17 ◽  
Author(s):  
Tomáš Bajtoš ◽  
Andrej Gajdoš ◽  
Lenka Kleinová ◽  
Katarína Lučivjanská ◽  
Pavol Sokol
Keyword(s):  
Network Security ◽  
Intrusion Detection ◽  
Network Traffic ◽  
Clustering Algorithms ◽  
Computer Systems ◽  
Network Intrusion Detection ◽  
Clustering Methods ◽  
Security Incident ◽  
Network Intrusion ◽  
Fine Classification

With the increase in usage of computer systems and computer networks, the problem of intrusion detection in network security has become an important issue. In this paper, we discuss approaches that simplify network administrator’s work. We applied clustering methods for security incident profiling. We considerK-means, PAM, and CLARA clustering algorithms. For this purpose, we used data collected in Warden system from various security tools. We do not aim to differentiate between normal and abnormal network traffic, but we focus on grouping similar threat agents based on attributes of security events. We suggest a case of a fine classification and a case of a coarse classification and discuss advantages of both cases.

scholarly journals High-Performance Feature Selection Model for Network Intrusion Detection System

2019 ◽  
Vol 8 (6S3) ◽  
pp. 1595-1597
Keyword(s):  
Feature Selection ◽  
Intrusion Detection ◽  
Intrusion Detection System ◽  
Detection System ◽  
Feature Selection Method ◽  
Reduction Process ◽  
Network Intrusion Detection ◽  
Network Intrusion ◽  
Network Intrusion Detection System ◽  
Network Intrusions

Network intrusions detection is a continuous vigilant task and to efficiently analyze the traffic in the corporate network to detect network intrusions. The efficiency of the Network Intrusion Detection System (NIDS) performance can be improved by adopting feature selection or reduction process to suit the present day high speed real time networks. This work is focused on identifying the key features of the audit dataset used to build an efficient light-weight NIDS. The NSL KDD dataset is used in this work titled Attribute Richness Based Feature Selection (ARFS) in order to analyze its performance.The obtained results are compared with the Correlation-based Feature Selection (CFS) and Information Gain (IG) feature selection methods. The proposed feature selection method produced better detection rate comparatively.

Sign in / Sign up

Export Citation Format

Share Document