scholarly journals XFinder: Detecting Unknown Anomalies in Distributed Machine Learning Scenario

2021 ◽  
Vol 3 ◽  
Author(s):  
Haizhou Du ◽  
Shiwei Wang ◽  
Huan Huo
Keyword(s):  
Machine Learning ◽  
Anomaly Detection ◽  
Real Time ◽  
Network Traffic ◽  
Security And Privacy ◽  
Closed Sets ◽  
Online Mode ◽  
Average Accuracy ◽  
Learning Scenarios ◽  
Distributed Machine Learning

In recent years, the emergence of distributed machine learning has enabled deep learning models to ensure data security and privacy while training efficiently. Anomaly detection for network traffic in distributed machine learning scenarios is of great significance for network security. Although deep neural networks have made remarkable achievements in anomaly detection for network traffic, they mainly focus on closed sets, that is, assuming that all anomalies are known. However, in a real network environment, unknown abnormalities are fatal risks faced by the system because they have no labels and occur before the known anomalies. In this study, we design and implement XFinder, a dynamic unknown traffic anomaly detection framework in distributed machine learning. XFinder adopts an online mode to detect unknown anomalies in real-time. XFinder detects unknown anomalies by the unknowns detector, transfers the unknown anomalies to the prior knowledge base by the network updater, and adopts the online mode to report new anomalies in real-time. The experimental results show that the average accuracy of the unknown anomaly detection of our model is increased by 27% and the average F1-Score is improved by 20%. Compared with the offline mode, XFinder’s detection time is reduced by an average of approximately 33% on three datasets, and can better meet the network requirement.

scholarly journals Detection of Abnormalities in Real-Time Computer Network Traffic Empowered by Machine Learning

2021 ◽  
Vol 10 (3) ◽  
pp. 2072-2079
Keyword(s):  
Machine Learning ◽  
Anomaly Detection ◽  
Real Time ◽  
Network Traffic ◽  
Computer Network ◽  
Data Gathering ◽  
Network Systems ◽  
Data Set ◽  
Synthetic Network ◽  
Scientific Papers

This research discloses how to utilize machine learning methods for anomaly detection in real-time on a computer network. While utilizing machine learning for this task is definitely not a novel idea, little literature is about the matter of doing it in real-time. Most machine learning research in PC network anomaly detection depends on the KDD '99 data set and means to demonstrate the proficiency of the algorithms introduced. The emphasis on this data set has caused a lack of scientific papers disclosing how to assemble network data, remove features, and train algorithms for use inreal-time networks. It has been contended that utilizing the KDD '99 dataset for anomaly detection is not appropriate for real-time network systems. This research proposes how the data gathering procedure will be possible utilizing a dummy network and generating synthetic network traffic by analyzing the importance of One-class SVM. As the efficiency of k-means clustering and LTSM neural networks is lower than one-class SVM, that is why this research uses the results of existing research of LSTM and k-means clustering for the comparison with reported outcomes of a similar algorithm on the KDD '99 dataset. Precisely, without engaging KDD ’99 data set by using synthetic network traffic, this research achieved the higher accuracy as compared to the previous researches.

scholarly journals An Adaptive Multi-Layer Botnet Detection Technique Using Machine Learning Classifiers

2019 ◽  
Vol 9 (11) ◽  
pp. 2375 ◽  
Author(s):  
Riaz Ullah Khan ◽  
Xiaosong Zhang ◽  
Rajesh Kumar ◽  
Abubakar Sharif ◽  
Noorbakhsh Amiri Golilarz ◽  
...  
Keyword(s):  
Machine Learning ◽  
Decision Tree ◽  
Network Traffic ◽  
Traffic Classification ◽  
Decision Tree Classifier ◽  
Machine Learning Classifiers ◽  
Learning Classifiers ◽  
Average Accuracy ◽  
Final Layer ◽  
Tree Classifier

In recent years, the botnets have been the most common threats to network security since it exploits multiple malicious codes like a worm, Trojans, Rootkit, etc. The botnets have been used to carry phishing links, to perform attacks and provide malicious services on the internet. It is challenging to identify Peer-to-peer (P2P) botnets as compared to Internet Relay Chat (IRC), Hypertext Transfer Protocol (HTTP) and other types of botnets because P2P traffic has typical features of the centralization and distribution. To resolve the issues of P2P botnet identification, we propose an effective multi-layer traffic classification method by applying machine learning classifiers on features of network traffic. Our work presents a framework based on decision trees which effectively detects P2P botnets. A decision tree algorithm is applied for feature selection to extract the most relevant features and ignore the irrelevant features. At the first layer, we filter non-P2P packets to reduce the amount of network traffic through well-known ports, Domain Name System (DNS). query, and flow counting. The second layer further characterized the captured network traffic into non-P2P and P2P. At the third layer of our model, we reduced the features which may marginally affect the classification. At the final layer, we successfully detected P2P botnets using decision tree Classifier by extracting network communication features. Furthermore, our experimental evaluations show the significance of the proposed method in P2P botnets detection and demonstrate an average accuracy of 98.7%.

scholarly journals Internet of Things: A Comprehensive Study on Machine Learning-based Intrusion Detection approaches

2021 ◽  
Author(s):  
Priyanka Gupta ◽  
Lokesh Yadav ◽  
Deepak Singh Tomar
Keyword(s):  
Machine Learning ◽  
Internet Of Things ◽  
Intrusion Detection ◽  
Real Time ◽  
Detection System ◽  
Research Work ◽  
Security And Privacy ◽  
Iot Security ◽  
Advantages And Disadvantages ◽  
Iot Devices

The Internet of Things (IoT) connects billions of interconnected devices that can exchange information with each other with minimal user intervention. The goal of IoT to become accessible to anyone, anytime, and anywhere. IoT has engaged in multiple fields, including education, healthcare, businesses, and smart home. Security and privacy issues have been significant obstacles to the widespread adoption of IoT. IoT devices cannot be entirely secure from threats; detecting attacks in real-time is essential for securing devices. In the real-time communication domain and especially in IoT, security and protection are the major issues. The resource-constrained nature of IoT devices makes traditional security techniques difficult. In this paper, the research work carried out in IoT Intrusion Detection System is presented. The Machine learning methods are explored to provide an effective security solution for IoT Intrusion Detection systems. Then discussed the advantages and disadvantages of the selected methodology. Further, the datasets used in IoT security are also discussed. Finally, the examination of the open issues and directions for future trends are also provided.

scholarly journals Adaptive Real-Time Method for Anomaly Detection Using Machine Learning

Proceedings ◽  
2020 ◽  
Vol 54 (1) ◽  
pp. 38
Author(s):  
David Novoa-Paradela ◽  
Óscar Fontenla-Romero ◽  
Bertha Guijarro-Berdiñas
Keyword(s):  
Machine Learning ◽  
Anomaly Detection ◽  
Real Time ◽  
State Of The Art ◽  
Learning Ability ◽  
Convex Hulls ◽  
Detection Systems ◽  
Anomalous Data ◽  
Frequent Use ◽  
Normal Class

Anomaly detection is a sub-area of machine learning that deals with the development of methods to distinguish among normal and anomalous data. Due to the frequent use of anomaly-detection systems in monitoring and the lack of methods capable of learning in real time, this research presents a new method that provides such online adaptability. The method bases its operation on the properties of scaled convex hulls. It begins building a convex hull, using a minimum set of data, that is adapted and subdivided along time to accurately fit the boundary of the normal class data. The model has online learning ability and its execution can be carried out in a distributed and parallel way, all of them interesting advantages when dealing with big datasets. The method has been compared to other state-of-the-art algorithms demonstrating its effectiveness.

Sign in / Sign up

Export Citation Format

Share Document