Semi-Supervised Machine Learning Aided Anomaly Detection Method in Cellular Networks

The ever-increasing amount of data in cellular networks poses challenges for network operators to monitor the quality of experience (QoE). Traditional key quality indicators (KQIs)-based hard decision methods are difficult to undertake the task of QoE anomaly detection in the case of big data. To solve this problem, in this paper, we propose a KQIs-based QoE anomaly detection framework using semi-supervised machine learning algorithm, i.e., iterative positive sample aided one-class support vector machine (IPS-OCSVM). There are four steps for realizing the proposed method while the key step is combining machine learning with the network operator's expert knowledge using OCSVM. Our proposed IPS-OCSVM framework realizes QoE anomaly detection through soft decision and can easily fine-tune the anomaly detection ability on demand. Moreover, we prove that the fluctuation of KQIs thresholds based on expert knowledge has a limited impact on the result of anomaly detection. Finally, experiment results are given to confirm the proposed IPS-OCSVM framework for QoE anomaly detection in cellular networks.

Download Full-text

Semi-supervised Machine Learning Aided Anomaly Detection Method in Cellular Networks

10.36227/techrxiv.11634720.v1 ◽

2020 ◽

Author(s):

Yutao Lu ◽

Juan Wang ◽

Miao Liu ◽

Kaixuan Zhang ◽

Guan Gui ◽

...

Keyword(s):

Machine Learning ◽

Anomaly Detection ◽

Cellular Networks ◽

Expert Knowledge ◽

Learning Algorithm ◽

Positive Sample ◽

Supervised Machine Learning ◽

Support Vector ◽

Soft Decision ◽

Decision Methods

The ever-increasing amount of data in cellular networks poses challenges for network operators to monitor the quality of experience (QoE). Traditional key quality indicators (KQIs)-based hard decision methods are difficult to undertake the task of QoE anomaly detection in the case of big data. To solve this problem, in this paper, we propose a KQIs-based QoE anomaly detection framework using semi-supervised machine learning algorithm, i.e., iterative positive sample aided one-class support vector machine (IPS-OCSVM). There are four steps for realizing the proposed method while the key step is combining machine learning with the network operator's expert knowledge using OCSVM. Our proposed IPS-OCSVM framework realizes QoE anomaly detection through soft decision and can easily fine-tune the anomaly detection ability on demand. Moreover, we prove that the fluctuation of KQIs thresholds based on expert knowledge has a limited impact on the result of anomaly detection. Finally, experiment results are given to confirm the proposed IPS-OCSVM framework for QoE anomaly detection in cellular networks.

Download Full-text

Effective and efficient network anomaly detection system using machine learning algorithm

Bulletin of Electrical Engineering and Informatics ◽

10.11591/eei.v8i1.1387 ◽

2019 ◽

Vol 8 (1) ◽

pp. 46-51 ◽

Cited By ~ 7

Author(s):

Mukrimah Nawir ◽

Amiza Amir ◽

Naimah Yaakob ◽

Ong Bi Lynn

Keyword(s):

Machine Learning ◽

Anomaly Detection ◽

Processing Time ◽

Learning Algorithm ◽

Detection System ◽

Malicious Code ◽

Supervised Machine Learning ◽

Machine Learning Algorithm ◽

Network Anomaly Detection ◽

Anomaly Detection System

Network anomaly detection system enables to monitor computer network that behaves differently from the network protocol and it is many implemented in various domains. Yet, the problem arises where different application domains have different defining anomalies in their environment. These make a difficulty to choose the best algorithms that suit and fulfill the requirements of certain domains and it is not straightforward. Additionally, the issue of centralization that cause fatal destruction of network system when powerful malicious code injects in the system. Therefore, in this paper we want to conduct experiment using supervised Machine Learning (ML) for network anomaly detection system that low communication cost and network bandwidth minimized by using UNSW-NB15 dataset to compare their performance in term of their accuracy (effective) and processing time (efficient) for a classifier to build a model. Supervised machine learning taking account the important features by labelling it from the datasets. The best machine learning algorithm for network dataset is AODE with a comparable accuracy is 97.26% and time taken approximately 7 seconds. Also, distributed algorithm solves the issue of centralization with the accuracy and processing time still a considerable compared to a centralized algorithm even though a little drop of the accuracy and a bit longer time needed.

Download Full-text

Machine Learning-Based System Electromagnetic Environment Anomaly Detection Method

2018 International Conference on Smart Grid and Electrical Automation (ICSGEA) ◽

10.1109/icsgea.2018.00036 ◽

2018 ◽

Author(s):

Zhang Weisha ◽

Sun Jinguang ◽

Lu Jiazhong

Keyword(s):

Machine Learning ◽

Anomaly Detection ◽

Detection Method ◽

Electromagnetic Environment

Download Full-text

Gaussian Distribution-Based Machine Learning Scheme for Anomaly Detection in Healthcare Sensor Cloud

International Journal of Cloud Applications and Computing ◽

10.4018/ijcac.2021010103 ◽

2021 ◽

Vol 11 (1) ◽

pp. 52-72

Author(s):

Rajendra Kumar Dwivedi ◽

Rakesh Kumar ◽

Rajkumar Buyya

Keyword(s):

Machine Learning ◽

Information Systems ◽

Anomaly Detection ◽

Gaussian Distribution ◽

Supervised Machine Learning ◽

Sensor Data ◽

Support Vector ◽

Learning Scheme ◽

Healthcare Monitoring ◽

Sensor Cloud

Smart information systems are based on sensors that generate a huge amount of data. This data can be stored in cloud for further processing and efficient utilization. Anomalous data might be present within the sensor data due to various reasons (e.g., malicious activities by intruders, low quality sensors, and node deployment in harsh environments). Anomaly detection is crucial in some applications such as healthcare monitoring systems, forest fire information systems, and other internet of things (IoT) systems. This paper proposes a Gaussian distribution-based supervised machine learning scheme of anomaly detection (GDA) for healthcare monitoring sensor cloud, which is an integration of various body sensors of different patients and cloud. This work is implemented in Python. Use of Gaussian statistical model in the proposed scheme improves precision, throughput, and efficiency. GDA provides 98% efficiency with 3% and 4% improvements as compared to the other supervised learning-based anomaly detection schemes (e.g., support vector machine [SVM] and self-organizing map [SOM], respectively).

Download Full-text

Danger-Pose Detection System Using Commodity Wi-Fi for Bathroom Monitoring

Sensors ◽

10.3390/s19040884 ◽

2019 ◽

Vol 19 (4) ◽

pp. 884 ◽

Cited By ~ 9

Author(s):

Zizheng Zhang ◽

Shigemi Ishida ◽

Shigeaki Tagashira ◽

Akira Fukuda

Keyword(s):

Machine Learning ◽

Anomaly Detection ◽

Detection Method ◽

Detection System ◽

Wearable Sensors ◽

Low Frequency ◽

Training Data ◽

Support Vector ◽

Dynamic Features ◽

Pose Detection

A bathroom has higher probability of accidents than other rooms due to a slippery floor and temperature change. Because of high privacy and humidity, we face difficulties in monitoring inside a bathroom using traditional healthcare methods based on cameras and wearable sensors. In this paper, we present a danger-pose detection system using commodity Wi-Fi devices, which can be applied to bathroom monitoring, preserving privacy. A machine learning-based detection method usually requires data collected in target situations, which is difficult in detection-of-danger situations. We therefore employ a machine learning-based anomaly-detection method that requires a small amount of data in anomaly conditions, minimizing the required training data collected in dangerous conditions. We first derive the amplitude and phase shift from Wi-Fi channel state information (CSI) to extract low-frequency components that are related to human activities. We then separately extract static and dynamic features from the CSI changes in time. Finally, the static and dynamic features are fed into a one-class support vector machine (SVM), which is used as an anomaly-detection method, to classify whether a user is not in bathtub, bathing safely, or in dangerous conditions. We conducted experimental evaluations and demonstrated that our danger-pose detection system achieved a high detection performance in a non-line-of-sight (NLOS) scenario.

Download Full-text

VADAF: Visualization for Abnormal Client Detection and Analysis in Federated Learning

ACM Transactions on Interactive Intelligent Systems ◽

10.1145/3426866 ◽

2021 ◽

Vol 11 (3-4) ◽

pp. 1-23

Author(s):

Linhao Meng ◽

Yating Wei ◽

Rusheng Pan ◽

Shuyue Zhou ◽

Jianwei Zhang ◽

...

Keyword(s):

Machine Learning ◽

Anomaly Detection ◽

Visual Analytics ◽

Detection Method ◽

Training Process ◽

Local Data ◽

Privacy And Security ◽

Potential Client ◽

Distributed Machine Learning ◽

Large Corpus

Federated Learning (FL) provides a powerful solution to distributed machine learning on a large corpus of decentralized data. It ensures privacy and security by performing computation on devices (which we refer to as clients) based on local data to improve the shared global model. However, the inaccessibility of the data and the invisibility of the computation make it challenging to interpret and analyze the training process, especially to distinguish potential client anomalies. Identifying these anomalies can help experts diagnose and improve FL models. For this reason, we propose a visual analytics system, VADAF, to depict the training dynamics and facilitate analyzing potential client anomalies. Specifically, we design a visualization scheme that supports massive training dynamics in the FL environment. Moreover, we introduce an anomaly detection method to detect potential client anomalies, which are further analyzed based on both the client model’s visual and objective estimation. Three case studies have demonstrated the effectiveness of our system in understanding the FL training process and supporting abnormal client detection and analysis.

Download Full-text

FMCW Radar-Based Anomaly Detection in Toilet by Supervised Machine Learning Classifier

2019 IEEE Global Communications Conference (GLOBECOM) ◽

10.1109/globecom38437.2019.9014123 ◽

2019 ◽

Author(s):

Wataru Takabatake ◽

Kohei Yamamoto ◽

Kentaroh Toyoda ◽

Tomoaki Ohtsuki ◽

Yohei Shibata ◽

...

Keyword(s):

Machine Learning ◽

Anomaly Detection ◽

Supervised Machine Learning ◽

Fmcw Radar ◽

Learning Classifier

Download Full-text

An online anomaly detection method for stream data using isolation principle and statistic histogram

International Journal of Modeling Simulation and Scientific Computing ◽

10.1142/s1793962315500178 ◽

2015 ◽

Vol 06 (02) ◽

pp. 1550017 ◽

Cited By ~ 1

Author(s):

Zhiguo Ding ◽

Minrui Fei ◽

Dajun Du

Keyword(s):

Machine Learning ◽

Anomaly Detection ◽

Ensemble Learning ◽

Detection Method ◽

Dynamic Evolution ◽

Stream Data ◽

Massive Datasets ◽

Online Anomaly Detection

Online anomaly detection for stream data has been explored recently, where the detector is supposed to be able to perform an accurate and timely judgment for the upcoming observation. However, due to the inherent complex characteristics of stream data, such as quick generation, tremendous volume and dynamic evolution distribution, how to develop an effective online anomaly detection method is a challenge. The main objective of this paper is to propose an adaptive online anomaly detection method for stream data. This is achieved by combining isolation principle with online ensemble learning, which is then optimized by statistic histogram. Three main algorithms are developed, i.e., online detector building algorithm, anomaly detecting algorithm and adaptive detector updating algorithm. To evaluate our proposed method, four massive datasets from the UCI machine learning repository recorded from real events were adopted. Extensive simulations based on these datasets show that our method is effective and robust against different scenarios.

Download Full-text