scholarly journals A Framework for Anomaly Detection under Dynamic and Distributed Scenarios

2021 ◽  
Author(s):  
◽  
Murugaraj Odiathevar
Keyword(s):  
Anomaly Detection ◽  
Real World ◽  
Detection System ◽  
Traffic Monitoring ◽  
Streaming Data ◽  
Multiple Sources ◽  
Real World Data ◽  
Detection Model ◽  
Healthcare Data ◽  
Robust Model

<p><b>Anomaly Detection is an important aspect of many application domains. It refers to the problem of finding patterns in data that do not conform to expected behaviour. Hence, understanding of expected behaviour well is fundamental to performing effective anomaly detection. However, data profiles constantly evolve in certain domains such as computer networks. In other domains such as traffic monitoring and healthcare, data are distributed and are either too large or there are privacy concerns in transmitting them to a central location. These situations pose a challenge to obtain an accurate understanding of non-anomalous profiles. Changing profiles undermine existing anomaly detection models and make them less effective. Training a robust model with data from multiple sources is also challenging. Moreover, in real world scenarios, it is not apparent how an anomaly detection model can be built to address the problem.</b></p> <p>This thesis focuses on the building of a robust anomaly detection system where data profiles evolve and/or are distributed. It proposes a novel Online Offline Framework to separate existing expected behaviour, new possible expected behaviour and anomalies in streaming data. It also addresses the distributed scenario using a theoretically sound fully Bayesian approach. These methods improve performances of anomaly detection systems and work well with biased and uneven data partitions.</p> <p>The proposed methods are validated using real world data in three different domains. This thesis identifies the implementation difficulties in these domains and produces three novel methodologies to address each of the core anomaly detection problems.</p>

scholarly journals A Framework for Anomaly Detection under Dynamic and Distributed Scenarios

2021 ◽  
Author(s):  
◽  
Murugaraj Odiathevar
Keyword(s):  
Anomaly Detection ◽  
Real World ◽  
Detection System ◽  
Traffic Monitoring ◽  
Streaming Data ◽  
Multiple Sources ◽  
Real World Data ◽  
Detection Model ◽  
Healthcare Data ◽  
Robust Model

<p><b>Anomaly Detection is an important aspect of many application domains. It refers to the problem of finding patterns in data that do not conform to expected behaviour. Hence, understanding of expected behaviour well is fundamental to performing effective anomaly detection. However, data profiles constantly evolve in certain domains such as computer networks. In other domains such as traffic monitoring and healthcare, data are distributed and are either too large or there are privacy concerns in transmitting them to a central location. These situations pose a challenge to obtain an accurate understanding of non-anomalous profiles. Changing profiles undermine existing anomaly detection models and make them less effective. Training a robust model with data from multiple sources is also challenging. Moreover, in real world scenarios, it is not apparent how an anomaly detection model can be built to address the problem.</b></p> <p>This thesis focuses on the building of a robust anomaly detection system where data profiles evolve and/or are distributed. It proposes a novel Online Offline Framework to separate existing expected behaviour, new possible expected behaviour and anomalies in streaming data. It also addresses the distributed scenario using a theoretically sound fully Bayesian approach. These methods improve performances of anomaly detection systems and work well with biased and uneven data partitions.</p> <p>The proposed methods are validated using real world data in three different domains. This thesis identifies the implementation difficulties in these domains and produces three novel methodologies to address each of the core anomaly detection problems.</p>

scholarly journals Anomaly Detection Using Deep Neural Network for IoT Architecture

2021 ◽  
Vol 11 (15) ◽  
pp. 7050
Author(s):  
Zeeshan Ahmad ◽  
Adnan Shahid Khan ◽  
Kashif Nisar ◽  
Iram Haider ◽  
Rosilah Hassan ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Anomaly Detection ◽  
Deep Neural Network ◽  
Detection System ◽  
Traffic Monitoring ◽  
Detection Accuracy ◽  
Learning Models ◽  
Prime Concern ◽  
Entry Points

The revolutionary idea of the internet of things (IoT) architecture has gained enormous popularity over the last decade, resulting in an exponential growth in the IoT networks, connected devices, and the data processed therein. Since IoT devices generate and exchange sensitive data over the traditional internet, security has become a prime concern due to the generation of zero-day cyberattacks. A network-based intrusion detection system (NIDS) can provide the much-needed efficient security solution to the IoT network by protecting the network entry points through constant network traffic monitoring. Recent NIDS have a high false alarm rate (FAR) in detecting the anomalies, including the novel and zero-day anomalies. This paper proposes an efficient anomaly detection mechanism using mutual information (MI), considering a deep neural network (DNN) for an IoT network. A comparative analysis of different deep-learning models such as DNN, Convolutional Neural Network, Recurrent Neural Network, and its different variants, such as Gated Recurrent Unit and Long Short-term Memory is performed considering the IoT-Botnet 2020 dataset. Experimental results show the improvement of 0.57–2.6% in terms of the model’s accuracy, while at the same time reducing the FAR by 0.23–7.98% to show the effectiveness of the DNN-based NIDS model compared to the well-known deep learning models. It was also observed that using only the 16–35 best numerical features selected using MI instead of 80 features of the dataset result in almost negligible degradation in the model’s performance but helped in decreasing the overall model’s complexity. In addition, the overall accuracy of the DL-based models is further improved by almost 0.99–3.45% in terms of the detection accuracy considering only the top five categorical and numerical features.

scholarly journals Network Anomaly Detection System with Optimized DS Evidence Theory

2014 ◽  
Vol 2014 ◽  
pp. 1-13 ◽  
Author(s):  
Yuan Liu ◽  
Xiaofeng Wang ◽  
Kaiyu Liu
Keyword(s):  
Anomaly Detection ◽  
Detection Rate ◽  
Computer Network ◽  
Detection System ◽  
Evidence Theory ◽  
Optimization Methods ◽  
High Detection Rate ◽  
Detection Model ◽  
Network Anomaly Detection ◽  
Anomaly Detection System

Network anomaly detection has been focused on by more people with the fast development of computer network. Some researchers utilized fusion method and DS evidence theory to do network anomaly detection but with low performance, and they did not consider features of network—complicated and varied. To achieve high detection rate, we present a novel network anomaly detection system with optimized Dempster-Shafer evidence theory (ODS) and regression basic probability assignment (RBPA) function. In this model, we add weights for each senor to optimize DS evidence theory according to its previous predict accuracy. And RBPA employs sensor’s regression ability to address complex network. By four kinds of experiments, we find that our novel network anomaly detection model has a better detection rate, and RBPA as well as ODS optimization methods can improve system performance significantly.

scholarly journals Emulated Clinical Trials from Longitudinal Real-World Data Efficiently Identify Candidates for Neurological Disease Modification: Examples from Parkinson’s Disease

2020 ◽  
Author(s):  
Daphna Laifenfeld ◽  
Chen Yanover ◽  
Michal Ozery-Flato ◽  
Oded Shaham ◽  
Michal Rozen-Zvi ◽  
...  
Keyword(s):  
Parkinson’S Disease ◽  
Parkinson's Disease ◽  
Clinical Trials ◽  
Real World ◽  
Late Onset ◽  
Real World Data ◽  
World Data ◽  
Healthcare Data ◽  
Beneficial Effects ◽  
Disease Modifying

AbstractReal-world healthcare data hold the potential to identify therapeutic solutions for progressive diseases by efficiently pinpointing safe and efficacious repurposing drug candidates. This approach circumvents key early clinical development challenges, particularly relevant for neurological diseases, concordant with the vision of the 21stCentury Cures Act. However, to-date, these data have been utilized mainly for confirmatory purposes rather than as drug discovery engines. Here, we demonstrate the usefulness of real-world data in identifying drug repurposing candidates for disease-modifying effects, specifically candidate marketed drugs that exhibit beneficial effects on Parkinson’s disease (PD) progression. We performed an observational study in cohorts of ascertained PD patients extracted from two large medical databases, Explorys SuperMart (N=88,867) and IBM MarketScan Research Databases (N=106,395); and applied two conceptually different, well-established causal inference methods to estimate the effect of hundreds of drugs on delaying dementia onset as a proxy for slowing PD progression. Using this approach, we identified two drugs that manifested significant beneficial effects on PD progression in both datasets: rasagiline, narrowly indicated for PD motor symptoms; and zolpidem, a psycholeptic. Each confers its effects through distinct mechanisms, which we explored via a comparison of estimated effects within the drug classification ontology. We conclude that analysis of observational healthcare data, emulating otherwise costly, large, and lengthy clinical trials, can highlight promising repurposing candidates, to be validated in prospective registration trials, for common, late-onset progressive diseases for which disease-modifying therapeutic solutions are scarce.

scholarly journals Increasing Trust in Real-World Evidence Through Evaluation of Observational Data Quality

2021 ◽  
Author(s):  
Clair Blacketer ◽  
Frank J Defalco ◽  
Patrick B Ryan ◽  
Peter R Rijnbeek
Keyword(s):  
Data Quality ◽  
Real World ◽  
R Package ◽  
Observational Research ◽  
Real World Data ◽  
Quality Reporting ◽  
Healthcare Data ◽  
Real World Evidence ◽  
Quality Checks

Advances in standardization of observational healthcare data have enabled methodological breakthroughs, rapid global collaboration, and generation of real-world evidence to improve patient outcomes. Standardizations in data structure, such as use of Common Data Models (CDM), need to be coupled with standardized approaches for data quality assessment. To ensure confidence in real-world evidence generated from the analysis of real-world data, one must first have confidence in the data itself. The Data Quality Dashboard is an open-source R package that reports potential quality issues in an OMOP CDM instance through the systematic execution and summarization of over 3,300 configurable data quality checks. We describe the implementation of check types across a data quality framework of conformance, completeness, plausibility, with both verification and validation. We illustrate how data quality checks, paired with decision thresholds, can be configured to customize data quality reporting across a range of observational health data sources. We discuss how data quality reporting can become part of the overall real-world evidence generation and dissemination process to promote transparency and build confidence in the resulting output. Transparently communicating how well CDM standardized databases adhere to a set of quality measures adds a crucial piece that is currently missing from observational research. Assessing and improving the quality of our data will inherently improve the quality of the evidence we generate.

scholarly journals A Hybrid Network Anomaly Detection system using Glowworm Swarm Optimization with Principal Component Analysis

2021 ◽  
Author(s):  
Rashmita Khilar ◽  
K. Mariyappan ◽  
Mary Subaja Christo ◽  
J Amutharaj ◽  
Anitha T ◽  
...  
Keyword(s):  
Principal Component Analysis ◽  
Intrusion Detection ◽  
Anomaly Detection ◽  
Detection Rate ◽  
Detection System ◽  
Principal Component ◽  
Swarm Optimization ◽  
Detection Model ◽  
Glowworm Swarm Optimization ◽  
Proposed Model

Abstract The security of the network is a significant issue in any distributed system. For that intrusion detection system (IDS), have been proposed for securing the network from malicious activities. This research is proposed to design and develop an anomaly detection model for detecting attacks and unusual activities in IoT networks. The primary objective of this research is to design efficient IDS for IoT network. The intrusion detection plays an essential role in detecting different attacks on IoT and enhances the performance of the IoT. In this research, anomaly detection in IoT networks using glowworm swarm optimization (GSO) algorithm with principal component analysis (PCA) is proposed. However, the proposed model is metaheuristic algorithm-based anomaly detection model to identify attacks by using the NSL-KDD dataset. The GSO algorithm based on PCA is implemented to perform the anomaly detection. For feature extraction, the PCA is used, and for classification, the GSO algorithm is used. For performance analysis, various parameters like accuracy, precision, recall, detection rate and FAR are evaluated. For normal class the proposed model achieved 94.14% accuracy, for DoS 95.52%, for R2L 93.15%, for probe 93.50% and for U2R 88.62% accuracy. Overall the detection rate was 94.08% and FAR was 3.41%.

Modbus/TCP Communication Anomaly Detection Based on PSO-SVM

2014 ◽  
Vol 490-491 ◽  
pp. 1745-1753 ◽  
Author(s):  
Wen Li Shang ◽  
Sheng Shan Zhang ◽  
Ming Wan
Keyword(s):  
Anomaly Detection ◽  
Intrusion Detection System ◽  
Detection System ◽  
Code Sequence ◽  
Industrial Control ◽  
Industrial Control Systems ◽  
Detection Model ◽  
Tcp Protocol ◽  
Function Code ◽  
Systems Simulation

Industrial firewall and intrusion detection system based on Modbus TCP protocol analysis and whitelist policy cannot effectively identify attacks on Modbus controller which exactly take advantage of the configured rules. An Industrial control systems simulation environment is established and a data preprocessing method for Modbus TCP traffic captured is designed to meet the need of anomaly detection module. Furthermore a Modbus function code sequence anomaly detection model based on SVM optimized by PSO method is designed. And the model can effectively identify abnormal Modbus TCP traffic, according to frequency of different short mode sequences in a Modbus code sequence.

scholarly journals Emulated Clinical Trials from Longitudinal Real-World Data Efficiently Identify Candidates for Neurological Disease Modification: Examples from Parkinson’s Disease

2021 ◽  
Vol 12 ◽  
Author(s):  
Daphna Laifenfeld ◽  
Chen Yanover ◽  
Michal Ozery-Flato ◽  
Oded Shaham ◽  
Michal Rosen-Zvi ◽  
...  
Keyword(s):  
Parkinson’S Disease ◽  
Parkinson's Disease ◽  
Clinical Trials ◽  
Real World ◽  
Late Onset ◽  
Real World Data ◽  
World Data ◽  
Healthcare Data ◽  
Beneficial Effects ◽  
Disease Modifying

Real-world healthcare data hold the potential to identify therapeutic solutions for progressive diseases by efficiently pinpointing safe and efficacious repurposing drug candidates. This approach circumvents key early clinical development challenges, particularly relevant for neurological diseases, concordant with the vision of the 21st Century Cures Act. However, to-date, these data have been utilized mainly for confirmatory purposes rather than as drug discovery engines. Here, we demonstrate the usefulness of real-world data in identifying drug repurposing candidates for disease-modifying effects, specifically candidate marketed drugs that exhibit beneficial effects on Parkinson’s disease (PD) progression. We performed an observational study in cohorts of ascertained PD patients extracted from two large medical databases, Explorys SuperMart (N = 88,867) and IBM MarketScan Research Databases (N = 106,395); and applied two conceptually different, well-established causal inference methods to estimate the effect of hundreds of drugs on delaying dementia onset as a proxy for slowing PD progression. Using this approach, we identified two drugs that manifested significant beneficial effects on PD progression in both datasets: rasagiline, narrowly indicated for PD motor symptoms; and zolpidem, a psycholeptic. Each confers its effects through distinct mechanisms, which we explored via a comparison of estimated effects within the drug classification ontology. We conclude that analysis of observational healthcare data, emulating otherwise costly, large, and lengthy clinical trials, can highlight promising repurposing candidates, to be validated in prospective registration trials, beneficial against common, late-onset progressive diseases for which disease-modifying therapeutic solutions are scarce.

scholarly journals A Hierarchical Intrusion Detection System for Industrial Control Networks based on EtherNet/IP

2019 ◽  
Author(s):  
Wenbin Yu ◽  
Yiyin Wang ◽  
Lei Song
Keyword(s):  
Intrusion Detection ◽  
Anomaly Detection ◽  
Prediction Model ◽  
Intrusion Detection System ◽  
Detection System ◽  
Traffic Prediction ◽  
Support Vector ◽  
Industrial Control ◽  
Traffic Pattern ◽  
Detection Model

Standard Ethernet (IEEE 802.3 and the TCP/IP protocol suite) is gradually applied in industrial control system (ICS) with the development of information technology. It breaks the natural isolation of ICS, but contains no security mechanism. A modified intrusion detection system (IDS), which is strongly correlated to specific industrial scenario, is necessary for modern ICS. On the one hand, this paper outlines attack models, including infiltration attacks and our creative forging attack. On the other hand, we proposes a hierarchical IDS, which contains a traffic prediction model and an anomaly detection model. The traffic prediction model, which is based on autoregressive integrated moving average (ARIMA), can forecast the traffic of ICS network in the short term and precisely detect the infiltration attacks according to abnormal changes in traffic pattern. The anomaly detection model using one-class support vector machine (OCSVM) is able to detect malicious control instructions by analyzing the key field in EtherNet/IP packets. The experimental results show that the hierarchical IDS has an outstanding performance in detecting infiltration attacks and forging attack compared with other two innovative IDSs.

Sign in / Sign up

Export Citation Format

Share Document