Data Mining in Cyber Threat Analysis – Neural Networks for Intrusion Detection

2016 ◽  
Vol 15 (2) ◽  
pp. 187-196
Author(s):  
Eszter Katalin Bognár
Keyword(s):  
Neural Network ◽  
Data Mining ◽  
Neural Networks ◽  
Intrusion Detection ◽  
Training Data ◽  
Detection Accuracy ◽  
Threat Analysis ◽  
Very High ◽  
Computation Speed ◽  
Selection Of

The most important features and constraints of the commercial intrusion detection (IDS) and prevention (IPS) systems and the possibility of application of artificial intelligence and neural networks such as IDS or IPS were investigated. A neural network was trained using the Levenberg-Marquardt backpropagation algorithm and applied on the Knowledge Discovery and Data Mining (KDD)’99 [14] reference dataset. A very high (99.9985%) accuracy and rather low (3.006%) false alert rate was achieved, but only at the expense of high memory consumption and low computation speed. To overcome these limitations, the selection of training data size was investigated. Result shows that a neural network trained on ca. 50,000 data is enough to achieve a detection accuracy of 99.82%.

A VISUALIZATION AND DESIGN TOOL (AVID) FOR DATA MINING WITH THE SELF-ORGANIZING FEATURE MAP

2000 ◽  
Vol 09 (03) ◽  
pp. 369-375
Author(s):  
SUSAN E. GEORGE
Keyword(s):  
Neural Network ◽  
Data Mining ◽  
Software Tool ◽  
Design Tool ◽  
The Self ◽  
Training Data ◽  
Feature Map ◽  
Network Training ◽  
Som Network ◽  
Selection Of

This paper presents a software tool called AVID (A VIsualization and Design) which is particularly useful for data mining with an artificial neural network known as the self-organising feature map (SOM). AVID supports network training in both the i) selection of network inputs and ii) visualisation of the trained SOM. Both these features are novel aids to SOM network training and are particularly important when consideration is given to using the SOM for data mining. Once trained the SOM produces a 2-dimensional topological ordering of the input training data and it is particularly useful for representing the relationships within multi-dimensional data. The main classes within the data can be identified from the output map. AVID is an important software tool which enables data mining with the SOM by the selection of network inputs and the subsequent visualisation of the classes within these input vectors.

Operational Determination of the Activated Sludge Process Using Neural Networks

1992 ◽  
Vol 26 (9-11) ◽  
pp. 2461-2464 ◽  
Author(s):  
R. D. Tyagi ◽  
Y. G. Du
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Steady State ◽  
Activated Sludge ◽  
Feedforward Neural Network ◽  
Training Data ◽  
Activated Sludge Process

A steady-statemathematical model of an activated sludgeprocess with a secondary settler was developed. With a limited number of training data samples obtained from the simulation at steady state, a feedforward neural network was established which exhibits an excellent capability for the operational prediction and determination.

The Research of Intrusion Detection Based on Intelligentization

2014 ◽  
Vol 490-491 ◽  
pp. 1588-1591
Author(s):  
Liang Zhang ◽  
Hao Yue Sun ◽  
Guo Lv ◽  
Xiao Lu Sun
Keyword(s):  
Neural Network ◽  
Genetic Algorithm ◽  
Intrusion Detection ◽  
Convergence Speed ◽  
Global Property ◽  
Detection Accuracy ◽  
Accuracy Rate ◽  
The Neural Network

In this paper, the intelligentized way is applied to detecting anomaly intrusion. Based on the global property of genetic algorithm and the locality of neural network, this method effectively improves the convergence speed of the network and the detection accuracy rate. It not only avoids the defect of the neural network, but also improves the precision.

scholarly journals Evaluation of Power Insulator Detection Efficiency with the Use of Limited Training Dataset

2020 ◽  
Vol 10 (6) ◽  
pp. 2104
Author(s):  
Michał Tomaszewski ◽  
Paweł Michalski ◽  
Jakub Osuchowski
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Object Detection ◽  
Convolutional Neural Network ◽  
Deep Neural Networks ◽  
Detection Efficiency ◽  
Training Data ◽  
Training Dataset ◽  
Training Set ◽  
Convolutional Network

This article presents an analysis of the effectiveness of object detection in digital images with the application of a limited quantity of input. The possibility of using a limited set of learning data was achieved by developing a detailed scenario of the task, which strictly defined the conditions of detector operation in the considered case of a convolutional neural network. The described solution utilizes known architectures of deep neural networks in the process of learning and object detection. The article presents comparisons of results from detecting the most popular deep neural networks while maintaining a limited training set composed of a specific number of selected images from diagnostic video. The analyzed input material was recorded during an inspection flight conducted along high-voltage lines. The object detector was built for a power insulator. The main contribution of the presented papier is the evidence that a limited training set (in our case, just 60 training frames) could be used for object detection, assuming an outdoor scenario with low variability of environmental conditions. The decision of which network will generate the best result for such a limited training set is not a trivial task. Conducted research suggests that the deep neural networks will achieve different levels of effectiveness depending on the amount of training data. The most beneficial results were obtained for two convolutional neural networks: the faster region-convolutional neural network (faster R-CNN) and the region-based fully convolutional network (R-FCN). Faster R-CNN reached the highest AP (average precision) at a level of 0.8 for 60 frames. The R-FCN model gained a worse AP result; however, it can be noted that the relationship between the number of input samples and the obtained results has a significantly lower influence than in the case of other CNN models, which, in the authors’ assessment, is a desired feature in the case of a limited training set.

scholarly journals RICNN: A ResNet&Inception convolutional neural network for intrusion detection of abnormal traffic

2021 ◽  
pp. 55-55
Author(s):  
Benhui Xia ◽  
Dezhi Han ◽  
Ximing Yin ◽  
Gao Na
Keyword(s):  
Neural Network ◽  
Intrusion Detection ◽  
Convolutional Neural Network ◽  
Detection Rate ◽  
Recognition Rate ◽  
Identification Accuracy ◽  
Detection Accuracy ◽  
Traffic Classification ◽  
High Detection Rate ◽  
Network Intrusion

To secure cloud computing and outsourced data while meeting the requirements of automation, many intrusion detection schemes based on deep learn ing are proposed. Though the detection rate of many network intrusion detection solutions can be quite high nowadays, their identification accuracy on imbalanced abnormal network traffic still remains low. Therefore, this paper proposes a ResNet &Inception-based convolutional neural network (RICNN) model to abnormal traffic classification. RICNN can learn more traffic features through the Inception unit, and the degradation problem of the network is eliminated through the direct map ping unit of ResNet, thus the improvement of the model?s generalization ability can be achievable. In addition, to simplify the network, an improved version of RICNN, which makes it possible to reduce the number of parameters that need to be learnt without degrading identification accuracy, is also proposed in this paper. The experimental results on the dataset CICIDS2017 show that RICNN not only achieves an overall accuracy of 99.386% but also has a high detection rate across different categories, especially for small samples. The comparison experiments show that the recognition rate of RICNN outperforms a variety of CNN models and RNN models, and the best detection accuracy can be achieved.

scholarly journals Data Size Requirement for Forecasting Daily Crude Oil Price with Neural Networks

2019 ◽  
Vol 66 (3) ◽  
pp. 363-388
Author(s):  
Serkan Aras ◽  
Manel Hamdi
Keyword(s):  
Neural Networks ◽  
Crude Oil ◽  
Significant Interaction ◽  
Statistical Tests ◽  
Oil Price ◽  
Training Data ◽  
Forecasting Model ◽  
Crude Oil Price ◽  
Crude Oil Prices ◽  
Selection Of

When the literature regarding applications of neural networks is investigated, it appears that a substantial issue is what size the training data should be when modelling a time series through neural networks. The aim of this paper is to determine the size of training data to be used to construct a forecasting model via a multiple-breakpoint test and compare its performance with two general methods, namely, using all available data and using just two years of data. Furthermore, the importance of the selection of the final neural network model is investigated in detail. The results obtained from daily crude oil prices indicate that the data from the last structural change lead to simpler architectures of neural networks and have an advantage in reaching more accurate forecasts in terms of MAE value. In addition, the statistical tests show that there is a statistically significant interaction between data size and stopping rule.

scholarly journals Lightweight Convolutional Neural Network Based Intrusion Detection System

2020 ◽  
pp. 808-817
Author(s):  
Vinh Pham ◽  
◽  
Eunil Seo ◽  
Tai-Myoung Chung
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Intrusion Detection ◽  
Convolutional Neural Network ◽  
Network Traffic ◽  
Detection System ◽  
Image Data ◽  
Training Data ◽  
Deep Packet Inspection ◽  
Detection Model

Identifying threats contained within encrypted network traffic poses a great challenge to Intrusion Detection Systems (IDS). Because traditional approaches like deep packet inspection could not operate on encrypted network traffic, machine learning-based IDS is a promising solution. However, machine learning-based IDS requires enormous amounts of statistical data based on network traffic flow as input data and also demands high computing power for processing, but is slow in detecting intrusions. We propose a lightweight IDS that transforms raw network traffic into representation images. We begin by inspecting the characteristics of malicious network traffic of the CSE-CIC-IDS2018 dataset. We then adapt methods for effectively representing those characteristics into image data. A Convolutional Neural Network (CNN) based detection model is used to identify malicious traffic underlying within image data. To demonstrate the feasibility of the proposed lightweight IDS, we conduct three simulations on two datasets that contain encrypted traffic with current network attack scenarios. The experiment results show that our proposed IDS is capable of achieving 95% accuracy with a reasonable detection time while requiring relatively small size training data.

A Neural Networks Pruning and Data Fusion Based Intrusion Detection Model

2014 ◽  
Vol 651-653 ◽  
pp. 1772-1775
Author(s):  
Wei Gong
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Intrusion Detection ◽  
Data Fusion ◽  
Input Data ◽  
The Other ◽  
Pruning Algorithm ◽  
Detection Model ◽  
Process Ability

The abilities of summarization, learning and self-fitting and inner-parallel computing make artificial neural networks suitable for intrusion detection. On the other hand, data fusion based IDS has been used to solve the problem of distorting rate and failing-to-report rate and improve its performance. However, multi-sensor input-data makes the IDS lose its efficiency. The research of neural network based data fusion IDS tries to combine the strong process ability of neural network with the advantages of data fusion IDS. A neural network is designed to realize the data fusion and intrusion analysis and Pruning algorithm of neural networks is used for filtering information from multi-sensors. In the process of intrusion analysis pruning algorithm of neural networks is used for filtering information from multi-sensors so as to increase its performance and save the bandwidth of networks.

Prediction of slip in cable-drum systems using structured neural networks

2013 ◽  
Vol 228 (3) ◽  
pp. 441-456 ◽  
Author(s):  
Ergin Kilic ◽  
Melik Dolen
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Size ◽  
Network Architectures ◽  
Network Development ◽  
Low Velocity ◽  
Error Band ◽  
Artificial Neural ◽  
Recurrent Type ◽  
Selection Of

This study focuses on the slip prediction in a cable-drum system using artificial neural networks for the prospect of developing linear motion sensing scheme for such mechanisms. Both feed-forward and recurrent-type artificial neural network architectures are considered to capture the slip dynamics of cable-drum mechanisms. In the article, the network development is presented in a progressive (step-by-step) fashion for the purpose of not only making the design process transparent to the readers but also highlighting the corresponding challenges associated with the design phase (i.e. selection of architecture, network size, training process parameters, etc.). Prediction performances of the devised networks are evaluated rigorously via an experimental study. Finally, a structured neural network, which embodies the network with the best prediction performance, is further developed to overcome the drift observed at low velocity. The study illustrates that the resulting structured neural network could predict the slip in the mechanism within an error band of 100 µm when an absolute reference is utilized.

NEURAL NETWORK METHODS FOR PLANAR IMAGE ANALYSIS IN AUTOMATED SCREENING SYSTEMS

2021 ◽  
Vol 4 (1) ◽  
pp. 71-79
Author(s):  
Borys Igorovych Tymchenko
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Early Stage ◽  
Semantic Segmentation ◽  
Input Image ◽  
Training Data ◽  
Classification Task ◽  
Automated Screening ◽  
The Cost ◽  
Screening Systems

Nowadays, means of preventive management in various spheres of human life are actively developing. The task of automated screening is to detect hidden problems at an early stage without human intervention, while the cost of responding to them is low. Visual inspection is often used to perform a screening task. Deep artificial neural networks are especially popular in image processing. One of the main problems when working with them is the need for a large amount of well-labeled data for training. In automated screening systems, available neural network approaches have limitations on the reliability of predictions due to the lack of accurately marked training data, as obtaining quality markup from professionals is very expensive, and sometimes not possible in principle. Therefore, there is a contradiction between increasing the requirements for the precision of predictions of neural network models without increasing the time spent on the one hand, and the need to reduce the cost of obtaining the markup of educational data. In this paper, we propose the parametric model of the segmentation dataset, which can be used to generate training data for model selection and benchmarking; and the multi-task learning method for training and inference of deep neural networks for semantic segmentation. Based on the proposed method, we develop a semi-supervised approach for segmentation of salient regions for classification task. The main advantage of the proposed method is that it uses semantically-similar general tasks, that have better labeling than original one, what allows users to reduce the cost of the labeling process. We propose to use classification task as a more general to the problem of semantic segmentation. As semantic segmentation aims to classify each pixel in the input image, classification aims to assign a class to all of the pixels in the input image. We evaluate our methods using the proposed dataset model, observing the Dice score improvement by seventeen percent. Additionally, we evaluate the robustness of the proposed method to different amount of the noise in labels and observe consistent improvement over baseline version.

Sign in / Sign up

Export Citation Format

Share Document