A Comparison of Selected Training Algorithms for Recurrent Neural Networks

Author(s):  
Suwat Pattamavorakun ◽  
Suwarin Pattamavorakun

Cyber security threats are an ever increasing, frequent and complex issue in the modern information era. With the advent of big data, incremental increase of huge amounts of data has further increased the security problems. Intrusion Detection Systems (IDS) were been developed to monitor and secure the cyber data systems and networks from any intrusions. However, the intrusion detection is difficult due to the rapid evolution of security attacks and the high volume, variety and speed of big data. In addition, the shallow architectures of existing IDS models lead to high computation cost and high memory requirements, thus further diminishing the efficiency of intrusion detection. The recent studies have suggested the use of data analytics and the deep learning algorithms can be effective in improving the IDS. An efficient IDS model is developed in this study by using the improved Elman-type Recurrent Neural Networks (RNN) in which the Improved Chicken Swarm Optimization (ICSO) optimally determines RNN parameters. RNN is an efficient method for classifying network traffic data but its traditional training algorithms are slow in convergence and faces local optimum problem. The introduction of ICSO with enhanced global search ability significantly avoids those limitations and improves the training process of RNN. This optimized deep learning algorithm of RNN, named as ICSO-RNN, is employed in the IDS with Intuitionistic Fuzzy Mutual Information feature selection to analyze larger network traffic datasets. The proposed IDS model using ICSO-RNN is tested on UNSW NB15 dataset. The final outcomes suggested that ICSO-RNN model has high performance in intrusion detection, with minimum training time and is proficient for big data


2020 ◽  
Author(s):  
Dean Sumner ◽  
Jiazhen He ◽  
Amol Thakkar ◽  
Ola Engkvist ◽  
Esben Jannik Bjerrum

<p>SMILES randomization, a form of data augmentation, has previously been shown to increase the performance of deep learning models compared to non-augmented baselines. Here, we propose a novel data augmentation method we call “Levenshtein augmentation” which considers local SMILES sub-sequence similarity between reactants and their respective products when creating training pairs. The performance of Levenshtein augmentation was tested using two state of the art models - transformer and sequence-to-sequence based recurrent neural networks with attention. Levenshtein augmentation demonstrated an increase performance over non-augmented, and conventionally SMILES randomization augmented data when used for training of baseline models. Furthermore, Levenshtein augmentation seemingly results in what we define as <i>attentional gain </i>– an enhancement in the pattern recognition capabilities of the underlying network to molecular motifs.</p>


Author(s):  
Faisal Ladhak ◽  
Ankur Gandhe ◽  
Markus Dreyer ◽  
Lambert Mathias ◽  
Ariya Rastrow ◽  
...  

Sign in / Sign up

Export Citation Format

Share Document