Deep Belief Neural Networks and Bidirectional Long-Short Term Memory Hybrid for Speech Recognition

2015 ◽  
Vol 40 (2) ◽  
pp. 191-195 ◽  
Author(s):  
Łukasz Brocki ◽  
Krzysztof Marasek
Keyword(s):  
Neural Networks ◽  
Speech Recognition ◽  
Short Term Memory ◽  
Recognition Accuracy ◽  
Superior Performance ◽  
Acoustic Model ◽  
Short Term ◽  
Term Memory ◽  
Internal Memory ◽  
Long Short Term Memory

Abstract This paper describes a Deep Belief Neural Network (DBNN) and Bidirectional Long-Short Term Memory (LSTM) hybrid used as an acoustic model for Speech Recognition. It was demonstrated by many independent researchers that DBNNs exhibit superior performance to other known machine learning frameworks in terms of speech recognition accuracy. Their superiority comes from the fact that these are deep learning networks. However, a trained DBNN is simply a feed-forward network with no internal memory, unlike Recurrent Neural Networks (RNNs) which are Turing complete and do posses internal memory, thus allowing them to make use of longer context. In this paper, an experiment is performed to make a hybrid of a DBNN with an advanced bidirectional RNN used to process its output. Results show that the use of the new DBNN-BLSTM hybrid as the acoustic model for the Large Vocabulary Continuous Speech Recognition (LVCSR) increases word recognition accuracy. However, the new model has many parameters and in some cases it may suffer performance issues in real-time applications.

scholarly journals Developing Deep Survival Model for Remaining Useful Life Estimation Based on Convolutional and Long Short-Term Memory Neural Networks

2020 ◽  
Vol 2020 ◽  
pp. 1-12
Author(s):  
Chia-Hua Chu ◽  
Chia-Jung Lee ◽  
Hsiang-Yuan Yeh
Keyword(s):  
Neural Networks ◽  
Short Term Memory ◽  
Survival Model ◽  
Remaining Useful Life ◽  
Superior Performance ◽  
Short Term ◽  
Term Memory ◽  
Useful Life ◽  
Long Short Term Memory ◽  
Prediction Approach

The application of mechanical equipment in manufacturing is becoming more and more complicated with technology development and adoption. In order to keep the high reliability and stability of the production line, reducing the downtime to repair and the frequency of routine maintenance is necessary. Since machine and components’ degradations are inevitable, accurately estimating the remaining useful life of them is crucial. We propose an integrated deep learning approach with convolutional neural networks and long short-term memory networks to learn the latent features and estimate remaining useful life value with deep survival model based on the discrete Weibull distribution. We conduct the turbofan engine degradation simulation dataset from Commercial Modular Aero-Propulsion System Simulation dataset provided by NASA to validate our approach. The improved results have proven that our proposed model can capture the degradation trend of a fault and has superior performance under complex conditions compared with existing state-of-the-art methods. Our study provides an efficient feature extraction scheme and offers a promising prediction approach to make better maintenance strategies.

scholarly journals Applying long short-term memory recurrent neural networks to intrusion detection

2015 ◽  
Vol 56 ◽  
Author(s):  
Ralf C. Staudemeyer
Keyword(s):  
Neural Networks ◽  
Intrusion Detection ◽  
Recurrent Neural Networks ◽  
Short Term Memory ◽  
Superior Performance ◽  
Short Term ◽  
Term Memory ◽  
Feature Sets ◽  
Long Short Term Memory ◽  
Kdd Cup 99

We claim that modelling network traffic as a time series with a supervised learning approach, using known genuine and malicious behaviour, improves intrusion detection. To substantiate this, we trained long short-term memory (LSTM) recurrent neural networks with the training data provided by the DARPA / KDD Cup ’99 challenge. To identify suitable LSTM-RNN network parameters and structure we experimented with various network topologies. We found networks with four memory blocks containing two cells each offer a good compromise between computational cost and detection performance. We applied forget gates and shortcut connections respectively. A learning rate of 0.1 and up to 1,000 epochs showed good results. We tested the performance on all features and on extracted minimal feature sets respectively. We evaluated different feature sets for the detection of all attacks within one network and also to train networks specialised on individual attack classes. Our results show that the LSTM classifier provides superior performance in comparison to results previously published results of strong static classifiers. With 93.82% accuracy and 22.13 cost, LSTM outperforms the winning entries of the KDD Cup ’99 challenge by far. This is due to the fact that LSTM learns to look back in time and correlate consecutive connection records. For the first time ever, we have demonstrated the usefulness of LSTM networks to intrusion detection.

Sign in / Sign up

Export Citation Format

Share Document