PROBLEMS AND FEATURES OF EVOLUTIONARY ALGORITHMS TO BUILD HYBRID TRAINING METHODS FOR RECURRENT NEURAL NETWORKS

2007 ◽  
Keyword(s):  
Neural Networks ◽  
Evolutionary Algorithms ◽  
Recurrent Neural Networks ◽  
Training Methods

scholarly journals Enhancement of Speech Recognition System by neural network approaches of Clustering

2013 ◽  
Vol 6 (1) ◽  
pp. 266-271
Author(s):  
Anurag Upadhyay ◽  
Chitranjanjit Kaur
Keyword(s):  
Neural Networks ◽  
Speech Recognition ◽  
Recurrent Neural Networks ◽  
Alternative Energy ◽  
Recognition Rate ◽  
Speech Sound ◽  
Recognition System ◽  
Training Methods ◽  
Indian Languages ◽  
Phone Recognition

This paper addresses the problem of speech recognition to identify various modes of speech data. Speaker sounds are the acoustic sounds of speech. Statistical models of speech have been widely used for speech recognition under neural networks. In paper we propose and try to justify a new model in which speech co articulation the effect of phonetic context on speech sound is modeled explicitly under a statistical framework. We study speech phone recognition by recurrent neural networks and SOUL Neural Networks. A general framework for recurrent neural networks and considerations for network training are discussed in detail. SOUL NN clustering the large vocabulary that compresses huge data sets of speech. This project also different Indian languages utter by different speakers in different modes such as aggressive, happy, sad, and angry. Many alternative energy measures and training methods are proposed and implemented. A speaker independent phone recognition rate of 82% with 25% frame error rate has been achieved on the neural data base. Neural speech recognition experiments on the NTIMIT database result in a phone recognition rate of 68% correct. The research results in this thesis are competitive with the best results reported in the literature. 

TRAINING RECURRENT NEURAL NETWORKS — THE MINIMAL TRAJECTORY ALGORITHM

1992 ◽  
Vol 03 (01) ◽  
pp. 83-101 ◽  
Author(s):  
D. Saad
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Learning Rule ◽  
Weight Matrix ◽  
Training Methods ◽  
Input Output ◽  
Feed Forward ◽  
End Point ◽  
Common Weight ◽  
Perceptron Learning

The Minimal Trajectory (MINT) algorithm for training recurrent neural networks with a stable end point is based on an algorithmic search for the systems’ representations in the neighbourhood of the minimal trajectory connecting the input-output representations. The said representations appear to be the most probable set for solving the global perceptron problem related to the common weight matrix, connecting all representations of successive time steps in a recurrent discrete neural networks. The search for a proper set of system representations is aided by representation modification rules similar to those presented in our former paper,1 aimed to support contributing hidden and non-end-point representations while supressing non-contributing ones. Similar representation modification rules were used in other training methods for feed-forward networks,2–4 based on modification of the internal representations. A feed-forward version of the MINT algorithm will be presented in another paper.5 Once a proper set of system representations is chosen, the weight matrix is then modified accordingly, via the Perceptron Learning Rule (PLR) to obtain the proper input-output relation. Computer simulations carried out for the restricted cases of parity and teacher-net problems show rapid convergence of the algorithm in comparison with other existing algorithms, together with modest memory requirements.

scholarly journals Examining Attention Mechanisms in Deep Learning Models for Sentiment Analysis

2021 ◽  
Vol 11 (9) ◽  
pp. 3883
Author(s):  
Spyridon Kardakis ◽  
Isidoros Perikos ◽  
Foteini Grivokostopoulou ◽  
Ioannis Hatzilygeroudis
Keyword(s):  
Neural Networks ◽  
Experimental Study ◽  
Deep Learning ◽  
Comparative Analysis ◽  
Sentiment Analysis ◽  
Recurrent Neural Networks ◽  
Deep Neural Networks ◽  
Training Methods ◽  
Learning Models ◽  
Neural Models

Attention-based methods for deep neural networks constitute a technique that has attracted increased interest in recent years. Attention mechanisms can focus on important parts of a sequence and, as a result, enhance the performance of neural networks in a variety of tasks, including sentiment analysis, emotion recognition, machine translation and speech recognition. In this work, we study attention-based models built on recurrent neural networks (RNNs) and examine their performance in various contexts of sentiment analysis. Self-attention, global-attention and hierarchical-attention methods are examined under various deep neural models, training methods and hyperparameters. Even though attention mechanisms are a powerful recent concept in the field of deep learning, their exact effectiveness in sentiment analysis is yet to be thoroughly assessed. A comparative analysis is performed in a text sentiment classification task where baseline models are compared with and without the use of attention for every experiment. The experimental study additionally examines the proposed models’ ability in recognizing opinions and emotions in movie reviews. The results indicate that attention-based models lead to great improvements in the performance of deep neural models showcasing up to a 3.5% improvement in their accuracy.

The Dynamics of Discrete-Time Computation, with Application to Recurrent Neural Networks and Finite State Machine Extraction

1996 ◽  
Vol 8 (6) ◽  
pp. 1135-1178 ◽  
Author(s):  
Mike Casey
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Finite State Machine ◽  
State Machine ◽  
Training Methods ◽  
Precise Description ◽  
Activation Dynamics ◽  
Finite State ◽  
Successful Approach ◽  
Space Dynamics

Recurrent neural networks (RNNs) can learn to perform finite state computations. It is shown that an RNN performing a finite state computation must organize its state space to mimic the states in the minimal deterministic finite state machine that can perform that computation, and a precise description of the attractor structure of such systems is given. This knowledge effectively predicts activation space dynamics, which allows one to understand RNN computation dynamics in spite of complexity in activation dynamics. This theory provides a theoretical framework for understanding finite state machine (FSM) extraction techniques and can be used to improve training methods for RNNs performing FSM computations. This provides an example of a successful approach to understanding a general class of complex systems that has not been explicitly designed, e.g., systems that have evolved or learned their internal structure.

Sign in / Sign up

Export Citation Format

Share Document