scholarly journals Optical Recognition of Handwritten Logic Formulas Using Neural Networks

Electronics ◽  
2021 ◽  
Vol 10 (22) ◽  
pp. 2761
Author(s):  
Vaios Ampelakiotis ◽  
Isidoros Perikos ◽  
Ioannis Hatzilygeroudis ◽  
George Tsihrintzis
Keyword(s):  
Neural Networks ◽  
Character Recognition ◽  
Gradient Descent ◽  
Feedforward Neural Networks ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Training Algorithms ◽  
Gradient Descent Algorithm ◽  
Two Stages ◽  
And Training

In this paper, we present a handwritten character recognition (HCR) system that aims to recognize first-order logic handwritten formulas and create editable text files of the recognized formulas. Dense feedforward neural networks (NNs) are utilized, and their performance is examined under various training conditions and methods. More specifically, after three training algorithms (backpropagation, resilient propagation and stochastic gradient descent) had been tested, we created and trained an NN with the stochastic gradient descent algorithm, optimized by the Adam update rule, which was proved to be the best, using a trainset of 16,750 handwritten image samples of 28 × 28 each and a testset of 7947 samples. The final accuracy achieved is 90.13%. The general methodology followed consists of two stages: the image processing and the NN design and training. Finally, an application has been created that implements the methodology and automatically recognizes handwritten logic formulas. An interesting feature of the application is that it allows for creating new, user-oriented training sets and parameter settings, and thus new NN models.

Training Feedforward Neural Networks with Gain Constraints

2000 ◽  
Vol 12 (4) ◽  
pp. 811-829 ◽  
Author(s):  
Eric Hartman
Keyword(s):  
Neural Networks ◽  
Objective Function ◽  
Network Models ◽  
Feedforward Neural Networks ◽  
Bound Constraints ◽  
Neural Network Models ◽  
Optimization Control ◽  
Input Variables ◽  
Derivatives Of ◽  
And Training

Inaccurate input-output gains (partial derivatives of outputs with respect to inputs) are common in neural network models when input variables are correlated or when data are incomplete or inaccurate. Accurate gains are essential for optimization, control, and other purposes. We develop and explore a method for training feedforward neural networks subject to inequality or equality-bound constraints on the gains of the learned mapping. Gain constraints are implemented as penalty terms added to the objective function, and training is done using gradient descent. Adaptive and robust procedures are devised for balancing the relative strengths of the various terms in the objective function, which is essential when the constraints are inconsistent with the data. The approach has the virtue that the model domain of validity can be extended via extrapolation training, which can dramatically improve generalization. The algorithm is demonstrated here on artificial and real-world problems with very good results and has been advantageously applied to dozens of models currently in commercial use.

scholarly journals Training Neural Network Elements Created From Long Shot Term Memory

2017 ◽  
Vol 10 (1) ◽  
pp. 01-10
Author(s):  
Kostantin Nikolic
Keyword(s):  
Neural Networks ◽  
Feedforward Neural Networks ◽  
Stochastic Search ◽  
Training Algorithms ◽  
Recurrent Networks ◽  
Training Process ◽  
Term Memory ◽  
And Training ◽  
Training Neural Network

This paper presents the application of stochastic search algorithms to train artificial neural networks. Methodology approaches in the work created primarily to provide training complex recurrent neural networks. It is known that training recurrent networks is more complex than the type of training feedforward neural networks. Through simulation of recurrent networks is realized propagation signal from input to output and training process achieves a stochastic search in the space of parameters. The performance of this type of algorithm is superior to most of the training algorithms, which are based on the concept of gradient. The efficiency of these algorithms is demonstrated in the training network created from units that are characterized by long term and long shot term memory of networks. The presented methology is effective and relative simple.

scholarly journals Statistical modeling with neural nets: nuclear masses and halflives

2020 ◽  
Vol 9 ◽  
pp. 266
Author(s):  
E. Mavrommatis ◽  
S. Athanassopoulos ◽  
A. Dakos ◽  
K. A. Gernoth ◽  
J. W. Clark
Keyword(s):  
Neural Networks ◽  
Feedforward Neural Networks ◽  
Neural Nets ◽  
Theoretical Physics ◽  
Global Modeling ◽  
Nuclear Masses ◽  
Region Of Stability ◽  
Statistical Approaches ◽  
Training Schemes ◽  
And Training

Multilayer feedforward neural networks are used to create global models of atomic masses and lifetimes of nuclear states, with the goal of effective prediction of the properties of nuclides outside the region of stability. Innovations in coding and training schemes are used to improve the extrapolation capability of models of the mass table. Studies of nuclear lifetimes have focused on ground states that decay 100% via the β- mode. Results are described which demonstrate that in predictive acuity, statistical approaches to global modeling based on neural networks are potentially competitive with the best phenomenological models based on the traditional methods of theoretical physics.

scholarly journals Speech Emotion Recognition Using Deep Learning LSTM for Tamil Language

2021 ◽  
Vol 29 (3) ◽  
Author(s):  
Bennilo Fernandes ◽  
Kasiprasad Mannepalli
Keyword(s):  
Neural Networks ◽  
Speech Processing ◽  
Performance Test ◽  
Short Term Memory ◽  
Feedforward Neural Networks ◽  
Speech Emotion Recognition ◽  
Training Time ◽  
Hierarchical Learning ◽  
And Training

Deep Neural Networks (DNN) are more than just neural networks with several hidden units that gives better results with classification algorithm in automated voice recognition activities. Then spatial correlation was considered in traditional feedforward neural networks and which do not manage speech signal properly to it extend, so recurrent neural networks (RNNs) were implemented. Long Short-Term Memory (LSTM) systems is a unique case of RNNs for speech processing, thus considering long-term dependencies Deep Hierarchical LSTM and BiLSTM is designed with dropout layers to reduce the gradient and long-term learning error in emotional speech analysis. Thus, four different combinations of deep hierarchical learning architecture Deep Hierarchical LSTM and LSTM (DHLL), Deep Hierarchical LSTM and BiLSTM (DHLB), Deep Hierarchical BiLSTM and LSTM (DHBL) and Deep Hierarchical dual BiLSTM (DHBB) is designed with dropout layers to improve the networks. The performance test of all four model were compared in this paper and better efficiency of classification is attained with minimal dataset of Tamil Language. The experimental results show that DHLB reaches the best precision of about 84% in recognition of emotions for Tamil database, however, the DHBL gives 83% of efficiency. Other design layers also show equal performance but less than the above models DHLL & DHBB shows 81% of efficiency for lesser dataset and minimal execution and training time.

A Self-Adaptive Hybrid Bat Algorithm for Training Feedforward Neural Networks

2021 ◽  
Vol 12 (3) ◽  
pp. 149-171
Author(s):  
Rabab Bousmaha ◽  
Reda Mohamed Hamou ◽  
Abdelmalek Amine
Keyword(s):  
Neural Networks ◽  
Differential Evolution ◽  
Credit Card ◽  
Differential Evolution Algorithm ◽  
Solution Space ◽  
Feedforward Neural Networks ◽  
Classification Error ◽  
Training Methods ◽  
Adaptive Differential Evolution ◽  
Self Adaptive

Training feedforward neural network (FFNN) is a complex task in the supervised learning field. An FFNN trainer aims to find the best set of weights that minimizes classification error. This paper presents a new training method based on hybrid bat optimization with self-adaptive differential evolution to train the feedforward neural networks. The hybrid training algorithm combines bat and the self-adaptive differential evolution algorithm called BAT-SDE. BAT-SDE is used to better search in the solution space, which proves its effectiveness in large space solutions. The performance of the proposed approach was compared with eight evolutionary techniques and the standard momentum backpropagation and adaptive learning rate. The comparison was benchmarked and evaluated using seven bio-medical datasets and one large credit card fraud detection dataset. The results of the comparative study show that BAT-SDE outperformed other training methods in most datasets and can be an alternative to other training methods.

scholarly journals Optimizing connection weights in neural networks using hybrid metaheuristics algorithm

2022 ◽  
Vol 12 (1) ◽  
pp. 0-0
Keyword(s):  
Neural Networks ◽  
Adaptive Learning ◽  
Solution Space ◽  
Feedforward Neural Networks ◽  
Fine Tuning ◽  
Training Methods ◽  
Control Parameters ◽  
Hybrid Metaheuristics ◽  
Hybrid Particle Swarm Optimization ◽  
The Comparative Study

The learning process of artificial neural networks is an important and complex task in the supervised learning field. The main difficulty of training a neural network is the process of fine-tuning the best set of control parameters in terms of weight and bias. This paper presents a new training method based on hybrid particle swarm optimization with Multi-Verse Optimization (PMVO) to train the feedforward neural networks. The hybrid algorithm is utilized to search better in solution space which proves its efficiency in reducing the problems of trapping in local minima. The performance of the proposed approach was compared with five evolutionary techniques and the standard momentum backpropagation and adaptive learning rate. The comparison was benchmarked and evaluated using six bio-medical datasets. The results of the comparative study show that PMVO outperformed other training methods in most datasets and can be an alternative to other training methods.

The Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks

1990 ◽  
Vol 2 (2) ◽  
pp. 198-209 ◽  
Author(s):  
Marcus Frean
Keyword(s):  
Neural Networks ◽  
Feedforward Neural Networks ◽  
Multilayer Perceptrons ◽  
Training Set ◽  
Binary Variables ◽  
Perceptron Algorithm ◽  
Linear Threshold ◽  
Recursive Rule ◽  
And Training ◽  
General Method

A general method for building and training multilayer perceptrons composed of linear threshold units is proposed. A simple recursive rule is used to build the structure of the network by adding units as they are needed, while a modified perceptron algorithm is used to learn the connection strengths. Convergence to zero errors is guaranteed for any boolean classification on patterns of binary variables. Simulations suggest that this method is efficient in terms of the numbers of units constructed, and the networks it builds can generalize over patterns not in the training set.

Sign in / Sign up

Export Citation Format

Share Document