scholarly journals Optical Recognition of Handwritten Logic Formulas Using Neural Networks

Electronics ◽  
2021 ◽  
Vol 10 (22) ◽  
pp. 2761
Author(s):  
Vaios Ampelakiotis ◽  
Isidoros Perikos ◽  
Ioannis Hatzilygeroudis ◽  
George Tsihrintzis
Keyword(s):  
Neural Networks ◽  
Character Recognition ◽  
Gradient Descent ◽  
Feedforward Neural Networks ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Training Algorithms ◽  
Gradient Descent Algorithm ◽  
Two Stages ◽  
And Training

In this paper, we present a handwritten character recognition (HCR) system that aims to recognize first-order logic handwritten formulas and create editable text files of the recognized formulas. Dense feedforward neural networks (NNs) are utilized, and their performance is examined under various training conditions and methods. More specifically, after three training algorithms (backpropagation, resilient propagation and stochastic gradient descent) had been tested, we created and trained an NN with the stochastic gradient descent algorithm, optimized by the Adam update rule, which was proved to be the best, using a trainset of 16,750 handwritten image samples of 28 × 28 each and a testset of 7947 samples. The final accuracy achieved is 90.13%. The general methodology followed consists of two stages: the image processing and the NN design and training. Finally, an application has been created that implements the methodology and automatically recognizes handwritten logic formulas. An interesting feature of the application is that it allows for creating new, user-oriented training sets and parameter settings, and thus new NN models.

Training Feedforward Neural Networks with Gain Constraints

2000 ◽  
Vol 12 (4) ◽  
pp. 811-829 ◽  
Author(s):  
Eric Hartman
Keyword(s):  
Neural Networks ◽  
Objective Function ◽  
Network Models ◽  
Feedforward Neural Networks ◽  
Bound Constraints ◽  
Neural Network Models ◽  
Optimization Control ◽  
Input Variables ◽  
Derivatives Of ◽  
And Training

Inaccurate input-output gains (partial derivatives of outputs with respect to inputs) are common in neural network models when input variables are correlated or when data are incomplete or inaccurate. Accurate gains are essential for optimization, control, and other purposes. We develop and explore a method for training feedforward neural networks subject to inequality or equality-bound constraints on the gains of the learned mapping. Gain constraints are implemented as penalty terms added to the objective function, and training is done using gradient descent. Adaptive and robust procedures are devised for balancing the relative strengths of the various terms in the objective function, which is essential when the constraints are inconsistent with the data. The approach has the virtue that the model domain of validity can be extended via extrapolation training, which can dramatically improve generalization. The algorithm is demonstrated here on artificial and real-world problems with very good results and has been advantageously applied to dozens of models currently in commercial use.

scholarly journals Training Neural Network Elements Created From Long Shot Term Memory

2017 ◽  
Vol 10 (1) ◽  
pp. 01-10
Author(s):  
Kostantin Nikolic
Keyword(s):  
Neural Networks ◽  
Feedforward Neural Networks ◽  
Stochastic Search ◽  
Training Algorithms ◽  
Recurrent Networks ◽  
Training Process ◽  
Term Memory ◽  
And Training ◽  
Training Neural Network

This paper presents the application of stochastic search algorithms to train artificial neural networks. Methodology approaches in the work created primarily to provide training complex recurrent neural networks. It is known that training recurrent networks is more complex than the type of training feedforward neural networks. Through simulation of recurrent networks is realized propagation signal from input to output and training process achieves a stochastic search in the space of parameters. The performance of this type of algorithm is superior to most of the training algorithms, which are based on the concept of gradient. The efficiency of these algorithms is demonstrated in the training network created from units that are characterized by long term and long shot term memory of networks. The presented methology is effective and relative simple.

scholarly journals Statistical modeling with neural nets: nuclear masses and halflives

2020 ◽  
Vol 9 ◽  
pp. 266
Author(s):  
E. Mavrommatis ◽  
S. Athanassopoulos ◽  
A. Dakos ◽  
K. A. Gernoth ◽  
J. W. Clark
Keyword(s):  
Neural Networks ◽  
Feedforward Neural Networks ◽  
Neural Nets ◽  
Theoretical Physics ◽  
Global Modeling ◽  
Nuclear Masses ◽  
Region Of Stability ◽  
Statistical Approaches ◽  
Training Schemes ◽  
And Training

Multilayer feedforward neural networks are used to create global models of atomic masses and lifetimes of nuclear states, with the goal of effective prediction of the properties of nuclides outside the region of stability. Innovations in coding and training schemes are used to improve the extrapolation capability of models of the mass table. Studies of nuclear lifetimes have focused on ground states that decay 100% via the β- mode. Results are described which demonstrate that in predictive acuity, statistical approaches to global modeling based on neural networks are potentially competitive with the best phenomenological models based on the traditional methods of theoretical physics.

scholarly journals Speech Emotion Recognition Using Deep Learning LSTM for Tamil Language

2021 ◽  
Vol 29 (3) ◽  
Author(s):  
Bennilo Fernandes ◽  
Kasiprasad Mannepalli
Keyword(s):  
Neural Networks ◽  
Speech Processing ◽  
Performance Test ◽  
Short Term Memory ◽  
Feedforward Neural Networks ◽  
Speech Emotion Recognition ◽  
Training Time ◽  
Hierarchical Learning ◽  
And Training

Deep Neural Networks (DNN) are more than just neural networks with several hidden units that gives better results with classification algorithm in automated voice recognition activities. Then spatial correlation was considered in traditional feedforward neural networks and which do not manage speech signal properly to it extend, so recurrent neural networks (RNNs) were implemented. Long Short-Term Memory (LSTM) systems is a unique case of RNNs for speech processing, thus considering long-term dependencies Deep Hierarchical LSTM and BiLSTM is designed with dropout layers to reduce the gradient and long-term learning error in emotional speech analysis. Thus, four different combinations of deep hierarchical learning architecture Deep Hierarchical LSTM and LSTM (DHLL), Deep Hierarchical LSTM and BiLSTM (DHLB), Deep Hierarchical BiLSTM and LSTM (DHBL) and Deep Hierarchical dual BiLSTM (DHBB) is designed with dropout layers to improve the networks. The performance test of all four model were compared in this paper and better efficiency of classification is attained with minimal dataset of Tamil Language. The experimental results show that DHLB reaches the best precision of about 84% in recognition of emotions for Tamil database, however, the DHBL gives 83% of efficiency. Other design layers also show equal performance but less than the above models DHLL & DHBB shows 81% of efficiency for lesser dataset and minimal execution and training time.

The Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks

1990 ◽  
Vol 2 (2) ◽  
pp. 198-209 ◽  
Author(s):  
Marcus Frean
Keyword(s):  
Neural Networks ◽  
Feedforward Neural Networks ◽  
Multilayer Perceptrons ◽  
Training Set ◽  
Binary Variables ◽  
Perceptron Algorithm ◽  
Linear Threshold ◽  
Recursive Rule ◽  
And Training ◽  
General Method

A general method for building and training multilayer perceptrons composed of linear threshold units is proposed. A simple recursive rule is used to build the structure of the network by adding units as they are needed, while a modified perceptron algorithm is used to learn the connection strengths. Convergence to zero errors is guaranteed for any boolean classification on patterns of binary variables. Simulations suggest that this method is efficient in terms of the numbers of units constructed, and the networks it builds can generalize over patterns not in the training set.

scholarly journals Automatic Fault Detection for Deep Learning Programs Using Graph Transformations

2022 ◽  
Vol 31 (1) ◽  
pp. 1-27
Author(s):  
Amin Nikanjam ◽  
Houssem Ben Braiek ◽  
Mohammad Mehdi Morovati ◽  
Foutse Khomh
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Fault Detection ◽  
Real World ◽  
Feedforward Neural Networks ◽  
Training Dataset ◽  
Meta Model ◽  
Detection Techniques ◽  
Graph Transformations ◽  
Learning Programs

Nowadays, we are witnessing an increasing demand in both corporates and academia for exploiting Deep Learning ( DL ) to solve complex real-world problems. A DL program encodes the network structure of a desirable DL model and the process by which the model learns from the training dataset. Like any software, a DL program can be faulty, which implies substantial challenges of software quality assurance, especially in safety-critical domains. It is therefore crucial to equip DL development teams with efficient fault detection techniques and tools. In this article, we propose NeuraLint , a model-based fault detection approach for DL programs, using meta-modeling and graph transformations. First, we design a meta-model for DL programs that includes their base skeleton and fundamental properties. Then, we construct a graph-based verification process that covers 23 rules defined on top of the meta-model and implemented as graph transformations to detect faults and design inefficiencies in the generated models (i.e., instances of the meta-model). First, the proposed approach is evaluated by finding faults and design inefficiencies in 28 synthesized examples built from common problems reported in the literature. Then NeuraLint successfully finds 64 faults and design inefficiencies in 34 real-world DL programs extracted from Stack Overflow posts and GitHub repositories. The results show that NeuraLint effectively detects faults and design issues in both synthesized and real-world examples with a recall of 70.5% and a precision of 100%. Although the proposed meta-model is designed for feedforward neural networks, it can be extended to support other neural network architectures such as recurrent neural networks. Researchers can also expand our set of verification rules to cover more types of issues in DL programs.

Sign in / Sign up

Export Citation Format

Share Document