scholarly journals Advanced supervised learning in multi-layer perceptrons to the recognition tasks based on correlation indicator

2021 ◽  
Vol 33 (1) ◽  
pp. 33-46
Author(s):  
Nikolay Anatolievich Vershkov ◽  
Mikhail Grigoryevich Babenko ◽  
Viktor Andreevich Kuchukov ◽  
Natalia Nikolaevna Kuchukova
Keyword(s):  
Neural Networks ◽  
Information Transmission ◽  
Error Function ◽  
Feedforward Neural Networks ◽  
Oscillatory System ◽  
Ideal Observer ◽  
Global Extremum ◽  
The Neural Network ◽  
Handwritten Digit ◽  
The Mathematical Model

The article deals with the problem of recognition of handwritten digits using feedforward neural networks (perceptrons) using a correlation indicator. The proposed method is based on the mathematical model of the neural network as an oscillatory system similar to the information transmission system. The article uses theoretical developments of the authors to search for the global extremum of the error function in artificial neural networks. The handwritten digit image is considered as a one-dimensional input discrete signal representing a combination of "perfect digit writing" and noise, which describes the deviation of the input implementation from "perfect writing". The ideal observer criterion (Kotelnikov criterion), which is widely used in information transmission systems and describes the probability of correct recognition of the input signal, is used to form the loss function. In the article is carried out a comparative analysis of the convergence of learning and experimentally obtained sequences on the basis of the correlation indicator and widely used in the tasks of classification of the function CrossEntropyLoss with the use of the optimizer and without it. Based on the experiments carried out, it is concluded that the proposed correlation indicator has an advantage of 2-3 times.

scholarly journals Rain–Induced Flood Prediction for Niger Delta Sub-Region of Nigeria Using Neural Networks

2020 ◽  
Vol 5 (9) ◽  
pp. 1124-1130
Author(s):  
Ledisi Giok Kabari ◽  
Young Claudius Mazi
Keyword(s):  
Neural Network ◽  
Climate Change ◽  
Neural Networks ◽  
Niger Delta ◽  
Feedforward Neural Networks ◽  
Flood Prediction ◽  
The Neural Network ◽  
Significant Damage ◽  
Weather Underground ◽  
Short Term Prediction

Climate change generates so many direct and indirect effects on the environment.  Some of those effects have serious consequences. Rain-induced flooding is one of the direct effects of climate change and its impact on the environment is usually devastating and worrisome. Floods are one of the most commonly occurring disasters and have caused significant damage to life, including agriculture and economy. They are usually caused in areas where there is excessive downpour and poor drainage systems. The study uses Feedforward Multilayer Neural Network to perform short-term prediction of the amount of rainfall flood for the Niger Delta   sub region of Nigeria given previous rainfall data for a specified period of time. The data for training and testing of the Neural Network was sourced from Weather Underground official web site https://www.wunderground.com.  An iterative Methodology was used and implemented in MATLAB. We adopted multi-layer Feedforward Neural Networks. The study accurately predicts the rain-induced flood for the Niger Delta   sub region of Nigeria.

Classification and Prediction with Neural Networks

2011 ◽  
pp. 76-107
Author(s):  
Arnošt Veselý
Keyword(s):  
Neural Networks ◽  
Error Function ◽  
Descent Method ◽  
Cross Entropy ◽  
Gradient Descent Method ◽  
Classification Problems ◽  
Network Training ◽  
The Neural Network ◽  
Gradient Calculation ◽  
Entropy Error

This chapter deals with applications of artificial neural networks in classification and regression problems. Based on theoretical analysis it demonstrates that in classification problems one should use cross-entropy error function rather than the usual sum-of-square error function. Using gradient descent method for finding the minimum of the cross entropy error function, leads to the well-known backpropagation of error scheme of gradient calculation if at the output layer of the neural network the neurons with logistic or softmax output functions are used. The author believes that understanding the underlying theory presented in this chapter will help researchers in medical informatics to choose more suitable network architectures for medical applications and that it helps them to carry out the network training more effectively.

Training Pi-Sigma Network by Online Gradient Algorithm with Penalty for Small Weight Update

2007 ◽  
Vol 19 (12) ◽  
pp. 3356-3368 ◽  
Author(s):  
Yan Xiong ◽  
Wei Wu ◽  
Xidai Kang ◽  
Chao Zhang
Keyword(s):  
Neural Networks ◽  
Numerical Experiments ◽  
Error Function ◽  
Feedforward Neural Networks ◽  
Gradient Algorithm ◽  
Training Method ◽  
Penalty Term ◽  
Slow Convergence ◽  
Output Layer ◽  
Adaptive Penalty

A pi-sigma network is a class of feedforward neural networks with product units in the output layer. An online gradient algorithm is the simplest and most often used training method for feedforward neural networks. But there arises a problem when the online gradient algorithm is used for pi-sigma networks in that the update increment of the weights may become very small, especially early in training, resulting in a very slow convergence. To overcome this difficulty, we introduce an adaptive penalty term into the error function, so as to increase the magnitude of the update increment of the weights when it is too small. This strategy brings about faster convergence as shown by the numerical experiments carried out in this letter.

scholarly journals Modified Neural Network Algorithms for Predicting Trading Signals of Stock Market Indices

2009 ◽  
Vol 2009 ◽  
pp. 1-22 ◽  
Author(s):  
C. D. Tilakaratne ◽  
M. A. Mammadov ◽  
S. A. Morris
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Global Optimization ◽  
Stock Market ◽  
Historical Data ◽  
Error Function ◽  
Feedforward Neural Networks ◽  
Ordinary Least Squares ◽  
Global Optimization Algorithm ◽  
Network Algorithms

The aim of this paper is to present modified neural network algorithms to predict whether it is best to buy, hold, or sell shares (trading signals) of stock market indices. Most commonly used classification techniques are not successful in predicting trading signals when the distribution of the actual trading signals, among these three classes, is imbalanced. The modified network algorithms are based on the structure of feedforward neural networks and a modified Ordinary Least Squares (OLSs) error function. An adjustment relating to the contribution from the historical data used for training the networks and penalisation of incorrectly classified trading signals were accounted for, when modifying the OLS function. A global optimization algorithm was employed to train these networks. These algorithms were employed to predict the trading signals of the Australian All Ordinary Index. The algorithms with the modified error functions introduced by this study produced better predictions.

The Effects of Adding Noise During Backpropagation Training on a Generalization Performance

1996 ◽  
Vol 8 (3) ◽  
pp. 643-674 ◽  
Author(s):  
Guozhong An
Keyword(s):  
Neural Networks ◽  
Objective Function ◽  
Feedforward Neural Networks ◽  
Classification Problem ◽  
Weight Changes ◽  
Generalization Performance ◽  
Input Noise ◽  
The Neural Network ◽  
Network Output ◽  
Noise Limit

We study the effects of adding noise to the inputs, outputs, weight connections, and weight changes of multilayer feedforward neural networks during backpropagation training. We rigorously derive and analyze the objective functions that are minimized by the noise-affected training processes. We show that input noise and weight noise encourage the neural-network output to be a smooth function of the input or its weights, respectively. In the weak-noise limit, noise added to the output of the neural networks only changes the objective function by a constant. Hence, it cannot improve generalization. Input noise introduces penalty terms in the objective function that are related to, but distinct from, those found in the regularization approaches. Simulations have been performed on a regression and a classification problem to further substantiate our analysis. Input noise is found to be effective in improving the generalization performance for both problems. However, weight noise is found to be effective in improving the generalization performance only for the classification problem. Other forms of noise have practically no effect on generalization.

scholarly journals Solving Differential Equations Using Feedforward Neural Networks

2021 ◽  
pp. 385-399
Author(s):  
Wilson Guasti Junior ◽  
Isaac P. Santos
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Differential Equations ◽  
Feedforward Neural Networks ◽  
Activation Functions ◽  
Learning Models ◽  
Python Language ◽  
The Neural Network ◽  
Minimization Methods

Abstract In this work we explore the use of deep learning models based on deep feedforward neural networks to solve ordinary and partial differential equations. The illustration of this methodology is given by solving a variety of initial and boundary value problems. The numerical results, obtained based on different feedforward neural networks structures, activation functions and minimization methods, were compared to each other and to the exact solutions. The neural network was implemented using the Python language, with the Tensorflow library.

Sign in / Sign up

Export Citation Format

Share Document