BINARY NEURAL NETWORK TRAINING ALGORITHMS BASED ON LINEAR SEQUENTIAL LEARNING

2003 ◽  
Vol 13 (05) ◽  
pp. 333-351 ◽  
Author(s):  
DI WANG ◽  
NARENDRA S. CHAUDHARI
Keyword(s):  
Neural Network ◽  
Sequential Learning ◽  
Training Algorithms ◽  
Neural Network Training ◽  
Neural Network Learning ◽  
Network Learning ◽  
Linear Separability ◽  
Network Training ◽  
Binary Neural Network ◽  
Hidden Layer

A key problem in Binary Neural Network learning is to decide bigger linear separable subsets. In this paper we prove some lemmas about linear separability. Based on these lemmas, we propose Multi-Core Learning (MCL) and Multi-Core Expand-and-Truncate Learning (MCETL) algorithms to construct Binary Neural Networks. We conclude that MCL and MCETL simplify the equations to compute weights and thresholds, and they result in the construction of simpler hidden layer. Examples are given to demonstrate these conclusions.

scholarly journals A Geometric Perspective on Information Plane Analysis

Entropy ◽  
2021 ◽  
Vol 23 (6) ◽  
pp. 711
Author(s):  
Mina Basirat ◽  
Bernhard C. Geiger ◽  
Peter M. Roth
Keyword(s):  
Neural Network ◽  
Mutual Information ◽  
Geometric Interpretation ◽  
Neural Network Training ◽  
Neural Network Learning ◽  
Network Learning ◽  
Plane Analysis ◽  
Network Training ◽  
Hidden Layer ◽  
The Impact

Information plane analysis, describing the mutual information between the input and a hidden layer and between a hidden layer and the target over time, has recently been proposed to analyze the training of neural networks. Since the activations of a hidden layer are typically continuous-valued, this mutual information cannot be computed analytically and must thus be estimated, resulting in apparently inconsistent or even contradicting results in the literature. The goal of this paper is to demonstrate how information plane analysis can still be a valuable tool for analyzing neural network training. To this end, we complement the prevailing binning estimator for mutual information with a geometric interpretation. With this geometric interpretation in mind, we evaluate the impact of regularization and interpret phenomena such as underfitting and overfitting. In addition, we investigate neural network learning in the presence of noisy data and noisy labels.

Evaluation of Parameter Settings for Training Neural Networks Using Backpropagation Algorithms

2022 ◽  
pp. 202-226
Author(s):  
Leema N. ◽  
Khanna H. Nehemiah ◽  
Elgin Christo V. R. ◽  
Kannan A.
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Activation Function ◽  
Neural Network Training ◽  
Network Parameter ◽  
Network Parameters ◽  
Network Training ◽  
Rate Minimum ◽  
Hidden Layer ◽  
Function Number

Artificial neural networks (ANN) are widely used for classification, and the training algorithm commonly used is the backpropagation (BP) algorithm. The major bottleneck faced in the backpropagation neural network training is in fixing the appropriate values for network parameters. The network parameters are initial weights, biases, activation function, number of hidden layers and the number of neurons per hidden layer, number of training epochs, learning rate, minimum error, and momentum term for the classification task. The objective of this work is to investigate the performance of 12 different BP algorithms with the impact of variations in network parameter values for the neural network training. The algorithms were evaluated with different training and testing samples taken from the three benchmark clinical datasets, namely, Pima Indian Diabetes (PID), Hepatitis, and Wisconsin Breast Cancer (WBC) dataset obtained from the University of California Irvine (UCI) machine learning repository.

Local Learning Algorithms for Sequential Tasks in Neural Networks

1998 ◽  
Vol 2 (6) ◽  
pp. 221-227 ◽  
Author(s):  
Anthony Robins ◽  
◽  
Marcus Frean ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Learning Algorithms ◽  
Learning Task ◽  
Sequential Learning ◽  
Local Learning ◽  
Neural Network Learning ◽  
Network Learning ◽  
Sequential Task ◽  
Global And Local

In this paper, we explore the concept of sequential learning and the efficacy of global and local neural network learning algorithms on a sequential learning task. Pseudorehearsal, a method developed by Robins19) to solve the catastrophic forgetting problem which arises from the excessive plasticity of neural networks, is significantly more effective than other local learning algorithms for the sequential task. We further consider the concept of local learning and suggest that pseudorehearsal is so effective because it works directly at the level of the learned function, and not indirectly on the representation of the function within the network. We also briefly explore the effect of local learning on generalization within the task.

scholarly journals Performance Analysis of Traffic Congestion Using Designated Neural Network Training Algorithms

2021 ◽  
Vol 20 ◽  
pp. 23-33
Author(s):  
Ituabhor Odesanya ◽  
Joseph Femi Odesanya
Keyword(s):  
Neural Network ◽  
Traffic Flow ◽  
Traffic Congestion ◽  
Bayesian Regularization ◽  
Training Algorithms ◽  
Neural Network Training ◽  
Flow Data ◽  
Network Training ◽  
Ondo State ◽  
Coefficient Of Regression

A lot of neural network training algorithms on prediction exist and these algorithms are being used by researchers to solve evaluation, forecasting, clustering, function approximation etc. problems in traffic volume congestion. This study is aimed at analysing the performance of traffic congestion using some designated neural network training algorithms on traffic flow in some selected corridors within Akure, Ondo state, Nigeria. The selected corridors were Oba Adesida road, Oyemekun road and Oke Ijebu road all in Akure. The traffic flow data were collected manually with the help of field observers who monitored and record traffic movement along the corridors. To accomplish this, three common training algorithms were selected to train the traffic flow data. The data were trained using Bayesian Regularization (BR), Scaled Conjugate Gradient (SCG) and Levenberg-Marquardt (LM) training algorithms. The outputs/performances of these training functions were evaluated by using the Mean Square Error (MSE) and Coefficient of Regression (R) to find the best training algorithms. The results show that, the Bayesian regularization algorithm, performs better with MSE of 2.37e-13 and R of 0.9999 than SCG and LM algorithms.

Sign in / Sign up

Export Citation Format

Share Document