A Matrix Method for Optimizing a Neural Network

1991 ◽  
Vol 3 (3) ◽  
pp. 450-459 ◽  
Author(s):  
Simon A. Barton
Keyword(s):  
Neural Network ◽  
Matrix Method ◽  
Single Layer ◽  
Test Problems ◽  
Functional Representation ◽  
Iterative Minimization ◽  
Minimization Procedure ◽  
Input Side ◽  
Output Side ◽  
Hidden Layer

A matrix method is described that optimizes the set of weights and biases for the output side of a network with a single hidden layer of neurons, given any set of weights and biases for the input side of the hidden layer. All the input patterns are included in a single optimization cycle. A simple iterative minimization procedure is used to optimize the weights and biases on the input side of the hidden layer. Many test problems have been solved, confirming the validity of the method. The results suggest that for a network with a single layer of hidden sigmoidal nodes, the accuracy of a functional representation is reduced as the nonlinearity of the function increases.

Solving Nonlinearly Separable Classifications in a Single-Layer Neural Network

2017 ◽  
Vol 29 (3) ◽  
pp. 861-866 ◽  
Author(s):  
Nolan Conaway ◽  
Kenneth J. Kurtz
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Discriminant Function ◽  
Single Layer ◽  
Classification Problems ◽  
Learning Capabilities ◽  
Separable Problems ◽  
Hidden Layer ◽  
Single Layer Network

Since the work of Minsky and Papert ( 1969 ), it has been understood that single-layer neural networks cannot solve nonlinearly separable classifications (i.e., XOR). We describe and test a novel divergent autoassociative architecture capable of solving nonlinearly separable classifications with a single layer of weights. The proposed network consists of class-specific linear autoassociators. The power of the model comes from treating classification problems as within-class feature prediction rather than directly optimizing a discriminant function. We show unprecedented learning capabilities for a simple, single-layer network (i.e., solving XOR) and demonstrate that the famous limitation in acquiring nonlinearly separable problems is not just about the need for a hidden layer; it is about the choice between directly predicting classes or learning to classify indirectly by predicting features.

scholarly journals Enhanced Error Reduction of Signal Power Loss During Electromagnetic Propagation: Architectural Composition and Learning Rate Selection

2021 ◽  
pp. 450-456
Author(s):  
Virginia C. Ebhota ◽  
◽  
Viranjay M. Srivastava
Keyword(s):  
Neural Network ◽  
Power Loss ◽  
Single Layer ◽  
Signal Propagation ◽  
Learning Rate ◽  
Signal Power ◽  
Multi Layer Perceptron ◽  
Coefficient Of Correlation ◽  
Network Training ◽  
Hidden Layer

This research work analyses the effect of the architectural composition of Multi-Layer Perceptron (MLP) Artificial Neural Network (ANN) combined with the effect of the learning rate for effective prediction of signal power loss during electromagnetic signal propagation. A single hidden layer and two hidden layers of MLP ANN have been considered. Different configurations of the neural network architecture ranging from 4 to 100 for both MLP networks have been analyzed. The required hidden layer neurons for optimal training of a single layer multi-layer network were 40 neurons with 0.99670 coefficient of correlation and 1.28020 standard deviations, while [68 72] trained two hidden layers multi-layer perceptron with 0.98880 coefficient of correlation and standard deviation of 1.42820. Different learning rates were also adopted for the network training. The results further validate better MLP neural network training for signal power loss prediction using single-layer perceptron network compared to two hidden layers perceptron network with the coefficient of correlation of 0.99670 for single-layer network and 0.9888 for two hidden layers network. Furthermore, the learning rate of 0.003 shows the best training capability with lower mean squared error and higher training regression compared to other values of learning rate used for both single layer and two hidden layers perceptron MLP networks.

A New Technique for Scan Chain Failure Diagnosis

2002 ◽  
Author(s):  
Ruifeng Guo ◽  
Srikanth Venkataraman
Keyword(s):  
Hold Time ◽  
Failure Diagnosis ◽  
Second Step ◽  
Fault Type ◽  
Transition Faults ◽  
Input Side ◽  
Output Side ◽  
A New Technique ◽  
Scan Chain ◽  
Diagnostic Resolution

Abstract In this paper, we present a scan chain fault diagnosis procedure. The diagnosis for a single scan chain failure is performed in three steps. The first step uses special chain test patterns to determine both the faulty chain and the fault type in the faulty chain. The second step uses a novel procedure to generate special test patterns to identify the suspect scan cell within a range of scan cells. Unlike previously proposed methods that restrict the location of the faulty scan cell only from the scan chain output side, our method restricts the location of the faulty scan cell from both the scan chain output side and the scan chain input side. Hence the number of suspect scan cells is reduced significantly in this step. The final step further improves the diagnostic resolution by ranking the suspect scan cells inside this range. The proposed technique handles both stuck-at and timing failures (transition faults and hold time faults). The experimental results based on simulation and silicon units for several products show the effectiveness of the proposed method.

Using artificial neural network condensation to facilitate adaption of machine learning in medical settings by reducing computational burden (Preprint)

2020 ◽  
Author(s):  
Dianbo Liu
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Third World ◽  
Mortality Prediction ◽  
Neural Net ◽  
Medical Settings ◽  
Hidden Layer ◽  
Applications Of Machine Learning ◽  
Computational Resources ◽  
Developed Nations

BACKGROUND Applications of machine learning (ML) on health care can have a great impact on people’s lives. At the same time, medical data is usually big, requiring a significant amount of computational resources. Although it might not be a problem for wide-adoption of ML tools in developed nations, availability of computational resource can very well be limited in third-world nations and on mobile devices. This can prevent many people from benefiting of the advancement in ML applications for healthcare. OBJECTIVE In this paper we explored three methods to increase computational efficiency of either recurrent neural net-work(RNN) or feedforward (deep) neural network (DNN) while not compromising its accuracy. We used in-patient mortality prediction as our case analysis upon intensive care dataset. METHODS We reduced the size of RNN and DNN by applying pruning of “unused” neurons. Additionally, we modified the RNN structure by adding a hidden-layer to the RNN cell but reduce the total number of recurrent layers to accomplish a reduction of total parameters in the network. Finally, we implemented quantization on DNN—forcing the weights to be 8-bits instead of 32-bits. RESULTS We found that all methods increased implementation efficiency–including training speed, memory size and inference speed–without reducing the accuracy of mortality prediction. CONCLUSIONS This improvements allow the implementation of sophisticated NN algorithms on devices with lower computational resources.

scholarly journals Rolling Bearing Fault Prediction Method Based on QPSO-BP Neural Network and Dempster–Shafer Evidence Theory

Energies ◽  
2020 ◽  
Vol 13 (5) ◽  
pp. 1094 ◽  
Author(s):  
Lanjun Wan ◽  
Hongyang Li ◽  
Yiwei Chen ◽  
Changyun Li
Keyword(s):  
Neural Network ◽  
Working Conditions ◽  
Bp Neural Network ◽  
Evidence Theory ◽  
Prediction Method ◽  
Rolling Bearing ◽  
Fault Prediction ◽  
Fault Classification ◽  
Bearing Fault ◽  
Hidden Layer

To effectively predict the rolling bearing fault under different working conditions, a rolling bearing fault prediction method based on quantum particle swarm optimization (QPSO) backpropagation (BP) neural network and Dempster–Shafer evidence theory is proposed. First, the original vibration signals of rolling bearing are decomposed by three-layer wavelet packet, and the eigenvectors of different states of rolling bearing are constructed as input data of BP neural network. Second, the optimal number of hidden-layer nodes of BP neural network is automatically found by the dichotomy method to improve the efficiency of selecting the number of hidden-layer nodes. Third, the initial weights and thresholds of BP neural network are optimized by QPSO algorithm, which can improve the convergence speed and classification accuracy of BP neural network. Finally, the fault classification results of multiple QPSO-BP neural networks are fused by Dempster–Shafer evidence theory, and the final rolling bearing fault prediction model is obtained. The experiments demonstrate that different types of rolling bearing fault can be effectively and efficiently predicted under various working conditions.

scholarly journals A Geometric Perspective on Information Plane Analysis

Entropy ◽  
2021 ◽  
Vol 23 (6) ◽  
pp. 711
Author(s):  
Mina Basirat ◽  
Bernhard C. Geiger ◽  
Peter M. Roth
Keyword(s):  
Neural Network ◽  
Mutual Information ◽  
Geometric Interpretation ◽  
Neural Network Training ◽  
Neural Network Learning ◽  
Network Learning ◽  
Plane Analysis ◽  
Network Training ◽  
Hidden Layer ◽  
The Impact

Information plane analysis, describing the mutual information between the input and a hidden layer and between a hidden layer and the target over time, has recently been proposed to analyze the training of neural networks. Since the activations of a hidden layer are typically continuous-valued, this mutual information cannot be computed analytically and must thus be estimated, resulting in apparently inconsistent or even contradicting results in the literature. The goal of this paper is to demonstrate how information plane analysis can still be a valuable tool for analyzing neural network training. To this end, we complement the prevailing binning estimator for mutual information with a geometric interpretation. With this geometric interpretation in mind, we evaluate the impact of regularization and interpret phenomena such as underfitting and overfitting. In addition, we investigate neural network learning in the presence of noisy data and noisy labels.

Breast cancer diagnosis using multiple activation deep neural network

2021 ◽  
pp. 1063293X2110251
Author(s):  
K Vijayakumar ◽  
Vinod J Kadam ◽  
Sudhir Kumar Sharma
Keyword(s):  
Breast Cancer ◽  
Neural Network ◽  
Deep Neural Network ◽  
Breast Cancer Diagnosis ◽  
Activation Function ◽  
Linear Functions ◽  
Cancer Data ◽  
Final Layer ◽  
Improved Performance ◽  
Hidden Layer

Deep Neural Network (DNN) stands for multilayered Neural Network (NN) that is capable of progressively learn the more abstract and composite representations of the raw features of the input data received, with no need for any feature engineering. They are advanced NNs having repetitious hidden layers between the initial input and the final layer. The working principle of such a standard deep classifier is based on a hierarchy formed by the composition of linear functions and a defined nonlinear Activation Function (AF). It remains uncertain (not clear) how the DNN classifier can function so well. But it is clear from many studies that within DNN, the AF choice has a notable impact on the kinetics of training and the success of tasks. In the past few years, different AFs have been formulated. The choice of AF is still an area of active study. Hence, in this study, a novel deep Feed forward NN model with four AFs has been proposed for breast cancer classification: hidden layer 1: Swish, hidden layer, 2:-LeakyReLU, hidden layer 3: ReLU, and final output layer: naturally Sigmoidal. The purpose of the study is twofold. Firstly, this study is a step toward a more profound understanding of DNN with layer-wise different AFs. Secondly, research is also aimed to explore better DNN-based systems to build predictive models for breast cancer data with improved accuracy. Therefore, the benchmark UCI dataset WDBC was used for the validation of the framework and evaluated using a ten-fold CV method and various performance indicators. Multiple simulations and outcomes of the experimentations have shown that the proposed solution performs in a better way than the Sigmoid, ReLU, and LeakyReLU and Swish activation DNN in terms of different parameters. This analysis contributes to producing an expert and precise clinical dataset classification method for breast cancer. Furthermore, the model also achieved improved performance compared to many established state-of-the-art algorithms/models.

Building Electronic Commerce Recommendation System Based on Ontology Learning and BP Neural Network

2013 ◽  
Vol 718-720 ◽  
pp. 1961-1966
Author(s):  
Hong Sheng Xu ◽  
Qing Tan
Keyword(s):  
Neural Network ◽  
Electronic Commerce ◽  
Bp Neural Network ◽  
Optimization Problem ◽  
Recommendation System ◽  
High Efficiency ◽  
Ontology Learning ◽  
The Neural Network ◽  
Network Process ◽  
Hidden Layer

Electronic commerce recommendation system can effectively retain user, prevent users from erosion, and improve e-commerce system sales. BP neural network using iterative operation, solving the weights of the neural network and close values to corresponding network process of learning and memory, to join the hidden layer nodes of the optimization problem of adjustable parameters increase. Ontology learning is the use of machine learning and statistical techniques, with automatic or semi-automatic way, from the existing data resources and obtaining desired body. The paper presents building electronic commerce recommendation system based on ontology learning and BP neural network. Experimental results show that the proposed algorithm has high efficiency.

Sign in / Sign up

Export Citation Format

Share Document