scholarly journals Neural Network Identifiability for a Family of Sigmoidal Nonlinearities

2021 ◽  
Author(s):  
Verner Vlačić ◽  
Helmut Bölcskei
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Uniform Norm ◽  
Feed Forward Neural Network ◽  
The Subject ◽  
Hidden Layer ◽  
Arbitrary Precision ◽  
Necessary And Sufficient ◽  
Fully Connected

AbstractThis paper addresses the following question of neural network identifiability: Does the input–output map realized by a feed-forward neural network with respect to a given nonlinearity uniquely specify the network architecture, weights, and biases? The existing literature on the subject (Sussman in Neural Netw 5(4):589–593, 1992; Albertini et al. in Artificial neural networks for speech and vision, 1993; Fefferman in Rev Mat Iberoam 10(3):507–555, 1994) suggests that the answer should be yes, up to certain symmetries induced by the nonlinearity, and provided that the networks under consideration satisfy certain “genericity conditions.” The results in Sussman (1992) and Albertini et al. (1993) apply to networks with a single hidden layer and in Fefferman (1994) the networks need to be fully connected. In an effort to answer the identifiability question in greater generality, we derive necessary genericity conditions for the identifiability of neural networks of arbitrary depth and connectivity with an arbitrary nonlinearity. Moreover, we construct a family of nonlinearities for which these genericity conditions are minimal, i.e., both necessary and sufficient. This family is large enough to approximate many commonly encountered nonlinearities to within arbitrary precision in the uniform norm.

Fourier Neural Networks and Generalized Single Hidden Layer Networks in Aircraft Engine Fault Diagnostics

2005 ◽  
Vol 128 (4) ◽  
pp. 773-782 ◽  
Author(s):  
H. S. Tan
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Fourier Transforms ◽  
Aircraft Engine ◽  
Fault Classification ◽  
Noise Robustness ◽  
Fault Diagnostics ◽  
Multiple Faults ◽  
Hidden Layer

The conventional approach to neural network-based aircraft engine fault diagnostics has been mainly via multilayer feed-forward systems with sigmoidal hidden neurons trained by back propagation as well as radial basis function networks. In this paper, we explore two novel approaches to the fault-classification problem using (i) Fourier neural networks, which synthesizes the approximation capability of multidimensional Fourier transforms and gradient-descent learning, and (ii) a class of generalized single hidden layer networks (GSLN), which self-structures via Gram-Schmidt orthonormalization. Using a simulation program for the F404 engine, we generate steady-state engine parameters corresponding to a set of combined two-module deficiencies and require various neural networks to classify the multiple faults. We show that, compared to the conventional network architecture, the Fourier neural network exhibits stronger noise robustness and the GSLNs converge at a much superior speed.

Simultaneous Approximations of Polynomials and Derivatives and Their Applications to Neural Networks

2008 ◽  
Vol 20 (11) ◽  
pp. 2757-2791 ◽  
Author(s):  
Yoshifusa Ito
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Uniform Norm ◽  
Degree Of Approximation ◽  
Probability Measures ◽  
Compact Sets ◽  
Network Parameters ◽  
Whole Space ◽  
Hidden Layer ◽  
Approximating Polynomials

We have constructed one-hidden-layer neural networks capable of approximating polynomials and their derivatives simultaneously. Generally, optimizing neural network parameters to be trained at later steps of the BP training is more difficult than optimizing those to be trained at the first step. Taking into account this fact, we suppressed the number of parameters of the former type. We measure degree of approximation in both the uniform norm on compact sets and the Lp-norm on the whole space with respect to probability measures.

scholarly journals Identifikasi Penyakit Diabetes Millitus Menggunakan Jaringan Syaraf Tiruan Dengan Metode Perambatan-Balik (Backpropagation)

2018 ◽  
Author(s):  
Sutedi Sutedi
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Neural Network Architecture ◽  
Accuracy Rate ◽  
Layer 2 ◽  
Input Layer ◽  
Hidden Layer ◽  
Diabetes Melitus

Diabetes Melitus (DM) is dangerous disease that affect many of the variouslayer of work society. This disease is not easy to accurately recognized by thegeneral society. So we need to develop a system that can identify accurately. Systemis built using neural networks with backpropagation methods and the functionactivation sigmoid. Neural network architecture using 8 input layer, 2 output layerand 5 hidden layer. The results show that this methods succesfully clasifies datadiabetics and non diabetics with near 100% accuracy rate.

scholarly journals Estimation of Waste Mobile Phones in the Philippines using Neural Networks

2018 ◽  
Vol 20 (4) ◽  
pp. 767-772
Keyword(s):  
Neural Network ◽  
Mobile Phone ◽  
Mobile Phones ◽  
Network Architecture ◽  
The Philippines ◽  
Strategy Formulation ◽  
Feed Forward Neural Network ◽  
The Neural Network ◽  
Mobile Industry ◽  
Hidden Layer

<p>Waste mobile phone is one of the subgroups of e-waste which is defined as discarded electronic products in the Philippine context. This study estimated current and projected quantities of waste mobile phones in the country using feed forward neural network. The neural network architecture used had three layers: (i) input layer, (ii) hidden layer, and (iii) output layer. Seven input factors were fed to the network: (i) population, (ii) literacy rate, (iii) mobile connections, (iv) mobile subscribers, (v) gross domestic product (GDP), (vi) GDP per capita, and (vii) US dollar to Philippine peso exchange rate. These input factors were selected based on the criteria provided in the study by the Groupe Spéciale Mobile Association (GSMA) Intelligence in 2015 on why the Philippines is an innovation hub in mobile industry and the availability of data from the sources. The structure was designed with five hidden layers which consisted of (i) six neurons for layer 1, (ii) five neurons for layer 2, (iii) four neurons for layer 3, (iv) three neurons for layer 4, and (v) two neurons for layer 5. The neural network was designed to initially calculate the sales of mobile phones before estimating waste mobile phone generation. Visual Gene Developer 1.7 Software was used which achieved a sum of squared error of 0.00001. Estimated values were found to be in good agreement with a calculated accuracy of 99%. This study can be used by policy makers as basis for strategy formulation and as guideline and baseline data for establishing a proper management system. Neural network performed better than the traditional linear extrapolation method for forecasting of data.</p>

Selecting the architecture of a class of back-propagation neural networks used as approximators

1997 ◽  
Vol 11 (1) ◽  
pp. 33-44 ◽  
Author(s):  
William C. Carpenter ◽  
Margery E. Hoffman
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Back Propagation ◽  
Region Of Interest ◽  
Input Output ◽  
Advantages And Disadvantages ◽  
Back Propagation Neural Networks ◽  
Input Layer ◽  
Hidden Layer

AbstractThis paper examines the architecture of back-propagation neural networks used as approximators by addressing the interrelationship between the number of training pairs and the number of input, output, and hidden layer nodes required for a good approximation. It concentrates on nets with an input layer, one hidden layer, and one output layer. It shows that many of the currently proposed schemes for selecting network architecture for such nets are deficient. It demonstrates in numerous examples that overdetermined neural networks tend to give good approximations over a region of interest, while underdetermined networks give approximations which can satisfy the training pairs but may give poor approximations over that region of interest. A scheme is presented that adjusts the number of hidden layer nodes in a neural network so as to give an overdetermined approximation. The advantages and disadvantages of using multiple output nodes are discussed. Guidelines for selecting the number of output nodes are presented.

scholarly journals Comparison of Artificial Neural Network Architecture in Solving Ordinary Differential Equations

2013 ◽  
Vol 2013 ◽  
pp. 1-12 ◽  
Author(s):  
Susmita Mall ◽  
S. Chakraverty
Keyword(s):  
Neural Network ◽  
Differential Equations ◽  
Ordinary Differential Equations ◽  
Network Architecture ◽  
Initial Conditions ◽  
Error Function ◽  
Back Propagation ◽  
Feed Forward Neural Network ◽  
Error Back Propagation ◽  
Hidden Layer

This paper investigates the solution of Ordinary Differential Equations (ODEs) with initial conditions using Regression Based Algorithm (RBA) and compares the results with arbitrary- and regression-based initial weights for different numbers of nodes in hidden layer. Here, we have used feed forward neural network and error back propagation method for minimizing the error function and for the modification of the parameters (weights and biases). Initial weights are taken as combination of random as well as by the proposed regression based model. We present the method for solving a variety of problems and the results are compared. Here, the number of nodes in hidden layer has been fixed according to the degree of polynomial in the regression fitting. For this, the input and output data are fitted first with various degree polynomials using regression analysis and the coefficients involved are taken as initial weights to start with the neural training. Fixing of the hidden nodes depends upon the degree of the polynomial. For the example problems, the analytical results have been compared with neural results with arbitrary and regression based weights with four, five, and six nodes in hidden layer and are found to be in good agreement.

Contributions to Ranking an Ergonomic Workstation, Considering the Human Effort and the Microclimate Parameters, Using Neural Networks

2013 ◽  
Vol 371 ◽  
pp. 812-816 ◽  
Author(s):  
Daniel Constantin Anghel ◽  
Nadia Belu
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Experimental Results ◽  
Work Place ◽  
Feed Forward Neural Network ◽  
Linear Relationships ◽  
The Neural Network ◽  
Manufacture Industry ◽  
Human Effort ◽  
Hidden Layer

The paper presents a method to use a feed forward neural network in order to rank a working place from the manufacture industry. Neural networks excel in gathering difficult non-linear relationships between the inputs and outputs of a system. The neural network is simulated with a simple simulator: SSNN. In this paper, we considered as relevant for a work place ranking, 6 input parameters: temperature, humidity, noise, luminosity, load and frequency. The neural network designed for the study presented in this paper has 6 input neurons, 13 neurons in the hidden layer and 1 neuron in the output layer. We present also some experimental results obtained through simulations.

scholarly journals Particle Swarm Optimization to Obtain Weights in Neural Network

MATEMATIKA ◽  
2019 ◽  
Vol 35 (3) ◽  
Author(s):  
Budi Warsito ◽  
Hasbi Yasin ◽  
Alan Prahutama
Keyword(s):  
Neural Network ◽  
Particle Swarm Optimization ◽  
Network Model ◽  
Neural Network Model ◽  
Network Architecture ◽  
Optimization Technique ◽  
Particle Swarm ◽  
Feed Forward Neural Network ◽  
Swarm Optimization ◽  
Hidden Layer

This research discusses the use of a class of heuristic optimization to obtain the weights in neural network model for time series prediction. In this case, Feed Forward Neural Network (FFNN) was chosen as the class of network architecture. The heuristic algorithm determined to obtain the weights in network was Particle Swarm Optimization (PSO). It is a non-gradient optimization technique. This method was used for optimizing the connection weights of network. The lags used as the input were selected based on the strong relationship with the current. The eight architectures were conducted to improve the accuracy of the neural network model. In each architecture, we repeated the running thirty times to get the statistics of mean and variance. The comparison of the performance of various architectures based on the minimum MSE and the stability of the results is presented in this paper. The optimal number of neurons in hidden layer was determined by these criteria. The proposed procedure was applied in air pollution data, i.e. Solid Particulate Matter (SPM). The results showed that the proposed procedure gave promising results in terms of prediction accuracy. A few neurons in hidden layer are strongly recommended in choosing the optimal architecture.

scholarly journals Neural Networks for Driver Behavior Analysis

Electronics ◽  
2021 ◽  
Vol 10 (3) ◽  
pp. 342
Author(s):  
Fabio Martinelli ◽  
Fiammetta Marulli ◽  
Francesco Mercaldo ◽  
Antonella Santone
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Real Time ◽  
Real World ◽  
Experimental Analysis ◽  
Network Architecture ◽  
Driver Behavior ◽  
Neural Network Architecture ◽  
Real World Data ◽  
Fully Connected

The proliferation of info-entertainment systems in nowadays vehicles has provided a really cheap and easy-to-deploy platform with the ability to gather information about the vehicle under analysis. With the purpose to provide an architecture to increase safety and security in automotive context, in this paper we propose a fully connected neural network architecture considering position-based features aimed to detect in real-time: (i) the driver, (ii) the driving style and (iii) the path. The experimental analysis performed on real-world data shows that the proposed method obtains encouraging results.

scholarly journals Sound Event Detection Using Derivative Features in Deep Neural Networks

2020 ◽  
Vol 10 (14) ◽  
pp. 4911
Author(s):  
Jin-Yeol Kwak ◽  
Yong-Joo Chung
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Event Detection ◽  
Network Architecture ◽  
Deep Neural Networks ◽  
Audio Signal ◽  
Feed Forward Neural Network ◽  
Sound Event ◽  
Audio Data ◽  
Sound Event Detection

We propose using derivative features for sound event detection based on deep neural networks. As input to the networks, we used log-mel-filterbank and its first and second derivative features for each frame of the audio signal. Two deep neural networks were used to evaluate the effectiveness of these derivative features. Specifically, a convolutional recurrent neural network (CRNN) was constructed by combining a convolutional neural network and a recurrent neural networks (RNN) followed by a feed-forward neural network (FNN) acting as a classification layer. In addition, a mean-teacher model based on an attention CRNN was used. Both models had an average pooling layer at the output so that weakly labeled and unlabeled audio data may be used during model training. Under the various training conditions, depending on the neural network architecture and training set, the use of derivative features resulted in a consistent performance improvement by using the derivative features. Experiments on audio data from the Detection and Classification of Acoustic Scenes and Events 2018 and 2019 challenges indicated that a maximum relative improvement of 16.9% was obtained in terms of the F-score.

Sign in / Sign up

Export Citation Format

Share Document