A Comparison between Seven Heuristic Methods to Estimate the Number of Hidden Layer Neurons in a Multi Layer Perceptron Neural Network

2021 ◽  
Vol 10 (2) ◽  
pp. 955-963
Keyword(s):  
Neural Network ◽  
Heuristic Method ◽  
Heuristic Methods ◽  
Multi Layer Perceptron ◽  
Data Set ◽  
Benchmark Datasets ◽  
Hidden Layer ◽  
Hidden Neurons ◽  
Accuracy And Stability ◽  
Iris Data

Multilayer Perceptron Neural Network (MLPNNs) constructs of input, at least one hidden and output layer. Number of the neurons in the hidden layer affects the NNs performance. It also consider difficult task to overcome. This research, aims to exanimate the performance of seven heuristic methods that have been used to estimate the neurons numbers in the hidden layer. The effectiveness of these methods was verified using a six of benchmark datasets. The number of hidden layer neurons that selected by each heuristic method for every data set was used to train the MLP. The results demonstrate that the number of hidden neurons selected by each method provides different accuracy and stability compared with other methods. The number of neurons selected by Hush method for ine data set was 26 neurons. It’s achieved the best accuracy with 99.90%and lowest accuracy achieved by Sheela method with 67.51% using 4 neurons. Using 22 neurons with 97.97% accuracy Ke, J method received the best result for Ionosphere data set. While the lowest accuracy was 96.95% with 5 neurons achieved by Kayama method.For Iris data set with 8 neurons achieved 97.19 as best accuracy achieved by Hush method. For the same data set the lowest results were 92.33 % using 3 neurons obtained by using Kayama method. For WBC data set 96.40% the best accuracy achieved using Sheela and Kaastra methods using 4 and 7neurons, while Kanellopoulos method achieved the lowest accuracy 94.18% with 7neurons. For Glass dataset, 87.15% was the best obtained accuracy using 18 neurons Hush method and using Wang method 82.27 % with 6 neurons was the lowest accuracy. Finally for PID 75.31% accuracy achieved by Kayama method with 3 neurons, where Kanellopoulos method obtained 72.17% through using 24 neurons.

scholarly journals APPROACH TO THE SYNTHESIS OF NEURAL NETWORK STRUCTURE DURING CLASSIFICATION

2020 ◽  
pp. 20-26
Author(s):  
Avazjon R. Marakhimov ◽  
Kabul K. Khudaybergenov
Keyword(s):  
Neural Network ◽  
Pattern Recognition ◽  
Artificial Neural Network ◽  
Feed Forward Neural Network ◽  
Proposed Model ◽  
Benchmark Datasets ◽  
Neural Network Structure ◽  
Artificial Neural ◽  
Hidden Layer ◽  
Hidden Neurons

Evaluating the number of hidden neurons necessary for solving of pattern recognition and classification tasks is one of the key problems in artificial neural networks. Multilayer perceptron is the most useful artificial neural network to estimate the functional structure in classification. In this paper, we show that artificial neural network with a two hidden layer feed forward neural network with d inputs, d neurons in the first hidden layer, 2d+2 neurons in the second hidden layer, k outputs and with a sigmoidal infinitely differentiable function can solve classification and pattern problems with arbitrary accuracy. This result can be applied to design pattern recognition and classification models with optimal structure in the number of hidden neurons and hidden layers. The experimental results over well-known benchmark datasets show that the convergence and the accuracy of the proposed model of artificial neural network are acceptable. Findings in this paper are experimentally analyzed on four different datasets from machine learning repository.

scholarly journals Numerical Analysis of Modeling Based on Improved Elman Neural Network

2014 ◽  
Vol 2014 ◽  
pp. 1-12
Author(s):  
Shao Jie ◽  
Wang Li ◽  
Zhao WeiSong ◽  
Zhong YaQin ◽  
Reza Malekian
Keyword(s):  
Neural Network ◽  
Memory Effect ◽  
Iteration Step ◽  
Elman Neural Network ◽  
Chebyshev Neural Network ◽  
Class D ◽  
Proposed Model ◽  
Tone Signal ◽  
Hidden Layer ◽  
Hidden Neurons

A modeling based on the improved Elman neural network (IENN) is proposed to analyze the nonlinear circuits with the memory effect. The hidden layer neurons are activated by a group of Chebyshev orthogonal basis functions instead of sigmoid functions in this model. The error curves of the sum of squared error (SSE) varying with the number of hidden neurons and the iteration step are studied to determine the number of the hidden layer neurons. Simulation results of the half-bridge class-D power amplifier (CDPA) with two-tone signal and broadband signals as input have shown that the proposed behavioral modeling can reconstruct the system of CDPAs accurately and depict the memory effect of CDPAs well. Compared with Volterra-Laguerre (VL) model, Chebyshev neural network (CNN) model, and basic Elman neural network (BENN) model, the proposed model has better performance.

A Clustering Method for Pruning Fully Connected Neural Network

2011 ◽  
Vol 204-210 ◽  
pp. 600-603
Author(s):  
Gang Li ◽  
Xing San Qian ◽  
Chun Ming Ye ◽  
Lin Zhao
Keyword(s):  
Neural Network ◽  
Data Mining ◽  
Submersible Pump ◽  
Clustering Method ◽  
Backpropagation Neural Network ◽  
Data Set ◽  
Initial Network ◽  
Sample Data ◽  
Hidden Layer ◽  
Fully Connected

This paper focuses mainly on a clustering method for pruning Fully Connected Backpropagation Neural Network (FCBP). The initial neural network is fully connected, after training with sample data, a clustering method is employed to cluster weights between input to hidden layer and from hidden to output layer, and connections that are relatively unnecessary are deleted, thus the initial network becomes a PCBP (Partially Connected Backpropagation) Neural Network. PCBP can be used in prediction or data mining more efficiently than FCBP. At the end of this paper, An experiment is conducted to illustrate the effects of PCBP using the submersible pump repair data set.

ON THE CLASSIFICATION CAPABILITY OF A DYNAMIC THRESHOLD NEURAL NETWORK

1994 ◽  
Vol 05 (02) ◽  
pp. 103-114
Author(s):  
CHENG-CHIN CHIANG ◽  
HSIN-CHIA FU
Keyword(s):  
Neural Network ◽  
Upper Bound ◽  
Learning Algorithm ◽  
Dynamic Threshold ◽  
Training Set ◽  
New Type ◽  
Hidden Layer ◽  
Hidden Neurons ◽  
The Given ◽  
Classification Capability

This paper proposes a new type of neural network called the Dynamic Threshold Neural Network (DTNN) which is theoretically and experimentally superior to a conventional sigmoidal multilayer neural network in classification capability, Given a training set containing 4k+1 patterns in ℜn, to successfully learn this training set, the upper bound on the number of free parameters for a DTNN is (k+1)(n+2)+2(k +1), while the upper bound for a sigmoidal network is 2k(n+1)+(2k+1). We also derive a learning algorithm for the DTNN in a similar way to the derivation of the backprop learning algorithm. In simulations on learning the Two-Spirals problem, our DTNN with 30 neurons in one hidden layer takes only 3200 epochs on average to successfully learn the whole training set, while the single-hidden-layer feedforward sigmoidal neural networks have never been reported to successfully learn the given training set even though more hidden neurons are used.

Controlling Hidden Layer Capacity Through Lateral Connections

1997 ◽  
Vol 9 (6) ◽  
pp. 1381-1402 ◽  
Author(s):  
Kwabena Agyepong ◽  
Ravi Kothari
Keyword(s):  
Function Approximation ◽  
Learning Task ◽  
Feedforward Neural Network ◽  
Effective Number ◽  
Real World Data ◽  
Data Set ◽  
Approximation Problems ◽  
Free Parameters ◽  
Hidden Layer ◽  
Hidden Neurons

We investigate the effects of including selected lateral interconnections in a feedforward neural network. In a network with one hidden layer consisting of m hidden neurons labeled 1,2… m, hidden neuron j is connected fully to the inputs, the outputs, and hidden neuron j + 1. As a consequence of the lateral connections, each hidden neuron receives two error signals: one from the output layer and one through the lateral interconnection. We show that the use of these lateral interconnections among the hidden-layer neurons facilitates controlled assignment of role and specialization of the hidden-layer neurons. In particular, we show that as training progresses, hidden neurons become progressively specialized—starting from the fringes (i.e., lower and higher numbered hidden neurons, e.g., 1, 2, m — 1 m) and leaving the neurons in the center of the hidden layer (i.e., hidden-layer neurons numbered close to m/2) unspecialized or functionally identical. Consequently, the network behaves like network growing algorithms without the explicit need to add hidden units, and like soft weight sharing due to functionally identical neurons in the center of the hidden layer. Experimental results from one classification and one function approximation problems are presented to illustrate selective specialization of the hidden-layer neurons. In addition, the improved generalization that results from a decrease in the effective number of free parameters is illustrated through a simple function approximation example and with a real-world data set. Besides the reduction in the number of free parameters, the localization of weight sharing may also allow for a method that allows procedural determination for the number of hidden-layer neurons required for a given learning task.

scholarly journals Investigation of the Extrapolation Capability of an Artificial Neural Network Algorithm in Combination With Process Signals in Resistance Spot Welding of Advanced High-Strength Steels

2021 ◽  
Author(s):  
Bassel El-Sari ◽  
Max Biegler ◽  
Michael Rethmeier
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Process Parameters ◽  
Resistance Spot Welding ◽  
Spot Welding ◽  
Predictive Accuracy ◽  
Training Data ◽  
Multi Layer Perceptron ◽  
Data Set ◽  
Artificial Neural

Resistance spot welding is an established joining process in the production of safety-relevant components in the automotive industry. Therefore, a consecutive process monitoring is essential to meet the high-quality requirements. Artificial neural networks can be used to evaluate the process parameters and signals to ensure the individual spot weld quality. The predictive accuracy of such algorithms depends on the provided training data set and the prediction of untrained data is challenging. The aim of this paper is to investigate the extrapolation capability of the multi-layer perceptron model. That means, that the predictive performance of the model will be tested with data that clearly differs from the training data in terms of material and coating composition. Therefore, three multi-layer perceptron regression models were implemented to predict the nugget diameter from process data. The three models were able to predict the trained datasets very well. The models, which were provided with features from the dynamic resistance curve predicted the new dataset better than the model with only process parameters. This study shows the beneficial influence of the process signals on the predictive accuracy and robustness of artificial neural network algorithms. Especially, when predicting a data set from outside of the training space.

scholarly journals Prediction of Area and Production of Groundnut Using Box-Jenkins Arima and Neural Network Approach

2020 ◽  
Author(s):  
S. T. Pavana Kumar ◽  
Ferdinand B. Lyngdoh
Keyword(s):  
Neural Network ◽  
Moving Average ◽  
Arima Model ◽  
Network Models ◽  
Arima Models ◽  
Neural Network Models ◽  
Neural Network Approach ◽  
Data Set ◽  
The Neural Network ◽  
Hidden Layer

Selection of parameters for Auto Regressive Integrated Moving Average (ARIMA) model in the prediction process is one of the most important tasks. In the present study, groundnut data was utlised to decide appropriate p, d, q parameters for ARIMA model for the prediction purpose. Firstly, the models were fit to data without splitting into training and validation/testing sets and evaluated for their efficiency in predicting the area and production of groundnut over the years. Meanwhile, models are compared among other fitted ARIMA models with different p, d, q parameters based on decision criteria’s viz., ME, RMSE, MAPE, AIC, BIC and R-Square. The ARIMA model with parameters p-2 d-1-2, q-1-2 are found adequate in predicting the area as well as production of groundnut. The model ARIMA (2, 2, 2) and ARIMA (2,1,1) predicted the area of groundnut crop with minimum error estimates and residual characteristics (ei). The models were fit into split data i.e., training and test data set, but these models’ prediction power (R-Square) declined during testing. In case of predicting the area, ARIMA (2,2,2) was consistent over the split data but it was not consistent while predicting the production over years. Feed-forward neural networks with single hidden layer were fit to complete, training and split data. The neural network models provided better estimates compared to Box-Jenkins ARIMA models. The data was analysed using R-Studio.

scholarly journals Investigation of the Extrapolation Capability of an Artificial Neural Network Algorithm in Combination with Process Signals in Resistance Spot Welding of Advanced High-Strength Steels

Metals ◽  
2021 ◽  
Vol 11 (11) ◽  
pp. 1874
Author(s):  
Bassel El-Sari ◽  
Max Biegler ◽  
Michael Rethmeier
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Process Parameters ◽  
Resistance Spot Welding ◽  
Spot Welding ◽  
Predictive Accuracy ◽  
Training Data ◽  
Multi Layer Perceptron ◽  
Data Set ◽  
Artificial Neural

Resistance spot welding is an established joining process for the production of safety-relevant components in the automotive industry. Therefore, consecutive process monitoring is essential to meet the high quality requirements. Artificial neural networks can be used to evaluate the process parameters and signals, to ensure individual spot weld quality. The predictive accuracy of such algorithms depends on the provided training data set, and the prediction of untrained data is challenging. The aim of this paper was to investigate the extrapolation capability of a multi-layer perceptron model. That means, the predictive performance of the model was tested with data that clearly differed from the training data in terms of material and coating composition. Therefore, three multi-layer perceptron regression models were implemented to predict the nugget diameter from process data. The three models were able to predict the training datasets very well. The models, which were provided with features from the dynamic resistance curve predicted the new dataset better than the model with only process parameters. This study shows the beneficial influence of process signals on the predictive accuracy and robustness of artificial neural network algorithms. Especially, when predicting a data set from outside of the training space.

scholarly journals PEMODELAN PENGARUH SENTIMEN PUBLIK DI TWITTER TERKAIT POLITIK TERHADAP KURS RUPIAH DAN INDEKS HARGA SAHAM DENGAN PENDEKATAN MACHINE LEARNING

2021 ◽  
Vol 2020 (1) ◽  
pp. 989-999
Author(s):  
Epan Mareza Primahendra ◽  
Budi Yuniarto
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Support Vector Machine ◽  
Big Data ◽  
Least Square ◽  
Support Vector ◽  
Hidden Layer ◽  
Hidden Neurons

Kurs Rupiah dan indeks harga saham (IHS) berpengaruh terhadap perekonomian Indonesia. Pergerakan kurs Rupiah dan IHS dipengaruhi oleh, informasi publik, kondisi sosial, dan politik. Kejadian politik banyak menimbulkan sentimen dari masyarakat. Sentimen tersebut banyak disampaikan melalui media sosial terutama Twitter. Twitter merupakan sumber big data yang jika datanya tidak dimanfaatkan akan menjadi sampah. Pengumpulan data dilakukan pada periode 26 September 2019 - 27 Oktober 2019. Pola jumlah tweets harian yang sesuai dengan pergerakan kurs Rupiah dan IHS mengindikasikan bahwa terdapat hubungan antara sentimen di Twitter terkait situasi politik terhadap kurs Rupiah dan IHS. Penelitian ini menggunakan pendekatan machine learning dengan algoritma Neural Network dan Least Square Support Vector Machine. Penelitian ini bertujuan untuk mengetahui pengaruh sentimen terhadap kurs Rupiah dan IHS sekaligus mengkaji kedua algoritmanya. Hasilnya menjelaskan bahwa model terbaik untuk estimasi IHS yaitu NN dengan 1 hidden layer dan 2 hidden neurons. Modelnya menunjukan bahwa terdapat pengaruh antara sentimen tersebut terhadap IHS karena volatilitas estimasi IHS sudah cukup mengikuti pola pergerakan IHS aktual. Model terbaik untuk estimasi kurs Rupiah yaitu LSSVM. Pola pergerakan estimasi kurs Rupiah cenderung stagnan di atas nilai aktual. Ini mengindikasikan bahwa modelnya masih belum memuaskan dalam mengestimasi pengaruh sentimen publik terhadap kurs Rupiah.

Sign in / Sign up

Export Citation Format

Share Document