Comparison of Different Neural Network Training Algorithms for Wind Velocity Forecasting

2016 ◽  
Vol 819 ◽  
pp. 346-350
Author(s):  
Morteza Khalaji Assadi ◽  
Shervin Safaei
Keyword(s):  
Wind Speed ◽  
Wind Velocity ◽  
Input Data ◽  
Back Propagation ◽  
Activation Function ◽  
Gradient Algorithm ◽  
Training Algorithms ◽  
Use Of Data ◽  
Network Training ◽  
Quasi Newton

In this paper the wind speed is predicted by the use of data provided from the Mehrabad meteorological station located in Tehran, Iran, Collected between 2003 and 2008. A comprehensive analogy study is presented on Comparison of various Back Propagation neural networks methods in wind velocity forecasting. Four types of activation functions, namely, BFGS quasi-Newton, Bayesian regularized, Levenberg-Marquardt, and conjugate gradient algorithm, werestudied. The data was investigated by correlation coefficient and characterizing the amount of dependency between the wind speed and other input data. The meteorological parameters (pressure, direction, temperature and humidity) were used as input data, while the wind velocity is used as the output of the network.The results demonstrate that for the similar wind dataset, Bayesian Regularized algorithm can accurately predict compared with other method. In addition, choosing the type of activation function is dependent on the amount of input data, which should be acceptably large.

scholarly journals Using Feed Forward Neural Network to Solve Eigenvalue Problems

2014 ◽  
Vol 2014 ◽  
pp. 1-8
Author(s):  
Luma N. M. Tawfiq ◽  
Othman M. Salih
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Eigenvalue Problems ◽  
Back Propagation ◽  
Training Algorithms ◽  
Parallel Processor ◽  
Feed Forward Neural Network ◽  
Levenberg Marquardt ◽  
Quasi Newton

The aim of this paper is to presents a parallel processor technique for solving eigenvalue problem for ordinary differential equations using artificial neural networks. The proposed network is trained by back propagation with different training algorithms quasi-Newton, Levenberg-Marquardt, and Bayesian Regulation. The next objective of this paper was to compare the performance of aforementioned algorithms with regard to predicting ability.

scholarly journals Short-term wind speed prediction based on MLP and NARX network models

2020 ◽  
Vol 18 (1) ◽  
pp. 150
Author(s):  
Yousra Amellas ◽  
Outman El Bakkali ◽  
Abdelouahed Djebli ◽  
Adil Echchelh
Keyword(s):  
Wind Speed ◽  
Back Propagation ◽  
Network Models ◽  
Activation Function ◽  
Back Propagation Algorithm ◽  
Neuron Network ◽  
Short Term ◽  
Wind Speed Prediction ◽  
Single Output ◽  
Speed Prediction

The article aims to predict the wind speed by two artificial neural network’s models. The first model is a multilayer perceptron (MLP) treated by back-propagation algorithm and the second one is a recurrent neuron network type, processed by the NARX model. The two models having the same Network’s structure, which they are composed by 4 Inputs layers (Wind Speed, Pressure Temperature and Humidity), an intermediate layer defined by 20 neurons and an activation function, as well as a single output layer characterized by wind speed and a linear function. NARX shows the best results with a regression coefficient R = 0.984 et RMSE = 0.314.

scholarly journals Solving a System of Singular Perturbation Problems via Neural Networks

2020 ◽  
Vol 55 (1) ◽  
Author(s):  
Rana Taleb Shwayaa ◽  
Manar Joundy Hazar ◽  
Mustafa Radif ◽  
Jmal Aldeen Dahi
Keyword(s):  
Neural Networks ◽  
Singular Perturbation ◽  
New Technology ◽  
Back Propagation ◽  
Singular Perturbation Problems ◽  
Training Algorithms ◽  
The Neural Networks ◽  
Perturbation Problems ◽  
Quasi Newton ◽  
Accuracy Speed

In this work, we will introduce a new procedure to solve a system of singular perturbation problems (SSPPs) via artificial neural networks. The neural networks use the code of back propagation with altered training algorithms such as quasi-Newton, Levenberg-Marquardt, and Bayesian regularization. In our research, we provide examples of two different types of systems, showing the accuracy, speed, resolution, and convergence of the new technology, the effectiveness of using the network techniques for solving this type of equations. The convergence properties of the technique and accuracy of the interpolation technique are considered.

scholarly journals Multi-Target Learning Algorithm for Solar Radiation Components Forecasting Based on the Desired Tilt Angle of a Solar Energy System

2021 ◽  
Vol 20 (4) ◽  
pp. 187-193
Author(s):  
Mohammed Ali Jallal ◽  
Abdessalam El Yassini ◽  
Samira Chabaa ◽  
Abdelouhab Zeroual ◽  
Saida Ibnyaich
Keyword(s):  
Solar Radiation ◽  
Solar Energy ◽  
Learning Algorithm ◽  
Back Propagation ◽  
Energy System ◽  
Gradient Algorithm ◽  
Training Algorithms ◽  
Solar Panels ◽  
Grid Operators ◽  
Time Variables

Solar radiation components (SRC) forecasting with different tilt angles plays a key role for planning, managing, and controlling the solar energy system production. To overcome the gaps related to the intermittence and to the absence of SRC data, an accurate predictive model needs to be established. The main goal of the present work is to develop for solar system engineers and grid operators a precise predictive approach based on multi-target learning algorithm to forecast the hourly SRC measurements that is related to the city of Marrakesh (latitude 31°37′N, longitude 08°01′W, elevation 466m), Morocco, received by different inclined solar panels’ surfaces. For this purpose, eight training algorithms (Resilient back Propagation (Rp), One step secant (OSS), Levenberg-Marquardt (LM) Algorithm, Fletcher-Reeves algorithm (Cgf), Polak-Ribiere algorithm (Cgp), Powell-Beale algorithm (Cgb), gradient descent (Gd) algorithm and scaled conjugate gradient algorithm (Scg)) are tested to optimize the developed approach’s parameters. The forecasting results were performed based on the angle of inclination desired by the operator and some accessible meteorological measurements that are recorded at each hour, comprising time variables. The achieved performance demonstrates the stability and the accuracy of the established approach to estimate the hourly SRC time series compared to several recent literature studies.

scholarly journals Comparison of Neural Network Training Algorithms for Classification of Heart Diseases

2018 ◽  
Vol 7 (4) ◽  
pp. 185 ◽  
Author(s):  
Hesam Karim ◽  
Sharareh R. Niakan ◽  
Reza Safdari
Keyword(s):  
Neural Network ◽  
Heart Disease ◽  
Heart Diseases ◽  
Classification Model ◽  
Training Algorithms ◽  
Neural Network Training ◽  
Training Time ◽  
Network Training ◽  
Quasi Newton

<span lang="EN-US">Heart disease is the first cause of death in different countries. Artificial neural network (ANN) technique can be used to predict or classification patients getting a heart disease. There are different training algorithms for ANN. We compared eight neural network training algorithms for classification of heart disease data from UCI repository containing 303 samples. Performance measures of each algorithm containing the speed of training, the number of epochs, accuracy, and mean square error (MSE) were obtained and analyzed. Our results showed that training time for gradient descent algorithms was longer than other training algorithms (8-10 seconds). In contrast, Quasi-Newton algorithms were faster than others (&lt;=0 second). MSE for all algorithms was between 0.117 and 0.228. While there was a significant association between training algorithms and training time (p&lt;0.05), the number of neurons in hidden layer had not any significant effect on the MSE and/or accuracy of the models (p&gt;0.05). Based on our findings, for development an ANN classification model for heart diseases, it is best to use Quasi-Newton training algorithms because of the best speed and accuracy.</span>

Comparison of an Artificial Neural Network and a Multiple Linear Regression in Predicting the Heat of Combustion of Diesel Fuel Based on Hydrocarbon Groups

2020 ◽  
Vol 71 (6) ◽  
pp. 66-74
Author(s):  
Younis M. Younis ◽  
Salman H. Abbas ◽  
Farqad T. Najim ◽  
Firas Hashim Kamar ◽  
Gheorghe Nechifor
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Diesel Fuel ◽  
Back Propagation ◽  
Second Step ◽  
Heating Value ◽  
Network Training ◽  
Marquardt Algorithm ◽  
Levenberg Marquardt ◽  
Propagation Network

A comparison between artificial neural network (ANN) and multiple linear regression (MLR) models was employed to predict the heat of combustion, and the gross and net heat values, of a diesel fuel engine, based on the chemical composition of the diesel fuel. One hundred and fifty samples of Iraqi diesel provided data from chromatographic analysis. Eight parameters were applied as inputs in order to predict the gross and net heat combustion of the diesel fuel. A trial-and-error method was used to determine the shape of the individual ANN. The results showed that the prediction accuracy of the ANN model was greater than that of the MLR model in predicting the gross heat value. The best neural network for predicting the gross heating value was a back-propagation network (8-8-1), using the Levenberg�Marquardt algorithm for the second step of network training. R = 0.98502 for the test data. In the same way, the best neural network for predicting the net heating value was a back-propagation network (8-5-1), using the Levenberg�Marquardt algorithm for the second step of network training. R = 0.95112 for the test data.

scholarly journals Trigonometric Inference Providing Learning in Deep Neural Networks

2021 ◽  
Vol 11 (15) ◽  
pp. 6704
Author(s):  
Jingyong Cai ◽  
Masashi Takemoto ◽  
Yuming Qiu ◽  
Hironori Nakajo
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
Activation Function ◽  
Trigonometric Approximation ◽  
Model Parameters ◽  
Training Algorithms ◽  
Activation Functions ◽  
Classical Training ◽  
Sum Formula

Despite being heavily used in the training of deep neural networks (DNNs), multipliers are resource-intensive and insufficient in many different scenarios. Previous discoveries have revealed the superiority when activation functions, such as the sigmoid, are calculated by shift-and-add operations, although they fail to remove multiplications in training altogether. In this paper, we propose an innovative approach that can convert all multiplications in the forward and backward inferences of DNNs into shift-and-add operations. Because the model parameters and backpropagated errors of a large DNN model are typically clustered around zero, these values can be approximated by their sine values. Multiplications between the weights and error signals are transferred to multiplications of their sine values, which are replaceable with simpler operations with the help of the product to sum formula. In addition, a rectified sine activation function is utilized for further converting layer inputs into sine values. In this way, the original multiplication-intensive operations can be computed through simple add-and-shift operations. This trigonometric approximation method provides an efficient training and inference alternative for devices with insufficient hardware multipliers. Experimental results demonstrate that this method is able to obtain a performance close to that of classical training algorithms. The approach we propose sheds new light on future hardware customization research for machine learning.

scholarly journals Using Artificial Neural Networks in Predicting the Level of Stress among Military Conscripts

Mathematics ◽  
2021 ◽  
Vol 9 (6) ◽  
pp. 626
Author(s):  
Svajone Bekesiene ◽  
Rasa Smaliukiene ◽  
Ramute Vaicaitiene
Keyword(s):  
Neural Network ◽  
Transfer Functions ◽  
Back Propagation ◽  
Armed Forces ◽  
Activation Function ◽  
Optimal Number ◽  
Ann Model ◽  
Back Propagation Algorithm ◽  
Classification Rate ◽  
Artificial Neural

The present study aims to elucidate the main variables that increase the level of stress at the beginning of military conscription service using an artificial neural network (ANN)-based prediction model. Random sample data were obtained from one battalion of the Lithuanian Armed Forces, and a survey was conducted to generate data for the training and testing of the ANN models. Using nonlinearity in stress research, numerous ANN structures were constructed and verified to limit the optimal number of neurons, hidden layers, and transfer functions. The highest accuracy was obtained by the multilayer perceptron neural network (MLPNN) with a 6-2-2 partition. A standardized rescaling method was used for covariates. For the activation function, the hyperbolic tangent was used with 20 units in one hidden layer as well as the back-propagation algorithm. The best ANN model was determined as the model that showed the smallest cross-entropy error, the correct classification rate, and the area under the ROC curve. These findings show, with high precision, that cohesion in a team and adaptation to military routines are two critical elements that have the greatest impact on the stress level of conscripts.

Optimizing the Learning Process of Feedforward Neural Networks Using Lightning Search Algorithm

2016 ◽  
Vol 25 (06) ◽  
pp. 1650033 ◽  
Author(s):  
Hossam Faris ◽  
Ibrahim Aljarah ◽  
Nailah Al-Madi ◽  
Seyedali Mirjalili
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Optimization Problems ◽  
Search Algorithm ◽  
Optimization Technique ◽  
Back Propagation ◽  
Feedforward Neural Networks ◽  
Training Algorithms ◽  
Local Optima ◽  
Local Solutions

Evolutionary Neural Networks are proven to be beneficial in solving challenging datasets mainly due to the high local optima avoidance. Stochastic operators in such techniques reduce the probability of stagnation in local solutions and assist them to supersede conventional training algorithms such as Back Propagation (BP) and Levenberg-Marquardt (LM). According to the No-Free-Lunch (NFL), however, there is no optimization technique for solving all optimization problems. This means that a Neural Network trained by a new algorithm has the potential to solve a new set of problems or outperform the current techniques in solving existing problems. This motivates our attempts to investigate the efficiency of the recently proposed Evolutionary Algorithm called Lightning Search Algorithm (LSA) in training Neural Network for the first time in the literature. The LSA-based trainer is benchmarked on 16 popular medical diagnosis problems and compared to BP, LM, and 6 other evolutionary trainers. The quantitative and qualitative results show that the LSA algorithm is able to show not only better local solutions avoidance but also faster convergence speed compared to the other algorithms employed. In addition, the statistical test conducted proves that the LSA-based trainer is significantly superior in comparison with the current algorithms on the majority of datasets.

scholarly journals Protein Sequence Classification with Improved Extreme Learning Machine Algorithms

2014 ◽  
Vol 2014 ◽  
pp. 1-12 ◽  
Author(s):  
Jiuwen Cao ◽  
Lianglin Xiong
Keyword(s):  
Extreme Learning Machine ◽  
Protein Sequence ◽  
Protein Sequences ◽  
Activation Function ◽  
Majority Voting ◽  
Training Algorithms ◽  
Sequence Classification ◽  
Protein Sequence Classification ◽  
Learning Machine ◽  
Majority Voting Method

Precisely classifying a protein sequence from a large biological protein sequences database plays an important role for developing competitive pharmacological products. Comparing the unseen sequence with all the identified protein sequences and returning the category index with the highest similarity scored protein, conventional methods are usually time-consuming. Therefore, it is urgent and necessary to build an efficient protein sequence classification system. In this paper, we study the performance of protein sequence classification using SLFNs. The recent efficient extreme learning machine (ELM) and its invariants are utilized as the training algorithms. The optimal pruned ELM is first employed for protein sequence classification in this paper. To further enhance the performance, the ensemble based SLFNs structure is constructed where multiple SLFNs with the same number of hidden nodes and the same activation function are used as ensembles. For each ensemble, the same training algorithm is adopted. The final category index is derived using the majority voting method. Two approaches, namely, the basic ELM and the OP-ELM, are adopted for the ensemble based SLFNs. The performance is analyzed and compared with several existing methods using datasets obtained from the Protein Information Resource center. The experimental results show the priority of the proposed algorithms.

Sign in / Sign up

Export Citation Format

Share Document