scholarly journals Hyper-parameter optimization of convolutional neural network based on particle swarm optimization algorithm

2021 ◽  
Vol 10 (6) ◽  
pp. 3377-3384
Author(s):  
Zainab Fouad ◽  
Marco Alfonse ◽  
Mohamed Roushdy ◽  
Abdel-Badeeh M. Salem
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Particle Swarm Optimization ◽  
Convolutional Neural Network ◽  
Deep Neural Networks ◽  
Particle Swarm ◽  
Model Performance ◽  
Pso Algorithm ◽  
Swarm Optimization ◽  
Testing Error

Deep neural networks have accomplished enormous progress in tackling many problems. More specifically, convolutional neural network (CNN) is a category of deep networks that have been a dominant technique in computer vision tasks. Despite that these deep neural networks are highly effective; the ideal structure is still an issue that needs a lot of investigation. Deep Convolutional Neural Network model is usually designed manually by trials and repeated tests which enormously constrain its application. Many hyper-parameters of the CNN can affect the model performance. These parameters are depth of the network, numbers of convolutional layers, and numbers of kernels with their sizes. Therefore, it may be a huge challenge to design an appropriate CNN model that uses optimized hyper-parameters and reduces the reliance on manual involvement and domain expertise. In this paper, a design architecture method for CNNs is proposed by utilization of particle swarm optimization (PSO) algorithm to learn the optimal CNN hyper-parameters values. In the experiment, we used Modified National Institute of Standards and Technology (MNIST) database of handwritten digit recognition. The experiments showed that our proposed approach can find an architecture that is competitive to the state-of-the-art models with a testing error of 0.87%.

scholarly journals Position Control of Ultrasonic Motor Using IMC-PID Combined with Tribes Type NN Algorithm

2011 ◽  
Vol 2-3 ◽  
pp. 12-17
Author(s):  
Sheng Lin Mu ◽  
Kanya Tanaka
Keyword(s):  
Neural Network ◽  
Particle Swarm Optimization ◽  
Pid Control ◽  
Position Control ◽  
Back Propagation ◽  
Particle Swarm ◽  
Pso Algorithm ◽  
Ultrasonic Motor ◽  
Swarm Optimization

In this paper, we propose a novel scheme of IMC-PID control combined with a tribes type neural network (NN) for the position control of ultrasonic motor (USM). In this method, the NN controller is employed for tuning the parameter in IMC-PID control. The weights of NN are designed to be updated by the tribes-particle swarm optimization (PSO) algorithm. This method makes it possible to compensate for the characteristic changes and nonlinearity of USM. The parameter-free tribes-PSO requires no information about the USM beforehand; hence its application overcomes the problem of Jacobian estimation in the conventional back propagation (BP) method of NN. The effectiveness of the proposed method is confirmed by experiments.

WNN Model Based on Particle Swarm Optimization for Fault Diagnosis in Analog Circuit

2013 ◽  
Vol 427-429 ◽  
pp. 1048-1051
Author(s):  
Xu Sheng Gan ◽  
Hao Lin Cui ◽  
Ya Rong Wu
Keyword(s):  
Neural Network ◽  
Particle Swarm Optimization ◽  
Fault Diagnosis ◽  
Convergence Rate ◽  
Analog Circuit ◽  
Particle Swarm ◽  
Pso Algorithm ◽  
Wavelet Neural Network ◽  
Swarm Optimization ◽  
Fast Convergence Rate

In order to diagnose the fault in analog circuit correctly, a Wavelet Neural Network (WNN) method is proposed that uses the Particle Swarm Optimization (PSO) algorithm to optimize the network parameters. For the improvement of convergence rate in WNN based on PSO algorithm, a compressing method in research space is introduced into the traditional PSO algorithm to improve the convergence in WNN training. The simulation shows that the proposed method has a good diagnosis with fast convergence rate for the fault in analog circuit.

Using Particle Swarm Optimization Algorithm as an Optimization Tool Within Developed Neural Networks

2018 ◽  
pp. 215-244
Author(s):  
Goran Klepac
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Particle Swarm Optimization ◽  
Evolutionary Algorithms ◽  
Optimization Algorithm ◽  
Recommendation System ◽  
Particle Swarm Optimization Algorithm ◽  
Particle Swarm ◽  
Optimal Combination ◽  
Swarm Optimization

Developed neural networks as an output could have numerous potential outputs caused by numerous combinations of input values. When we are in position to find optimal combination of input values for achieving specific output value within neural network model it is not a trivial task. This request comes from profiling purposes if, for example, neural network gives information of specific profile regarding input or recommendation system realized by neural networks, etc. Utilizing evolutionary algorithms like particle swarm optimization algorithm, which will be illustrated in this chapter, can solve these problems.

scholarly journals Novel Ensemble Approach of Deep Learning Neural Network (DLNN) Model and Particle Swarm Optimization (PSO) Algorithm for Prediction of Gully Erosion Susceptibility

Sensors ◽  
2020 ◽  
Vol 20 (19) ◽  
pp. 5609 ◽  
Author(s):  
Shahab S. Band ◽  
Saeid Janizadeh ◽  
Subodh Chandra Pal ◽  
Asish Saha ◽  
Rabin Chakrabortty ◽  
...  
Keyword(s):  
Neural Network ◽  
Particle Swarm Optimization ◽  
Deep Learning ◽  
Particle Swarm ◽  
Pso Algorithm ◽  
Gully Erosion ◽  
Slope Aspect ◽  
Swarm Optimization ◽  
Proposed Model ◽  
Deep Learning Neural Network

This study aims to evaluate a new approach in modeling gully erosion susceptibility (GES) based on a deep learning neural network (DLNN) model and an ensemble particle swarm optimization (PSO) algorithm with DLNN (PSO-DLNN), comparing these approaches with common artificial neural network (ANN) and support vector machine (SVM) models in Shirahan watershed, Iran. For this purpose, 13 independent variables affecting GES in the study area, namely, altitude, slope, aspect, plan curvature, profile curvature, drainage density, distance from a river, land use, soil, lithology, rainfall, stream power index (SPI), and topographic wetness index (TWI), were prepared. A total of 132 gully erosion locations were identified during field visits. To implement the proposed model, the dataset was divided into the two categories of training (70%) and testing (30%). The results indicate that the area under the curve (AUC) value from receiver operating characteristic (ROC) considering the testing datasets of PSO-DLNN is 0.89, which indicates superb accuracy. The rest of the models are associated with optimal accuracy and have similar results to the PSO-DLNN model; the AUC values from ROC of DLNN, SVM, and ANN for the testing datasets are 0.87, 0.85, and 0.84, respectively. The efficiency of the proposed model in terms of prediction of GES was increased. Therefore, it can be concluded that the DLNN model and its ensemble with the PSO algorithm can be used as a novel and practical method to predict gully erosion susceptibility, which can help planners and managers to manage and reduce the risk of this phenomenon.

Monopole-Gear Design Based on Neural Network and Modified Particle Swarm Optimization

2013 ◽  
Vol 477-478 ◽  
pp. 368-373 ◽  
Author(s):  
Hai Rong Fang
Keyword(s):  
Neural Network ◽  
Particle Swarm Optimization ◽  
Simulated Annealing Algorithm ◽  
Particle Swarm ◽  
Pso Algorithm ◽  
Test Results ◽  
Swarm Optimization ◽  
Modified Particle Swarm Optimization ◽  
Gear Design ◽  
Annealing Algorithm

In order to raise the design efficiency and get the most excellent design effect, this paper combined Particle Swarm Optimization (PSO) algorithm and put forward a new kind of neural network, which based on PSO algorithm, and the implementing framework of PSO and NARMA model. It gives the basic theory, steps and algorithm; The test results show that rapid global convergence and reached the lesser mean square error MSE) when compared with Genetic Algorithm, Simulated Annealing Algorithm, the BP algorithm with momentum term.

scholarly journals A Hybrid Deep Neural Network for Electricity Theft Detection Using Intelligent Antenna-Based Smart Meters

2021 ◽  
Vol 2021 ◽  
pp. 1-19
Author(s):  
Ashraf Ullah ◽  
Nadeem Javaid ◽  
Adamu Sani Yahaya ◽  
Tanzeela Sultana ◽  
Fahad Ahmad Al-Zahrani ◽  
...  
Keyword(s):  
Neural Network ◽  
Particle Swarm Optimization ◽  
Convolutional Neural Network ◽  
Deep Neural Network ◽  
Particle Swarm ◽  
Smart Meters ◽  
Swarm Optimization ◽  
Electricity Theft ◽  
Unit Model ◽  
Gated Recurrent Unit

This paper presents a hybrid model, named as hybrid deep neural network, which combines convolutional neural network, particle swarm optimization, and gated recurrent unit, termed as convolutional neural network-particle swarm optimization-gated recurrent unit model. The major aims of the model are to perform accurate electricity theft detection and to overcome the issues in the existing models. The issues include overfitting and inability of the models to handle imbalanced data. For this purpose, the electricity consumption data of smart meters is taken from state grid corporation of China. An electric utility company gathers the data from the intelligent antenna-based smart meters installed at the consumers’ end. The dataset contains real-time data with missing values and outliers. Therefore, it is first preprocessed to get the refined data followed by feature engineering for selection and extraction of the finest features from the dataset using convolutional neural network. The classification of electricity consumers is performed by dividing them into honest and fraudulent classes using the proposed particle swarm optimization-gated recurrent unit model. The proposed model is evaluated by performing simulations in terms of several performance measures that include accuracy, area under the curve, F 1 -score, recall, and precision. The comparison between the proposed hybrid deep neural network and benchmark models is also performed. The benchmark models include gated recurrent unit, long short term memory, logistic regression, support vector machine, and genetic algorithm-based gated recurrent unit. The results indicate that the proposed hybrid deep neural network model is more efficient in handling class imbalanced issues and performing electricity theft detection. The robustness, accuracy, and generalization of the model are also analyzed in the proposed work.

Training Artificial Neural Networks With Improved Particle Swarm Optimization

2020 ◽  
pp. 63-77
Author(s):  
Kuruge Darshana Abeyrathna ◽  
Chawalit Jeenanunta
Keyword(s):  
Neural Networks ◽  
Particle Swarm Optimization ◽  
Artificial Neural Networks ◽  
Local Minimum ◽  
Optimization Problems ◽  
Particle Swarm ◽  
Pso Algorithm ◽  
Swarm Optimization ◽  
Extra Step ◽  
Artificial Neural

Particle Swarm Optimization (PSO) is popular for solving complex optimization problems. However, it easily traps in local minima. Authors modify the traditional PSO algorithm by adding an extra step called PSO-Shock. The PSO-Shock algorithm initiates similar to the PSO algorithm. Once it traps in a local minimum, it is detected by counting stall generations. When stall generation accumulates to a prespecified value, particles are perturbed. This helps particles to find better solutions than the current local minimum they found. The behavior of PSO-Shock algorithm is studied using a known: Schwefel's function. With promising performance on the Schwefel's function, PSO-Shock algorithm is utilized to optimize the weights and bias of Artificial Neural Networks (ANNs). The trained ANNs then forecast electricity consumption in Thailand. The proposed algorithm reduces the forecasting error compared to the traditional training algorithms. The percentage reduction of error is 23.81% compared to the Backpropagation algorithm and 16.50% compared to the traditional PSO algorithm.

Sign in / Sign up

Export Citation Format

Share Document