scholarly journals Approximation of phenol concentration using novel hybrid computational intelligence methods

2014 ◽  
Vol 24 (1) ◽  
pp. 165-181 ◽  
Author(s):  
Pawel Plawiak ◽  
Ryszard Tadeusiewicz
Keyword(s):  
Neural Networks ◽  
Genetic Algorithms ◽  
Computational Intelligence ◽  
Recurrent Neural Networks ◽  
Phenol Concentration ◽  
Neural Systems ◽  
Feed Forward ◽  
Computational Intelligence Methods ◽  
Lm Algorithm ◽  
Selection Of

Abstract This paper presents two innovative evolutionary-neural systems based on feed-forward and recurrent neural networks used for quantitative analysis. These systems have been applied for approximation of phenol concentration. Their performance was compared against the conventional methods of artificial intelligence (artificial neural networks, fuzzy logic and genetic algorithms). The proposed systems are a combination of data preprocessing methods, genetic algorithms and the Levenberg-Marquardt (LM) algorithm used for learning feed forward and recurrent neural networks. The initial weights and biases of neural networks chosen by the use of a genetic algorithm are then tuned with an LM algorithm. The evaluation is made on the basis of accuracy and complexity criteria. The main advantage of proposed systems is the elimination of random selection of the network weights and biases, resulting in increased efficiency of the systems.

scholarly journals A New Hyper-Parameter Optimization Method for Power Load Forecast Based on Recurrent Neural Networks

Algorithms ◽  
2021 ◽  
Vol 14 (6) ◽  
pp. 163
Author(s):  
Yaru Li ◽  
Yulai Zhang ◽  
Yongping Cai
Keyword(s):  
Neural Networks ◽  
Parameter Optimization ◽  
Recurrent Neural Networks ◽  
Critical Role ◽  
Optimization Method ◽  
Bayesian Optimization ◽  
Power Load ◽  
Gradient Calculation ◽  
Truncated Newton ◽  
Selection Of

The selection of the hyper-parameters plays a critical role in the task of prediction based on the recurrent neural networks (RNN). Traditionally, the hyper-parameters of the machine learning models are selected by simulations as well as human experiences. In recent years, multiple algorithms based on Bayesian optimization (BO) are developed to determine the optimal values of the hyper-parameters. In most of these methods, gradients are required to be calculated. In this work, the particle swarm optimization (PSO) is used under the BO framework to develop a new method for hyper-parameter optimization. The proposed algorithm (BO-PSO) is free of gradient calculation and the particles can be optimized in parallel naturally. So the computational complexity can be effectively reduced which means better hyper-parameters can be obtained under the same amount of calculation. Experiments are done on real world power load data,where the proposed method outperforms the existing state-of-the-art algorithms,BO with limit-BFGS-bound (BO-L-BFGS-B) and BO with truncated-newton (BO-TNC),in terms of the prediction accuracy. The errors of the prediction result in different models show that BO-PSO is an effective hyper-parameter optimization method.

TRAINING RECURRENT NEURAL NETWORKS — THE MINIMAL TRAJECTORY ALGORITHM

1992 ◽  
Vol 03 (01) ◽  
pp. 83-101 ◽  
Author(s):  
D. Saad
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Learning Rule ◽  
Weight Matrix ◽  
Training Methods ◽  
Input Output ◽  
Feed Forward ◽  
End Point ◽  
Common Weight ◽  
Perceptron Learning

The Minimal Trajectory (MINT) algorithm for training recurrent neural networks with a stable end point is based on an algorithmic search for the systems’ representations in the neighbourhood of the minimal trajectory connecting the input-output representations. The said representations appear to be the most probable set for solving the global perceptron problem related to the common weight matrix, connecting all representations of successive time steps in a recurrent discrete neural networks. The search for a proper set of system representations is aided by representation modification rules similar to those presented in our former paper,1 aimed to support contributing hidden and non-end-point representations while supressing non-contributing ones. Similar representation modification rules were used in other training methods for feed-forward networks,2–4 based on modification of the internal representations. A feed-forward version of the MINT algorithm will be presented in another paper.5 Once a proper set of system representations is chosen, the weight matrix is then modified accordingly, via the Perceptron Learning Rule (PLR) to obtain the proper input-output relation. Computer simulations carried out for the restricted cases of parity and teacher-net problems show rapid convergence of the algorithm in comparison with other existing algorithms, together with modest memory requirements.

Sign in / Sign up

Export Citation Format

Share Document