A fast learning algorithm of neural network with tunable activation function

2004 ◽  
Vol 47 (1) ◽  
pp. 126 ◽  
Author(s):  
Yanjun SHEN
Keyword(s):  
Neural Network ◽  
Learning Algorithm ◽  
Activation Function ◽  
Fast Learning

Relating the Slope of the Activation Function and the Learning Rate Within a Recurrent Neural Network

1999 ◽  
Vol 11 (5) ◽  
pp. 1069-1077 ◽  
Author(s):  
Danilo P. Mandic ◽  
Jonathon A. Chambers
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Recurrent Neural Network ◽  
Recurrent Neural Networks ◽  
Degrees Of Freedom ◽  
Learning Algorithm ◽  
Activation Function ◽  
Learning Rate ◽  
Optimization Task ◽  
Nonlinear Activation Function

A relationship between the learning rate η in the learning algorithm, and the slope β in the nonlinear activation function, for a class of recurrent neural networks (RNNs) trained by the real-time recurrent learning algorithm is provided. It is shown that an arbitrary RNN can be obtained via the referent RNN, with some deterministic rules imposed on its weights and the learning rate. Such relationships reduce the number of degrees of freedom when solving the nonlinear optimization task of finding the optimal RNN parameters.

scholarly journals Construction of a far-ultraviolet all-sky map from an incomplete survey: application of a deep learning algorithm

2021 ◽  
Vol 502 (3) ◽  
pp. 3200-3209
Author(s):  
Young-Soo Jo ◽  
Yeon-Ju Choi ◽  
Min-Gi Kim ◽  
Chang-Ho Woo ◽  
Kyoung-Wook Min ◽  
...  
Keyword(s):  
Neural Network ◽  
Galaxy Evolution ◽  
Network Architecture ◽  
Learning Algorithm ◽  
Activation Function ◽  
Dense Layer ◽  
Optical Parameters ◽  
Data Set ◽  
Deep Learning Algorithm ◽  
Far Ultraviolet

ABSTRACT We constructed a far-ultraviolet (FUV) all-sky map based on observations from the Far Ultraviolet Imaging Spectrograph (FIMS) aboard the Korean microsatellite Science and Technology SATellite-1. For the ${\sim}20{{\ \rm per\ cent}}$ of the sky not covered by FIMS observations, predictions from a deep artificial neural network were used. Seven data sets were chosen for input parameters, including five all-sky maps of H α, E(B − V), N(H i), and two X-ray bands, with Galactic longitudes and latitudes. 70 ${{\ \rm per\ cent}}$ of the pixels of the observed FIMS data set were randomly selected for training as target parameters and the remaining 30 ${{\ \rm per\ cent}}$ were used for validation. A simple four-layer neural network architecture, which consisted of three convolution layers and a dense layer at the end, was adopted, with an individual activation function for each convolution layer; each convolution layer was followed by a dropout layer. The predicted FUV intensities exhibited good agreement with Galaxy Evolution Explorer observations made in a similar FUV wavelength band for high Galactic latitudes. As a sample application of the constructed map, a dust scattering simulation was conducted with model optical parameters and a Galactic dust model for a region that included observed and predicted pixels. Overall, FUV intensities in the observed and predicted regions were reproduced well.

scholarly journals Stability Analysis Based on Caputo-Type Fractional-Order Quantum Neural Networks

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Yumin Dong ◽  
Xiang Li ◽  
Wei Liao ◽  
Dong Hou
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Learning Algorithm ◽  
Activation Function ◽  
Linear Matrix ◽  
Mixed Delays ◽  
Sigmoid Function ◽  
Matrix Inequality ◽  
Fractional Quantum ◽  
Quantum Neural Network

In this paper, a quantum neural network with multilayer activation function is proposed by using multilayer Sigmoid function superposition and learning algorithm to adjust quantum interval. On this basis, the quasiuniform stability of fractional quantum neural networks with mixed delays is studied. According to the order of two different cases, the conditions of quasi uniform stability of networks are given by using the techniques of linear matrix inequality analysis, and the sufficiency of the conditions is proved. Finally, the feasibility of the conclusion is verified by experiments.

Structure Impact Localization Using Emerging Artificial Intelligence Algorithms

2015 ◽  
pp. 103-123
Author(s):  
Qingsong Xu
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Learning Algorithm ◽  
Experimental Studies ◽  
Back Propagation ◽  
Damage Localization ◽  
Localization Accuracy ◽  
Fast Learning ◽  
Learning Machine ◽  
Hidden Layer

Extreme learning machine (ELM) is a learning algorithm for single-hidden layer feedforward neural networks. In theory, this algorithm is able to provide good generalization capability at extremely fast learning speed. Comparative studies of benchmark function approximation problems revealed that ELM can learn thousands of times faster than conventional neural network (NN) and can produce good generalization performance in most cases. Unfortunately, the research on damage localization using ELM is limited in the literature. In this chapter, the ELM is extended to the domain of damage localization of plate structures. Its effectiveness in comparison with typical neural networks such as back-propagation neural network (BPNN) and least squares support vector machine (LSSVM) is illustrated through experimental studies. Comparative investigations in terms of learning time and localization accuracy are carried out in detail. It is shown that ELM paves a new way in the domain of plate structure health monitoring. Both advantages and disadvantages of using ELM are discussed.

A fast learning algorithm based on extreme learning machine for regular fuzzy neural network

2019 ◽  
Vol 36 (4) ◽  
pp. 3263-3269 ◽  
Author(s):  
Chunmei He ◽  
Yaqi Liu ◽  
Tong Yao ◽  
Fanhua Xu ◽  
Yanyun Hu ◽  
...  
Keyword(s):  
Neural Network ◽  
Extreme Learning Machine ◽  
Fuzzy Neural Network ◽  
Learning Algorithm ◽  
Fast Learning ◽  
Fuzzy Neural ◽  
Learning Machine

An Alternative Fast Learning Algorithm of Neural Network

2009 ◽  
pp. 205-212 ◽  
Author(s):  
Pin-Hsuan Weng ◽  
Chih-Chien Huang ◽  
Yu-Ju Chen ◽  
Huang-Chu Huang ◽  
Rey-Chue Hwang
Keyword(s):  
Neural Network ◽  
Learning Algorithm ◽  
Fast Learning
Sign in / Sign up

Export Citation Format

Share Document