Echo state network activation function based on bistable stochastic resonance

2021 ◽  
Vol 153 ◽  
pp. 111503
Author(s):  
Zhiqiang Liao ◽  
Zeyu Wang ◽  
Hiroyasu Yamahara ◽  
Hitoshi Tabata
2019 ◽  
Vol 12 (3) ◽  
pp. 156-161 ◽  
Author(s):  
Aman Dureja ◽  
Payal Pahwa

Background: In making the deep neural network, activation functions play an important role. But the choice of activation functions also affects the network in term of optimization and to retrieve the better results. Several activation functions have been introduced in machine learning for many practical applications. But which activation function should use at hidden layer of deep neural networks was not identified. Objective: The primary objective of this analysis was to describe which activation function must be used at hidden layers for deep neural networks to solve complex non-linear problems. Methods: The configuration for this comparative model was used by using the datasets of 2 classes (Cat/Dog). The number of Convolutional layer used in this network was 3 and the pooling layer was also introduced after each layer of CNN layer. The total of the dataset was divided into the two parts. The first 8000 images were mainly used for training the network and the next 2000 images were used for testing the network. Results: The experimental comparison was done by analyzing the network by taking different activation functions on each layer of CNN network. The validation error and accuracy on Cat/Dog dataset were analyzed using activation functions (ReLU, Tanh, Selu, PRelu, Elu) at number of hidden layers. Overall the Relu gave best performance with the validation loss at 25th Epoch 0.3912 and validation accuracy at 25th Epoch 0.8320. Conclusion: It is found that a CNN model with ReLU hidden layers (3 hidden layers here) gives best results and improve overall performance better in term of accuracy and speed. These advantages of ReLU in CNN at number of hidden layers are helpful to effectively and fast retrieval of images from the databases.


Author(s):  
Adamu I. Abubakar ◽  
Haruna Chiroma ◽  
Sameem Abdulkareem ◽  
Abdulsalam Yau Gital ◽  
Sanah Abdullahi Muaz ◽  
...  

Author(s):  
Tao Yang ◽  
Yadong Wei ◽  
Zhijun Tu ◽  
Haolun Zeng ◽  
Michel A. Kinsy ◽  
...  

2021 ◽  
Vol 87 (10) ◽  
pp. 12-17
Author(s):  
E. I. Molchanova ◽  
E. N. Korzhova ◽  
V. V. Fedorov ◽  
A. D. Portnyagin

The use of artificial neural networks (ANNs) is considered justified when studying the problems that do not have a generally accepted solution algorithm. One of such problems in X-ray fluorescence analysis (XRF) is a control of the metal content in atmospheric air and air of the working area. Determination of the calibration characteristics is raveled by the lack of standard samples of the composition of aerosols collected on the filter. To solve this problem, synthetic calibration samples (CS) were manufactured as a thin organic film containing a powder material of the known chemical composition. The weight of the film samples varied within a range of 40 – 155 mg to simulate different aerosol loading of the filters and the content of components in them changed 20 – 200 times which corresponds to the samples of real aerosols. The possibility of modeling a nonlinear calibration multivariable function using artificial neural networks was evaluated in analysis of 38 film calibration samples (from 40 to 100 mg). The structure of the neural network, activation functions, learning algorithms have been investigated. Modeling was performed using an academic version of the BaseGroup Deductor analytical platform. It is shown that implementation of the back propagation of errors leads to much higher values of the error of analysis compared to the error of the regression calibration functions, whereas the Resilient Propagation algorithm provides the smallest values of the error of vanadium determination (Sr) in the calibration samples of aerosols. The range of low content of the elements in the training set is determined with a greater error compared to high content range, and therefore, the sigmoid activation function leads to unsatisfactory accuracy of the analysis results, and preference should be given to hyperbolic tangent (tanh).


2013 ◽  
Author(s):  
R. J. Elbin ◽  
Anthony P. Kontos ◽  
Jennine Wedge ◽  
Aiobheann Cline ◽  
Scott Dakan ◽  
...  

Sign in / Sign up

Export Citation Format

Share Document