Finite-Time Convergent Recurrent Neural Network With a Hard-Limiting Activation Function for Constrained Optimization With Piecewise-Linear Objective Functions

2011 ◽  
Vol 22 (4) ◽  
pp. 601-613 ◽  
Author(s):  
Qingshan Liu ◽  
Jun Wang
Keyword(s):  
Neural Network ◽  
Constrained Optimization ◽  
Recurrent Neural Network ◽  
Finite Time ◽  
Piecewise Linear ◽  
Activation Function ◽  
Objective Functions ◽  
Linear Objective Functions

All-optical recurrent neural network with sigmoid activation function

2020 ◽  
Author(s):  
George Mourgias-Alexandris ◽  
George Dabos ◽  
Nikolaos Passalis ◽  
Anastasios Tefas ◽  
Angelina Totovic ◽  
...  
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Activation Function ◽  
All Optical ◽  
Sigmoid Activation Function

Relating the Slope of the Activation Function and the Learning Rate Within a Recurrent Neural Network

1999 ◽  
Vol 11 (5) ◽  
pp. 1069-1077 ◽  
Author(s):  
Danilo P. Mandic ◽  
Jonathon A. Chambers
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Recurrent Neural Network ◽  
Recurrent Neural Networks ◽  
Degrees Of Freedom ◽  
Learning Algorithm ◽  
Activation Function ◽  
Learning Rate ◽  
Optimization Task ◽  
Nonlinear Activation Function

A relationship between the learning rate η in the learning algorithm, and the slope β in the nonlinear activation function, for a class of recurrent neural networks (RNNs) trained by the real-time recurrent learning algorithm is provided. It is shown that an arbitrary RNN can be obtained via the referent RNN, with some deterministic rules imposed on its weights and the learning rate. Such relationships reduce the number of degrees of freedom when solving the nonlinear optimization task of finding the optimal RNN parameters.

scholarly journals General Recurrent Neural Network for Solving Generalized Linear Matrix Equation

Complexity ◽  
2017 ◽  
Vol 2017 ◽  
pp. 1-7 ◽  
Author(s):  
Zhan Li ◽  
Hong Cheng ◽  
Hongliang Guo
Keyword(s):  
Neural Network ◽  
Network Model ◽  
Recurrent Neural Network ◽  
Neural Network Model ◽  
Matrix Equation ◽  
Activation Function ◽  
Linear Matrix ◽  
Activation Functions ◽  
State Matrix ◽  
Linear Matrix Equation

This brief proposes a general framework of the nonlinear recurrent neural network for solving online the generalized linear matrix equation (GLME) with global convergence property. If the linear activation function is utilized, the neural state matrix of the nonlinear recurrent neural network can globally and exponentially converge to the unique theoretical solution of GLME. Additionally, as compared with the case of using the linear activation function, two specific types of nonlinear activation functions are proposed for the general nonlinear recurrent neural network model to achieve superior convergence. Illustrative examples are shown to demonstrate the efficacy of the general nonlinear recurrent neural network model and its superior convergence when activated by the aforementioned nonlinear activation functions.

scholarly journals A recurrent neural network based on memristive activation function and its associative memory

2017 ◽  
Vol 47 (9) ◽  
pp. 1226-1241 ◽  
Author(s):  
Shukai DUAN ◽  
Tengteng Guo ◽  
Mengzhe ZHOU ◽  
Lidan WANG
Keyword(s):  
Neural Network ◽  
Associative Memory ◽  
Recurrent Neural Network ◽  
Activation Function
Sign in / Sign up

Export Citation Format

Share Document