A training algorithm with selectable search direction for complex-valued feedforward neural networks

Current neural networks architectures are many times harder to train because of the increasing size and complexity of the used datasets. Our objective is to design more efficient training algorithms utilizing causal relationships inferred from neural networks. The transfer entropy (TE) was initially introduced as an information transfer measure used to quantify the statistical coherence between events (time series). Later, it was related to causality, even if they are not the same. There are only few papers reporting applications of causality or TE in neural networks. Our contribution is an information-theoretical method for analyzing information transfer between the nodes of feedforward neural networks. The information transfer is measured by the TE of feedback neural connections. Intuitively, TE measures the relevance of a connection in the network and the feedback amplifies this connection. We introduce a backpropagation type training algorithm that uses TE feedback connections to improve its performance.

Download Full-text

A New Layer by Layer training algorithm for multilayer feedforward neural networks

2011 3rd International Conference on Advanced Computer Control ◽

10.1109/icacc.2011.6016485 ◽

2011 ◽

Author(s):

Yanlai Li ◽

Tao Li ◽

Kuanquan Wang

Keyword(s):

Neural Networks ◽

Feedforward Neural Networks ◽

Layer By Layer ◽

Training Algorithm

Download Full-text

The HJPS Training Algorithm for Multilayer Feedforward Neural Networks

Journal of Computer Research and Development ◽

10.1360/crad20051023 ◽

2005 ◽

Vol 42 (10) ◽

pp. 1790

Author(s):

Yanlai Li

Keyword(s):

Neural Networks ◽

Feedforward Neural Networks ◽

Training Algorithm

Download Full-text

Fully Coupled and Feedforward Neural Networks with Complex-Valued Neurons

Advances in Intelligent and Distributed Computing - Studies in Computational Intelligence ◽

10.1007/978-3-540-74930-1_5 ◽

2008 ◽

pp. 41-50 ◽

Cited By ~ 1

Author(s):

Jacek M. Zurada ◽

Igor Aizenberg

Keyword(s):

Neural Networks ◽

Feedforward Neural Networks ◽

Complex Valued ◽

Fully Coupled

Download Full-text

Fast training algorithm for feedforward neural networks: application to crowd estimation at underground stations

Artificial Intelligence in Engineering ◽

10.1016/s0954-1810(99)00016-3 ◽

1999 ◽

Vol 13 (3) ◽

pp. 301-307 ◽

Cited By ~ 15

Author(s):

T.W.S. Chow ◽

J.Y.-F. Yam ◽

S.-Y Cho

Keyword(s):

Neural Networks ◽

Feedforward Neural Networks ◽

Training Algorithm ◽

Fast Training

Download Full-text

Training Feedforward Neural Networks Using Symbiotic Organisms Search Algorithm

Computational Intelligence and Neuroscience ◽

10.1155/2016/9063065 ◽

2016 ◽

Vol 2016 ◽

pp. 1-14 ◽

Cited By ~ 20

Author(s):

Haizhou Wu ◽

Yongquan Zhou ◽

Qifang Luo ◽

Mohamed Abdel Basset

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Search Algorithm ◽

Feedforward Neural Networks ◽

Metaheuristic Algorithm ◽

Training Algorithm ◽

Symbiotic Interaction ◽

Symbiotic Organisms Search ◽

Symbiotic Organisms ◽

Better Than

Symbiotic organisms search (SOS) is a new robust and powerful metaheuristic algorithm, which stimulates the symbiotic interaction strategies adopted by organisms to survive and propagate in the ecosystem. In the supervised learning area, it is a challenging task to present a satisfactory and efficient training algorithm for feedforward neural networks (FNNs). In this paper, SOS is employed as a new method for training FNNs. To investigate the performance of the aforementioned method, eight different datasets selected from the UCI machine learning repository are employed for experiment and the results are compared among seven metaheuristic algorithms. The results show that SOS performs better than other algorithms for training FNNs in terms of converging speed. It is also proven that an FNN trained by the method of SOS has better accuracy than most algorithms compared.

Download Full-text