Long-term attraction in higher order neural networks

1998 ◽  
Vol 9 (1) ◽  
pp. 42-50 ◽  
Author(s):  
D. Burshtein
Keyword(s):  
Neural Networks ◽  
Higher Order ◽  
Higher Order Neural Networks

Higher Order Neural Networks with Bayesian Confidence Measure for the Prediction of the EUR/USD Exchange Rate

2009 ◽  
pp. 48-59 ◽  
Author(s):  
Adam Knowles ◽  
Abir Hussain ◽  
Wael El Deredy ◽  
Paulo G.J. Lisboa ◽  
Christian L. Dunis
Keyword(s):  
Neural Networks ◽  
Exchange Rate ◽  
Predictive Performance ◽  
Higher Order ◽  
Confidence Measure ◽  
Local Optima ◽  
Distributed Models ◽  
Higher Order Neural Networks ◽  
Long Term Forecasting

Multi-Layer Perceptrons (MLP) are the most common type of neural network in use, and their ability to perform complex nonlinear mappings and tolerance to noise in data is well documented. However, MLPs also suffer long training times and often reach only local optima. Another type of network is Higher Order Neural Networks (HONN). These can be considered a ‘stripped-down’ version of MLPs, where joint activation terms are used, relieving the network of the task of learning the relationships between the inputs. The predictive performance of the network is tested with the EUR/USD exchange rate and evaluated using standard financial criteria including the annualized return on investment, showing a 8% increase in the return compared with the MLP. The output of the networks that give the highest annualized return in each category was subjected to a Bayesian based confidence measure. This performance improvement may be explained by the explicit and parsimonious representation of high order terms in Higher Order Neural Networks, which combines robustness against noise typical of distributed models, together with the ability to accurately model higher order interactions for long-term forecasting. The effectiveness of the confidence measure is explained by examining the distribution of each network’s output. We speculate that the distribution can be taken into account during training, thus enabling us to produce neural networks with the properties to take advantage of the confidence measure.

On Some Dynamical Properties of Randomly Connected Higher Order Neural Networks

2013 ◽  
pp. 333-363
Author(s):  
Hiromi Miyajima ◽  
Noritaka Shigei ◽  
Shuji Yatsuki
Keyword(s):  
Neural Networks ◽  
Statistical Method ◽  
Complex Dynamics ◽  
Higher Order ◽  
State Model ◽  
Macroscopic Properties ◽  
Higher Order Neural Networks ◽  
Dynamical Properties ◽  
Model C ◽  
Convenient Model

This chapter presents macroscopic properties of higher order neural networks. Randomly connected Neural Networks (RNNs) are known as a convenient model to investigate the macroscopic properties of neural networks. They are investigated by using the statistical method of neuro-dynamics. By applying the approach to higher order neural networks, macroscopic properties of them are made clear. The approach establishes: (a) there are differences between stability of RNNs and Randomly connected Higher Order Neural Networks (RHONNs) in the cases of the digital state -model and the analog state model; (b) there is no difference between stability of RNNs and RHONNs in the cases of the digital state -model and the analog state -model; (c) with neural networks with oscillation, there are large differences between RNNs and RHONNs in the cases of the digital state -model and the analog state -model, that is, there exists complex dynamics in each model for ; (d) behavior of groups composed of RHONNs are represented as a combination of the behavior of each RHONN.

Fundamentals of Higher Order Neural Networks for Modeling and Simulation

2013 ◽  
pp. 103-133 ◽  
Author(s):  
Madan M. Gupta ◽  
Ivo Bukovsky ◽  
Noriyasu Homma ◽  
Ashu M. G. Solo ◽  
Zeng-Guang Hou
Keyword(s):  
Neural Networks ◽  
Modeling And Simulation ◽  
Learning Algorithm ◽  
Nonlinear Approximation ◽  
Higher Order ◽  
Higher Order Neural Networks ◽  
High Dynamic ◽  
Input Variables ◽  
Continuous Dynamic

In this chapter, the authors provide fundamental principles of Higher Order Neural Units (HONUs) and Higher Order Neural Networks (HONNs) for modeling and simulation. An essential core of HONNs can be found in higher order weighted combinations or correlations between the input variables and HONU. Except for the high quality of nonlinear approximation of static HONUs, the capability of dynamic HONUs for the modeling of dynamic systems is shown and compared to conventional recurrent neural networks when a practical learning algorithm is used. In addition, the potential of continuous dynamic HONUs to approximate high dynamic order systems is discussed, as adaptable time delays can be implemented. By using some typical examples, this chapter describes how and why higher order combinations or correlations can be effective for modeling of systems.

A New Topology for Artificial Higher Order Neural Networks

2009 ◽  
pp. 430-441 ◽  
Author(s):  
Zhao Lu ◽  
Leang-san Shieh ◽  
Guanrong Chen
Keyword(s):  
Neural Networks ◽  
Network Performance ◽  
System Modeling ◽  
Higher Order ◽  
Polynomial Kernel ◽  
Support Vector ◽  
Risk Minimization ◽  
Higher Order Neural Networks ◽  
Hidden Layer ◽  
Kernel Support Vector Machines

Aiming to develop a systematic approach for optimizing the structure of artificial higher order neural networks (HONN) for system modeling and function approximation, a new HONN topology, namely polynomial kernel networks, is proposed in this chapter. Structurally, the polynomial kernel network can be viewed as a three-layer feedforward neural network with a special polynomial activation function for the nodes in the hidden layer. The new network is equivalent to a HONN; however, due to the underlying connections with polynomial kernel support vector machines, the weights and the structure of the network can be determined simultaneously using structural risk minimization. The advantage of the topology of the polynomial kernel network and the use of a support vector kernel expansion paves the way to represent nonlinear functions or systems, and underpins some advanced analysis of the network performance. In this chapter, from the perspective of network complexity, both quadratic programming and linear programming based training of the polynomial kernel network are investigated.

Sign in / Sign up

Export Citation Format

Share Document