Higher order neural networks for combinatorial optimisation improving the scaling properties of the Hopfield network

2002 ◽  
Author(s):  
B.S. Cooper
Keyword(s):  
Neural Networks ◽  
Higher Order ◽  
Combinatorial Optimisation ◽  
Hopfield Network ◽  
Scaling Properties ◽  
Higher Order Neural Networks

STABILITY ANALYSIS OF HIGHER-ORDER NEURAL NETWORKS FOR COMBINATORIAL OPTIMIZATION

2002 ◽  
Vol 12 (03n04) ◽  
pp. 177-186 ◽  
Author(s):  
BRENTON COOPER
Keyword(s):  
Neural Networks ◽  
Stability Analysis ◽  
Combinatorial Optimization ◽  
Arbitrary Order ◽  
Higher Order ◽  
Hopfield Network ◽  
Trade Off ◽  
Higher Order Neural Networks ◽  
The Stability

Recurrent neural networks with higher order connections, from here on referred to as higher-order neural networks (HONNs), may be used for the solution of combinatorial optimization problems. In Ref. 5 a mapping of the traveling salesman problem (TSP) onto a HONN of arbitrary order was developed, thereby creating a family of related networks that can be used to solve the TSP. In this paper, we explore the trade-off between network complexity and quality of solution that is made available by the HONN mapping of the TSP. The trade-off is investigated by undertaking an analysis of the stability of valid solutions to the TSP in a HONN of arbitrary order. The techniques used to perform the stability analysis are not new, but have been widely used elsewhere in the literature.15–17 The original contribution in this paper is the application of these techniques to a HONN of arbitrary order used to solve the TSP. The results of the stability analysis show that the quality of solution is improved by increasing the network complexity, as measured by the order of the network. Furthermore, it is shown that the Hopfield network, as the simplest network in the family of higher-order networks, is expected to produce the poorest quality of solution.

On Some Dynamical Properties of Randomly Connected Higher Order Neural Networks

2013 ◽  
pp. 333-363
Author(s):  
Hiromi Miyajima ◽  
Noritaka Shigei ◽  
Shuji Yatsuki
Keyword(s):  
Neural Networks ◽  
Statistical Method ◽  
Complex Dynamics ◽  
Higher Order ◽  
State Model ◽  
Macroscopic Properties ◽  
Higher Order Neural Networks ◽  
Dynamical Properties ◽  
Model C ◽  
Convenient Model

This chapter presents macroscopic properties of higher order neural networks. Randomly connected Neural Networks (RNNs) are known as a convenient model to investigate the macroscopic properties of neural networks. They are investigated by using the statistical method of neuro-dynamics. By applying the approach to higher order neural networks, macroscopic properties of them are made clear. The approach establishes: (a) there are differences between stability of RNNs and Randomly connected Higher Order Neural Networks (RHONNs) in the cases of the digital state -model and the analog state model; (b) there is no difference between stability of RNNs and RHONNs in the cases of the digital state -model and the analog state -model; (c) with neural networks with oscillation, there are large differences between RNNs and RHONNs in the cases of the digital state -model and the analog state -model, that is, there exists complex dynamics in each model for ; (d) behavior of groups composed of RHONNs are represented as a combination of the behavior of each RHONN.

Fundamentals of Higher Order Neural Networks for Modeling and Simulation

2013 ◽  
pp. 103-133 ◽  
Author(s):  
Madan M. Gupta ◽  
Ivo Bukovsky ◽  
Noriyasu Homma ◽  
Ashu M. G. Solo ◽  
Zeng-Guang Hou
Keyword(s):  
Neural Networks ◽  
Modeling And Simulation ◽  
Learning Algorithm ◽  
Nonlinear Approximation ◽  
Higher Order ◽  
Higher Order Neural Networks ◽  
High Dynamic ◽  
Input Variables ◽  
Continuous Dynamic

In this chapter, the authors provide fundamental principles of Higher Order Neural Units (HONUs) and Higher Order Neural Networks (HONNs) for modeling and simulation. An essential core of HONNs can be found in higher order weighted combinations or correlations between the input variables and HONU. Except for the high quality of nonlinear approximation of static HONUs, the capability of dynamic HONUs for the modeling of dynamic systems is shown and compared to conventional recurrent neural networks when a practical learning algorithm is used. In addition, the potential of continuous dynamic HONUs to approximate high dynamic order systems is discussed, as adaptable time delays can be implemented. By using some typical examples, this chapter describes how and why higher order combinations or correlations can be effective for modeling of systems.

A New Topology for Artificial Higher Order Neural Networks

2009 ◽  
pp. 430-441 ◽  
Author(s):  
Zhao Lu ◽  
Leang-san Shieh ◽  
Guanrong Chen
Keyword(s):  
Neural Networks ◽  
Network Performance ◽  
System Modeling ◽  
Higher Order ◽  
Polynomial Kernel ◽  
Support Vector ◽  
Risk Minimization ◽  
Higher Order Neural Networks ◽  
Hidden Layer ◽  
Kernel Support Vector Machines

Aiming to develop a systematic approach for optimizing the structure of artificial higher order neural networks (HONN) for system modeling and function approximation, a new HONN topology, namely polynomial kernel networks, is proposed in this chapter. Structurally, the polynomial kernel network can be viewed as a three-layer feedforward neural network with a special polynomial activation function for the nodes in the hidden layer. The new network is equivalent to a HONN; however, due to the underlying connections with polynomial kernel support vector machines, the weights and the structure of the network can be determined simultaneously using structural risk minimization. The advantage of the topology of the polynomial kernel network and the use of a support vector kernel expansion paves the way to represent nonlinear functions or systems, and underpins some advanced analysis of the network performance. In this chapter, from the perspective of network complexity, both quadratic programming and linear programming based training of the polynomial kernel network are investigated.

Sign in / Sign up

Export Citation Format

Share Document