Fundamentals of Higher Order Neural Networks for Modeling and Simulation

Parameter Values ◽

Better Than

This chapter delivers general format of higher order neural networks (HONNs) for nonlinear data analysis and six different HONN models. Then, this chapter mathematically proves that HONN models could converge and have mean squared errors close to zero. Moreover, this chapter illustrates the learning algorithm with update formulas. HONN models are compared with SAS nonlinear (NLIN) models, and results show that HONN models are 3 to 12% better than SAS nonlinear models. Finally, this chapter shows how to use HONN models to find the best model, order, and coefficients without writing the regression expression, declaring parameter names, and supplying initial parameter values.

Artificial Higher Order Neural Networks for Modeling and Simulation

10.4018/978-1-4666-2175-6 ◽

2013 ◽

Cited By ~ 8

Keyword(s):

Neural Networks ◽

Modeling And Simulation ◽

Higher Order ◽

A Hybrid Higher Order Neural Structure for Pattern Recognition

Artificial Higher Order Neural Networks for Modeling and Simulation ◽

10.4018/978-1-4666-2175-6.ch017 ◽

2013 ◽

pp. 364-387 ◽

Cited By ~ 1

Author(s):

Mehdi Fallahnezhad ◽

Salman Zaferanlouei

Keyword(s):

Neural Networks ◽

Pattern Recognition ◽

Structure Learning ◽

Learning Algorithm ◽

Hybrid Structure ◽

Higher Order ◽

Superior Performance ◽

Data Sets ◽

Neural Structure ◽

Considering high order correlations of selected features next to the raw features of input can facilitate target pattern recognition. In artificial intelligence, this is being addressed by Higher Order Neural Networks (HONNs). In general, HONN structures provide superior specifications (e.g. resolving the dilemma of choosing the number of neurons and layers of networks, better fitting specs, quicker, and open-box specificity) to traditional neural networks. This chapter introduces a hybrid structure of higher order neural networks, which can be generally applied in various branches of pattern recognition. Structure, learning algorithm, and network configuration are introduced, and structure is applied either as classifier (where is called HHONC) to different benchmark statistical data sets or as functional behavior approximation (where is called HHONN) to a heat and mass transfer dilemma. In each structure, results are compared with previous studies, which show its superior performance next to other mentioned advantages.

Advances in Computational Intelligence and Robotics - Applied Artificial Higher Order Neural Networks for Control and Recognition ◽

A Theoretical Framework for Parallel Implementation of Deep Higher Order Neural Networks

10.4018/978-1-5225-0063-6.ch013 ◽

2016 ◽

pp. 351-361

Author(s):

Shuxiang Xu ◽

Yunling Liu

Keyword(s):

Neural Networks ◽

Distributed System ◽

Learning Algorithm ◽

Parallel Implementation ◽

Theoretical Framework ◽

Higher Order ◽

Area Network ◽

System Environment ◽

New Learning ◽

This chapter proposes a theoretical framework for parallel implementation of Deep Higher Order Neural Networks (HONNs). First, we develop a new partitioning approach for mapping HONNs to individual computers within a master-slave distributed system (a local area network). This will allow us to use a network of computers (rather than a single computer) to train a HONN to drastically increase its learning speed: all of the computers will be running the HONN simultaneously (parallel implementation). Next, we develop a new learning algorithm so that it can be used for HONN learning in a distributed system environment. Finally, we propose to improve the generalisation ability of the new learning algorithm as used in a distributed system environment. Theoretical analysis of the proposal is thoroughly conducted to verify the soundness of the new approach. Experiments will be performed to test the new algorithm in the future.

STABILITY ANALYSIS OF HIGHER-ORDER NEURAL NETWORKS FOR COMBINATORIAL OPTIMIZATION

International Journal of Neural Systems ◽

10.1142/s0129065702001151 ◽

2002 ◽

Vol 12 (03n04) ◽

pp. 177-186 ◽

Cited By ~ 8

Author(s):

BRENTON COOPER

Keyword(s):

Neural Networks ◽

Stability Analysis ◽

Combinatorial Optimization ◽

Arbitrary Order ◽

Higher Order ◽

Hopfield Network ◽

Trade Off ◽

The Stability

Recurrent neural networks with higher order connections, from here on referred to as higher-order neural networks (HONNs), may be used for the solution of combinatorial optimization problems. In Ref. 5 a mapping of the traveling salesman problem (TSP) onto a HONN of arbitrary order was developed, thereby creating a family of related networks that can be used to solve the TSP. In this paper, we explore the trade-off between network complexity and quality of solution that is made available by the HONN mapping of the TSP. The trade-off is investigated by undertaking an analysis of the stability of valid solutions to the TSP in a HONN of arbitrary order. The techniques used to perform the stability analysis are not new, but have been widely used elsewhere in the literature.15–17 The original contribution in this paper is the application of these techniques to a HONN of arbitrary order used to solve the TSP. The results of the stability analysis show that the quality of solution is improved by increasing the network complexity, as measured by the order of the network. Furthermore, it is shown that the Hopfield network, as the simplest network in the family of higher-order networks, is expected to produce the poorest quality of solution.

Emerging Capabilities and Applications of Artificial Higher Order Neural Networks - Advances in Computational Intelligence and Robotics ◽

Models of Artificial Higher Order Neural Networks

10.4018/978-1-7998-3563-9.ch001 ◽

2021 ◽

pp. 1-96

Keyword(s):

Neural Network ◽

Neural Networks ◽

High Frequency ◽

Learning Algorithm ◽

Network Models ◽

Higher Order ◽

Neural Network Models ◽

Ultra High Frequency ◽

This chapter introduces the background of the higher order neural network (HONN) model developing history and overviews 24 applied artificial higher order neural network models. This chapter provides 24 HONN models and uses a single uniform HONN architecture for all 24 HONN models. This chapter also uses a uniform learning algorithm for all 24 HONN models and uses uniform weight update formulae for all 24 HONN models. In this chapter, polynomial HONN, Trigonometric HONN, Sigmoid HONN, SINC HONN, and Ultra High Frequency HONN structure and models are overviewed too.

A Theoretical Framework for Parallel Implementation of Deep Higher Order Neural Networks

Nature-Inspired Computing ◽

10.4018/978-1-5225-0788-8.ch001 ◽

2016 ◽

pp. 1-11

Author(s):

Shuxiang Xu ◽

Yunling Liu

Keyword(s):

Neural Networks ◽

Distributed System ◽

Learning Algorithm ◽

Parallel Implementation ◽

Theoretical Framework ◽

Higher Order ◽

Area Network ◽

System Environment ◽

New Learning ◽

This chapter proposes a theoretical framework for parallel implementation of Deep Higher Order Neural Networks (HONNs). First, we develop a new partitioning approach for mapping HONNs to individual computers within a master-slave distributed system (a local area network). This will allow us to use a network of computers (rather than a single computer) to train a HONN to drastically increase its learning speed: all of the computers will be running the HONN simultaneously (parallel implementation). Next, we develop a new learning algorithm so that it can be used for HONN learning in a distributed system environment. Finally, we propose to improve the generalisation ability of the new learning algorithm as used in a distributed system environment. Theoretical analysis of the proposal is thoroughly conducted to verify the soundness of the new approach. Experiments will be performed to test the new algorithm in the future.

Fundamental Theory of Artificial Higher Order Neural Networks

Artificial Higher Order Neural Networks for Economics and Business ◽

10.4018/978-1-59904-897-0.ch017 ◽

2009 ◽

pp. 368-387 ◽

Cited By ~ 4

Author(s):

Madan M. Gupta ◽

Noriyasu Homma ◽

Zeng-Guang Hou ◽

Ashu M. G. Solo ◽

Takakuni Goto

Keyword(s):

Neural Networks ◽

Higher Order ◽

Fundamental Theory ◽

Input Variables

In this chapter, we aim to describe fundamental principles of artificial higher order neural units (AHONUs) and networks (AHONNs). An essential core of AHONNs can be found in higher order weighted combinations or correlations between the input variables. By using some typical examples, this chapter describes how and why higher order combinations or correlations can be effective.

Artificial Higher Order Neural Network Nonlinear Models

Artificial Higher Order Neural Networks for Economics and Business ◽

10.4018/978-1-59904-897-0.ch001 ◽

2009 ◽

pp. 1-47 ◽

Cited By ~ 9

Author(s):

Ming Zhang

Keyword(s):

Neural Network ◽

Neural Networks ◽

Data Analysis ◽

Learning Algorithm ◽

Nonlinear Models ◽

Higher Order ◽

Model Order ◽

Parameter Values ◽

Better Than

This chapter delivers general format of Higher Order Neural Networks (HONNs) for nonlinear data analysis and six different HONN models. This chapter mathematically proves that HONN models could converge and have mean squared errors close to zero. This chapter illustrates the learning algorithm with update formulas. HONN models are compared with SAS Nonlinear (NLIN) models and results show that HONN models are 3 to 12% better than SAS Nonlinear models. Moreover, this chapter shows how to use HONN models to find the best model, order and coefficients, without writing the regression expression, declaring parameter names, and supplying initial parameter values.

Higher Order Neural Networks

Artificial Higher Order Neural Networks for Computer Science and Engineering ◽

10.4018/978-1-61520-711-4.ch017 ◽

2010 ◽

pp. 397-422 ◽

Cited By ~ 7

Author(s):

Madan M. Gupta ◽

Noriyasu Homma ◽

Zeng-Guang Hou ◽

Ashu M. G. Solo ◽

Ivo Bukovsky

Keyword(s):

Neural Networks ◽

Higher Order ◽

Input Variables

In this chapter, we provide fundamental principles of higher order neural units (HONUs) and higher order neural networks (HONNs). An essential core of HONNs can be found in higher order weighted combinations or correlations between the input variables. By using some typical examples, this chapter describes how and why higher order combinations or correlations can be effective.