Ultra High Frequency Sigmoid and Trigonometric Higher Order Neural Networks for Data Pattern Recognition

This chapter develops a new nonlinear model, Ultra high frequency siGmoid and Trigonometric Higher Order Neural Networks (UGT-HONN), for data pattern recognition. UGT-HONN includes Ultra high frequency siGmoid and Sine function Higher Order Neural Networks (UGS-HONN) and Ultra high frequency siGmoid and Cosine functions Higher Order Neural Networks (UGC-HONN). UGS-HONN and UGC-HONN models are used to recognition data patterns. Results show that UGS-HONN and UGC-HONN models are better than other Polynomial Higher Order Neural Network (PHONN) and Trigonometric Higher Order Neural Network (THONN) models, since UGS-HONN and UGC-HONN models to recognize data pattern with error approaching 0.0000%.

Utilizing Feature Selection on Higher Order Neural Networks

Artificial Neural Network has shown its impressive ability on many real world problems such as pattern recognition, classification and function approximation. An extension of ANN, higher order neural network (HONN), improves ANN's computational and learning capabilities. However, the large number of higher order attributes leads to long learning time and complex network structure. Some irrelevant higher order attributes can also hinder the performance of HONN. In this chapter, feature selection algorithms will be used to simplify HONN architecture. Comparisons of fully connected HONN with feature selected HONN demonstrate that proper feature selection can be effective on decreasing number of inputs, reducing computational time, and improving prediction accuracy of HONN.

Improving Performance of Higher Order Neural Network using Artificial Chemical Reaction Optimization

Multilayer neural networks are commonly and frequently used technique for mapping complex nonlinear input-output relationship. However, they add more computational cost due to structural complexity in architecture. This chapter presents different functional link networks (FLN), a class of higher order neural network (HONN). FLNs are capable to handle linearly non-separable classes by increasing the dimensionality of the input space by using nonlinear combinations of input signals. Usually such network is trained with gradient descent based back propagation technique, but it suffers from many drawbacks. To overcome the drawback, here a natural chemical reaction inspired metaheuristic technique called as artificial chemical reaction optimization (ACRO) is used to train the network. As a case study, forecasting of the stock index prices of different stock markets such as BSE, NASDAQ, TAIEX, and FTSE are considered here to compare and analyze the performance gain over the traditional techniques.

Adaptive Hybrid Higher Order Neural Networks for Prediction of Stock Market Behavior

This chapter presents two higher order neural networks (HONN) for efficient prediction of stock market behavior. The models include Pi-Sigma, and Sigma-Pi higher order neural network models. Along with the traditional gradient descent learning, how the evolutionary computation technique such as genetic algorithm (GA) can be used effectively for the learning process is also discussed here. The learning process is made adaptive to handle the noise and uncertainties associated with stock market data. Further, different prediction approaches are discussed here and application of HONN for time series forecasting is illustrated with real life data taken from a number of stock markets across the globe.

Higher Order Neural Network for Financial Modeling and Simulation

Financial market creates a complex and ever changing environment in which population of investors are competing for profit. Predicting the future for financial gain is a difficult and challenging task, however at the same time it is a profitable activity. Hence, the ability to obtain the highly efficient financial model has become increasingly important in the competitive world. To cope with this, we consider functional link artificial neural networks (FLANNs) trained by particle swarm optimization (PSO) for stock index prediction (PSO-FLANN). Our strong experimental conviction confirms that the performance of PSO tuned FLANN model for the case of lower number of ahead prediction task is promising. In most cases LMS updated algorithm based FLANN model proved to be as good as or better than the RLS updated algorithm based FLANN but at the same time RLS updated FLANN model for the prediction of stock index system cannot be ignored.

Ant Colony Optimization Applied to the Training of a High Order Neural Network with Adaptable Exponential Weights

High order neural networks (HONN) are neural networks which employ neurons that combine their inputs non-linearly. The HONEST (High Order Network with Exponential SynapTic links) network is a HONN that uses neurons with product units and adaptable exponents. The output of a trained HONEST network can be expressed in terms of the network inputs by a polynomial-like equation. This makes the structure of the network more transparent and easier to interpret. This study adapts ACOR, an Ant Colony Optimization algorithm, to the training of an HONEST network. Using a collection of 10 widely-used benchmark datasets, we compare ACOR to the well-known gradient-based Resilient Propagation (R-Prop) algorithm, in the training of HONEST networks. We find that our adaptation of ACOR has better test set generalization than R-Prop, though not to a statistically significant extent.

A Theoretical Framework for Parallel Implementation of Deep Higher Order Neural Networks

This chapter proposes a theoretical framework for parallel implementation of Deep Higher Order Neural Networks (HONNs). First, we develop a new partitioning approach for mapping HONNs to individual computers within a master-slave distributed system (a local area network). This will allow us to use a network of computers (rather than a single computer) to train a HONN to drastically increase its learning speed: all of the computers will be running the HONN simultaneously (parallel implementation). Next, we develop a new learning algorithm so that it can be used for HONN learning in a distributed system environment. Finally, we propose to improve the generalisation ability of the new learning algorithm as used in a distributed system environment. Theoretical analysis of the proposal is thoroughly conducted to verify the soundness of the new approach. Experiments will be performed to test the new algorithm in the future.

Artificial Higher Order Neural Network Models

This chapter introduces the background of HONN model developing history and overview 24 applied artificial higher order neural network models. This chapter provides 24 HONN models and uses a single uniform HONN architecture for ALL 24 HONN models. This chapter also uses a uniform learning algorithm for all 24 HONN models and uses a uniform weight update formulae for all 24 HONN models. In this chapter, Polynomial HONN, Trigonometric HONN, Sigmoid HONN, SINC HONN, and Ultra High Frequency HONN structure and models are overviewed too.

Theoretical Analyses of the Universal Approximation Capability of a class of Higher Order Neural Networks based on Approximate Identity

One of the most important problems in the theory of approximation functions by means of neural networks is universal approximation capability of neural networks. In this study, we investigate the theoretical analyses of the universal approximation capability of a special class of three layer feedforward higher order neural networks based on the concept of approximate identity in the space of continuous multivariate functions. Moreover, we present theoretical analyses of the universal approximation capability of the networks in the spaces of Lebesgue integrable multivariate functions. The methods used in proving our results are based on the concepts of convolution and epsilon-net. The obtained results can be seen as an attempt towards the development of approximation theory by means of neural networks.

Ultra High Frequency SINC and Trigonometric Higher Order Neural Networks for Data Classification

This chapter develops a new nonlinear model, Ultra high frequency SINC and Trigonometric Higher Order Neural Networks (UNT-HONN), for Data Classification. UNT-HONN includes Ultra high frequency siNc and Sine Higher Order Neural Networks (UNS-HONN) and Ultra high frequency siNc and Cosine Higher Order Neural Networks (UNC-HONN). Data classification using UNS-HONN and UNC-HONN models are tested. Results show that UNS-HONN and UNC-HONN models are better than other Polynomial Higher Order Neural Network (PHONN) and Trigonometric Higher Order Neural Network (THONN) models, since UNS-HONN and UNC-HONN models can classify the data with error approaching 0.0000%.