A New Topology for Artificial Higher Order Neural Networks

Aiming to develop a systematic approach for optimizing the structure of artificial higher order neural networks (HONN) for system modeling and function approximation, a new HONN topology, namely polynomial kernel networks, is proposed in this chapter. Structurally, the polynomial kernel network can be viewed as a three-layer feedforward neural network with a special polynomial activation function for the nodes in the hidden layer. The new network is equivalent to a HONN; however, due to the underlying connections with polynomial kernel support vector machines, the weights and the structure of the network can be determined simultaneously using structural risk minimization. The advantage of the topology of the polynomial kernel network and the use of a support vector kernel expansion paves the way to represent nonlinear functions or systems, and underpins some advanced analysis of the network performance. In this chapter, from the perspective of network complexity, both quadratic programming and linear programming based training of the polynomial kernel network are investigated.

Electric Load Demand and Electricity Prices ForecastingUsing Higher Order Neural Networks Trained by Kalman Filtering

In this chapter, we propose the use of Higher Order Neural Networks (HONNs) trained with an extended Kalman filter based algorithm to predict the electric load demand as well as the electricity prices, with beyond a horizon of 24 hours. Due to the chaotic behavior of the electrical markets, it is not advisable to apply the traditional forecasting techniques used for time series; the results presented here confirm that HONNs can very well capture the complexity underlying electric load demand and electricity prices. The proposed neural network model produces very accurate next day predictions and also, prognosticates with very good accuracy, a week-ahead demand and price forecasts.

Generalized Correlation Higher Order Neural Networks for Financial Time Series Prediction

Generalized correlation higher order neural network designs are developed. Their performance is compared with that of first order networks, conventional higher order neural network designs, and higher order linear regression networks for financial time series prediction. The correlation higher order neural network design is shown to give the highest accuracy for prediction of stock market share prices and share indices. The simulations compare the performance for three different training algorithms, stationary versus non-stationary input data, different numbers of neurons in the hidden layer and several generalized correlation higher order neural network designs. Generalized correlation higher order linear regression networks are also introduced and two designs are shown by simulation to give good correct direction prediction and higher prediction accuracies, particularly for long-term predictions, than other linear regression networks for the prediction of inter-bank lending risk Libor and Swap interest rate yield curves. The simulations compare the performance for different input data sample lag lengths.

A Novel Recurrent Polynomial Neural Network for Financial Time Series Prediction

The research described in this chapter is concerned with the development of a novel artificial higherorder neural networks architecture called the recurrent Pi-sigma neural network. The proposed artificial neural network combines the advantages of both higher-order architectures in terms of the multi-linear interactions between inputs, as well as the temporal dynamics of recurrent neural networks, and produces highly accurate one-step ahead predictions of the foreign currency exchange rates, as compared to other feedforward and recurrent structures.

Higher Order Neural Networks for Stock Index Modeling

Artificial Neural Networks (ANNs) have become very important in making stock market predictions. Much research on the applications of ANNs has proven their advantages over statistical and other methods. In order to identify the main benefits and limitations of previous methods in ANNs applications, a comparative analysis of selected applications is conducted. It can be concluded from analysis that ANNs and HONNs are most implemented in forecasting stock prices and stock modeling. The aim of this chapter is to study higher order artificial neural networks for stock index modeling problems. New network architectures and their corresponding training algorithms are discussed. These structures demonstrate their processing capabilities over traditional ANNs architectures with a reduction in the number of processing elements. In this chapter, the performance of classical neural networks and higher order neural networks for stock index forecasting is evaluated. We will highlight a novel slide-window method for data forecasting. With each slide of the observed data, the model can adjusts the variable dynamically. Simulation results show the feasibility and effectiveness of the proposed methods.

Higher Order Neural Network Architectures for Agent-Based Computational Economics and Finance

As the study of agent-based computational economics and finance grows, so does the need for appropriate techniques for the modeling of complex dynamic systems and the intelligence of the constructive agent. These methods are important where the classic equilibrium analytics fail to provide sufficiently satisfactory understanding. In particular, one area of computational intelligence, Approximate Dynamic Programming, holds much promise for applications in this field and demonstrate the capacity for artificial Higher Order Neural Networks to add value in the social sciences and business. This chapter provides an overview of this area, introduces the relevant agent-based computational modeling systems, and suggests practical methods for their incorporation into the current research. A novel application of HONN to ADP specifically for the purpose of studying agent-based financial systems is presented.

Artificial Higher Order Pipeline Recurrent Neural Networks for Financial Time Series Prediction

The research described in this chapter is concerned with the development of a novel artificial higher order neural networks architecture called the second-order pipeline recurrent neural network. The proposed artificial neural network consists of a linear and a nonlinear section, extracting relevant features from the input signal. The structuring unit of the proposed neural network is the second-order recurrent neural network. The architecture consists of a series of second-order recurrent neural networks, which are concatenated with each other. Simulation results in one-step ahead predictions of the foreign currency exchange rates demonstrate the superior performance of the proposed pipeline architecture as compared to other feed-forward and recurrent structures.

Automatically Identifying Predictor Variables for Stock Return Prediction

Real-world financial systems are often nonlinear, do not follow any regular probability distribution, and comprise a large amount of financial variables. Not surprisingly, it is hard to know which variables are relevant to the prediction of the stock return based on data collected from such a system. In this chapter, we address this problem by developing a technique consisting of a top-down part using an artificial Higher Order Neural Network (HONN) model and a bottom-up part based on a Bayesian Network (BN) model to automatically identify predictor variables for the stock return prediction from a large financial variable set. Our study provides an operational guidance for using HONN and BN in selecting predictor variables from a large amount of financial variables to support the prediction of the stock return, including the prediction of future stock return value and future stock return movement trends.

Higher Order Neural Networks with Bayesian Confidence Measure for the Prediction of the EUR/USD Exchange Rate

Multi-Layer Perceptrons (MLP) are the most common type of neural network in use, and their ability to perform complex nonlinear mappings and tolerance to noise in data is well documented. However, MLPs also suffer long training times and often reach only local optima. Another type of network is Higher Order Neural Networks (HONN). These can be considered a ‘stripped-down’ version of MLPs, where joint activation terms are used, relieving the network of the task of learning the relationships between the inputs. The predictive performance of the network is tested with the EUR/USD exchange rate and evaluated using standard financial criteria including the annualized return on investment, showing a 8% increase in the return compared with the MLP. The output of the networks that give the highest annualized return in each category was subjected to a Bayesian based confidence measure. This performance improvement may be explained by the explicit and parsimonious representation of high order terms in Higher Order Neural Networks, which combines robustness against noise typical of distributed models, together with the ability to accurately model higher order interactions for long-term forecasting. The effectiveness of the confidence measure is explained by examining the distribution of each network’s output. We speculate that the distribution can be taken into account during training, thus enabling us to produce neural networks with the properties to take advantage of the confidence measure.

High Speed Optical Higher Order Neural Networks for Discovering Data Trends and Patterns in Very Large Databases

This chapter describes the progress in using optical technology to construct high-speed artificial higher order neural network systems. The chapter reviews how optical technology can speed up searches within large databases in order to identify relationships and dependencies between individual data records, such as financial or business time-series, as well as trends and relationships within them. Two distinct approaches in which optics may be used are reviewed. In the first approach, the chapter reviews current research replacing copper connections in a conventional data storage system, such as a several terabyte RAID array of magnetic hard discs, by optical waveguides to achieve very high data rates with low crosstalk interference. In the second approach, the chapter reviews how high speed optical correlators with feedback can be used to realize artificial higher order neural networks using Fourier Transform free space optics and holographic database storage.