Artificial Neural Networks (ANNs) are known to excellence in pattern recognition, pattern matching and mathematical function approximation. However, they suffer from several limitations. ANNs are often stuck in local, rather than global minima, as well as taking unacceptable long times to converge in the real word data. Especially from the perspective of economics and financial time series predictions, ANNs are unable to handle non-smooth, discontinuous training data, and complex mappings. Another limitation of ANN is a ‘black box’ nature. It means that explanations for their decisions are not hard to use expressions to describe. This then is the first motivation for developing Higher Order Neural Networks (HONNs), since HONNs are ‘open-box’ models and each neuron and weight are mapped to function variable and coefficient.
SAS Nonlinear (NLIN) procedure produces least squares or weighted least squares estimates of the parameters of a nonlinear model. SAS Nonlinear models are more difficult to specify and estimate than linear models. Instead of simply generating the parameter estimates, users must write the regression expression, declare parameter names, and supply initial parameter values. Some models are difficult to fit, and there is no guarantee that the procedure can fit the model successfully. For each nonlinear model to be analyzed, users must specify the model (using a single dependent variable) and the names and starting values of the parameters to be estimated. However, the objective of the users is to find the model and its coefficients. This is the second motivation for using HONNs in economics and business, since HONNs can automatically select the initial coefficients for nonlinear data analysis.
Let millions of people working in economics and business areas know that HONNs are much easier to use and can have better simulation results than SAS NLIN, and understand how to successfully use HONNs software packages for nonlinear data simulation and prediction. HONNs will challenge SAS NLIN procedures and change the research methodology that people are currently using in economics and business areas for the nonlinear date simulation and prediction.
Millions of people who are using SAS and who are doing nonlinear model research, in particular, professors, graduate students, and senior undergraduate students in economics, accounting, finance and other business departments, as well as the professionals and researchers in these areas.
The book is organized into four sections and a total of twenty two chapters. Section 1, Artificial Higher Order Neural Networks for Economics, includes chapter 1 to chapter 6. Section 2, Artificial Higher Order Neural Networks for Time Series Data, is from chapter 7 to chapter 12. Section 3, Artificial Higher Order Neural Networks for Business, contains chapter 13 to chapter 16. Section 4, Artificial Higher Order Neural Networks Fundamentals, consists of chapter 17 to chapter 22. A brief description of each of the chapters are as follows:
Chapter 1, “Artificial Higher Order Neural Network Nonlinear Model - SAS NLIN or HONNs”, delivers general format of Higher Order Neural Networks (HONNs) for nonlinear data analysis and six different HONN models. This chapter mathematically proves that HONN models could converge and have mean squared errors close to zero. This chapter illustrates the learning algorithm with update formulas. HONN models are compared with SAS Nonlinear (NLIN) models and results show that HONN models are 3 to 12% better than SAS Nonlinear models. Moreover, this chapter shows how to use HONN models to find the best model, order and coefficients, without writing the regression expression, declaring parameter names, and supplying initial parameter values.
Chapter 2, “Higher Order Neural Networks with Bayesian Confidence Measure for the Prediction of the EUR/USD Exchange Rate”, presents another type of network which is Higher Order Neural Networks (HONN). These can be considered a 'stripped-down' version of MLPs, where joint activation terms are used, relieving the network of the task of learning the relationships between the inputs. The predictive performance of the network is tested with the EUR/USD exchange rate and evaluated using standard financial criteria including the annualized return on investment, showing a 8% increase in the return compared with the MLP. The output of the networks that give the highest annualized return in each category was subjected to a Bayesian based confidence measure.
Chapter 3, “Automatically Identifying Predictor Variables for Stock Return Prediction”, addresses nonlinear problem by developing a technique consisting of a top-down part using an artificial Higher Order Neural Network (HONN) model and a bottom-up part based on a Bayesian Network (BN) model to automatically identify predictor variables for the stock return prediction from a large financial variable set. Our study provides an operational guidance for using HONN and BN in selecting predictor variables from a large amount of financial variables to support the prediction of the stock return, including the prediction of future stock return value and future stock return movement trends.
Chapter 4, “Higher Order Neural Network Architectures for Agent-Based Computational Economics and Finance”, studies the agent-based computational economics and finance grows, so does the need for appropriate techniques for the modeling of complex dynamic systems and the intelligence of the constructive agent. These methods are important where the classic equilibrium analytics fail to provide sufficiently satisfactory understanding. In particular, one area of computational intelligence, Approximate Dynamic Programming, holds much promise for applications in this field and demonstrates the capacity for artificial Higher Order Neural Networks to add value in the social sciences and business. This chapter provides an overview of this area, introduces the relevant agent-based computational modeling systems, and suggests practical methods for their incorporation into the current research. A novel application of HONN to ADP specifically for the purpose of studying agent-based financial systems is presented.
Chapter 5, “Foreign Exchange Rate Forecasting using Higher Order Flexible Neural Tree”, establishes that Forecasting exchange rates is an important financial problem that is receiving increasing attention especially because of its difficulty and practical applications. In this chapter, we apply Higher Order Flexible Neural Trees (HOFNTs), which are capable of designing flexible Artificial Neural Network (ANN) architectures automatically, to forecast the foreign exchange rates. To demonstrate the efficiency of HOFNTs, we consider three different datasets in our forecast performance analysis. The data sets used are daily foreign exchange rates obtained from the Pacific Exchange Rate Service. The data comprises of the US dollar exchange rate against Euro, Great Britain Pound (GBP) and Japanese Yen (JPY). Under the HOFNT framework, we consider the Gene Expression Programming (GEP) approach and the Grammar Guided Genetic Programming (GGGP) approach to evolve the structure of HOFNT. The particle swarm optimization algorithm is employed to optimize the free parameters of the two different HOFNT models. This chapter briefly explains how the two different learning paradigms could be formulated using various methods and then investigates whether they can provide a reliable forecast model for foreign exchange rates. Simulation results shown the effectiveness of the proposed methods.
Chapter 6, “Higher Order Neural Networks for Stock Index Modeling”, has the aim which is to study higher order artificial neural networks for stock index modeling problems. New network architectures and their corresponding training algorithms are discussed. These structures demonstrate their processing capabilities over traditional ANNs architectures with a reduction in the number of processing elements. In this chapter, the performance of classical neural networks and higher order neural networks for stock index forecasting is evaluated. We will highlight a novel slide-window method for data forecasting. With each slide of the observed data, the model can adjusts the variable dynamically. Simulation results show the feasibility and effectiveness of the proposed methods.
Chapter 7, “Ultra High Frequency Trigonometric Higher Order Neural Networks for Time Series Data Analysis”, develops a new nonlinear model, Ultra high frequency Trigonometric Higher Order Neural Networks (UTHONN), for time series data analysis. Results show that UTHONN models are 3 to 12% better than Equilibrium Real Exchange Rates (ERER) model, and 4 – 9% better than other Polynomial Higher Order Neural Network (PHONN) and Trigonometric Higher Order Neural Network (THONN) models. This study also uses UTHONN models to simulate foreign exchange rates and consumer price index with error approaching 0.0000%.
Chapter 8, “Artificial higher order pipeline recurrent neural networks for financial time series prediction”, is concerned with the development of a novel artificial higher order neural networks architecture called the second-order pipeline recurrent neural network. The proposed artificial neural network consists of a linear and a nonlinear section, extracting relevant features from the input signal. The structuring unit of the proposed neural network is the second-order recurrent neural network. The architecture consists of a series of second-order recurrent neural networks, which are concatenated with each other. Simulation results in one-step ahead predictions of the foreign currency exchange rates demonstrate the superior performance of the proposed pipeline architecture as compared to other feed-forward and recurrent structures.
Chapter 9, “A novel recurrent polynomial neural network for financial time series prediction”, is concerned with the development of a novel artificial higher-order neural networks architecture called the recurrent Pi-sigma neural network. The proposed artificial neural network combines the advantages of both higher-order architectures in terms of the multi-linear interactions between inputs, as well as the temporal dynamics of recurrent neural networks, and produces highly accurate one-step ahead predictions of the foreign currency exchange rates, as compared to other feedforward and recurrent structures.
Chapter 10, “Generalized correlation higher order neural networks for financial time series prediction”, develops a generalized correlation higher order neural network designs. Their performance is compared with that of first order networks, conventional higher order neural network designs, and higher order linear regression networks for financial time series prediction. The correlation higher order neural network design is shown to give the highest accuracy for prediction of stock market share prices and share indices. The simulations compare the performance for three different training algorithms, stationary versus non-stationary input data, different numbers of neurons in the hidden layer and several generalized correlation higher order neural network designs. Generalized correlation higher order linear regression networks are also introduced and two designs are shown by simulation to give good correct direction prediction and higher prediction accuracies, particularly for long-term predictions, than other linear regression networks for the prediction of inter-bank lending risk Libor and Swap interest rate yield curves. The simulations compare the performance for different input data sample lag lengths.
Chapter 11, “Artificial Higher Order Neural Networks in Time Series Prediction”, describes real world problems of non¬linear and chaotic processes, which make them hard to model and predict. This chapter first compares the neural network (NN) and the artificial higher order neural network (HONN) and then presents commonly known neural network architectures and a number of HONN architectures. The time series prediction problem is formulated as a system identification problem, where the input to the system is the past values of a time series, and its desired output is the future values of a time series. The polynomial neural network (PNN) is then chosen as the HONN for application to the time series prediction problem. This chapter presents the application of HONN model to the non¬linear time series prediction problems of three major international currency exchange rates, as well as two key U.S. interest rates—the Federal funds rate and the yield on the 5-year U.S. Treasury note. Empirical results indicate that the proposed method is competitive with other approaches for the exchange rate problem, and can be used as a feasible solution for interest rate forecasting problem. This implies that the HONN model can be used as a feasible solution for exchange rate forecasting as well as for interest rate forecasting.
Chapter 12, “ Application of Pi-Sigma Neural Networks and Ridge Polynomial Neural Networks to Financial Time Series Prediction”, discusses the use of two artificial Higher Order Neural Networks (HONNs) models; the Pi-Sigma Neural Networks and the Ridge Polynomial Neural Networks, in financial time series forecasting. The networks were used to forecast the upcoming trends of three noisy financial signals; the exchange rate between the US Dollar and the Euro, the exchange rate between the Japanese Yen and the Euro, and the United States 10-year government bond. In particular, we systematically investigate a method of pre-processing the signals in order to reduce the trends in them. The performance of the networks is benchmarked against the performance of Multilayer Perceptrons. From the simulation results, the predictions clearly demonstrated that HONNs models, particularly Ridge Polynomial Neural Networks generate higher profit returns with fast convergence, therefore show considerable promise as a decision making tool. It is hoped that individual investor could benefit from the use of this forecasting tool.
Chapter 13, “Electric Load Demand and Electricity Prices Forecasting using Higher Order Neural Networks Trained by Kalman Filtering”, proposes the use of Higher Order Neural Networks (HONNs) trained with an extended Kalman filter based algorithm to predict the electric load demand as well as the electricity prices, with beyond a horizon of 24 hours. Due to the chaotic behavior of the electrical markets, it is not advisable to apply the traditional forecasting techniques used for time series; the results presented here confirm that HONNs can very well capture the complexity underlying electric load demand and electricity prices. The proposed neural network model produces very accurate next day predictions and also, prognosticates with very good accuracy, a week-ahead demand and price forecasts.
Chapter 14, “Adaptive Higher Order Neural Network Models and Their Applications in Business”, introduces an Adaptive Higher Order Neural Network (HONN) model and applies the adaptive model in business applications such as simulating and forecasting share prices. This adaptive HONN model offers significant advantages over traditional Standard ANN models such as much reduced network size, faster training, as well as much improved simulation and forecasting errors, due to their ability to better approximate complex, non-smooth, often discontinuous training data sets. The generalization ability of this HONN model is explored and discussed.
Chapter 15, “CEO Tenure and Debt: An Artificial Higher Order Neural Network Approach”, proposes nonlinear models using artificial neural network models to study the relationship between chief elected official (CEO) tenure and debt. Using Higher Order Neural Network (HONN) simulator, this study analyzes debt of the municipalities as a function of population and CEO tenure, and compares the results with that from SAS. The linear models show that CEO tenure and the amount of debt vary inversely. Specifically, a longer length of CEO tenure leads to a decrease in debt, while a shorter tenure leads to an increase in debt. This chapter shows nonlinear model generated from HONN out performs linear models by 1%. The results from both models reveal that CEO tenure is negatively associated with the level of debt in local governments.
Chapter 16, “Modeling and Trading the Soybean-Oil Crush Spread with Recurrent and Higher Order Networks: A Comparative Analysis”, investigates the soybean-oil “crush” spread, that is the profit margin gained by processing soybeans into soy oil. Soybeans form a large proportion (over 1/5th) of the agricultural output of US farmers and the profit margins gained will therefore have a wide impact on the US economy in general. The chapter uses a number of techniques to forecast and trade the soybean crush spread. A traditional regression analysis is used as a benchmark against more sophisticated models such as a Multi-Layer Perceptron (MLP), Recurrent Neural Networks and Higher Order Neural Networks. These are then used to trade the spread, the implementation of a number of filtering techniques as used in the literature are utilized to further refine the trading statistics of the models. The results show that the best model before transactions costs both in- and out-of-sample is the Recurrent Network generating a superior risk adjusted return to all other models investigated. However in the case of most of the models investigated the cost of trading the spread all but eliminates any profit potential.
Chapter 17, “Fundamental Theory of Artificial Higher Order Neural Networks”, aims to describe fundamental principles of artificial higher order neural units (AHONUs) and networks (AHONNs). An essential core of AHONNs can be found in higher order weighted combinations or correlations between the input variables. By using some typical examples, this chapter describes how and why higher order combinations or correlations can be effective.
Chapter 18, “Dynamics in Artificial Higher Order Neural Networks with Delays”, concentrates on studying the dynamics of artificial higher order neural networks (HONNs) with delays. Both stability analysis and periodic oscillation are discussed here for a class of delayed HONNs with (or without) impulses. Most of the sufficient conditions obtained in this chapter are presented in linear matrix inequalities (LMIs), and so can be easily computed and checked in practice using the Matlab LMI Toolbox. In reality, stability is a necessary feature when applying artificial neural networks. Also periodic solution plays an important role in the dynamical behavior of all solutions though other dynamics such as bifurcation and chaos do coexist. So here we mainly focus on questions of the stability and periodic solutions of artificial HONNs with (or without) impulses. Firstly, stability analysis and periodic oscillation are analyzed for higher order bidirectional associative memory (BAM) neural networks without impulses. Secondly, global exponential stability and exponential convergence are studied for a class of impulsive higher order bidirectional associative memory neural networks with time-varying delays. The main methods and tools used in this chapter are linear matrix inequalities (LMIs), Lyapunov stability theory and coincidence degree theory.
Chapter 19, “A New Topology for Artificial Higher Order Neural Networks - Polynomial Kernel Networks”, is aiming to develop a systematic approach for optimizing the structure of artificial higher order neural networks (HONN) for system modeling and function approximation, a new HONN topology, namely polynomial kernel networks, is proposed in this chapter. Structurally, the polynomial kernel network can be viewed as a three-layer feed-forward neural network with a special polynomial activation function for the nodes in the hidden layer. The new network is equivalent to a HONN; however, due to the underlying connections with polynomial kernel support vector machines, the weights and the structure of the network can be determined simultaneously using structural risk minimization. The advantage of the topology of the polynomial kernel network and the use of a support vector kernel expansion paves the way to represent nonlinear functions or systems, and underpins some advanced analysis of the network performance. In this chapter, from the perspective of network complexity, both quadratic programming and linear programming based training of the polynomial kernel network are investigated.
Chapter 20, “High Speed Optical Higher Order Neural Networks for Discovering Data Trends and Patterns in Very Large Database”, describes the progress in using optical technology to construct high-speed artificial higher order neural network systems. The chapter reviews how optical technology can speed up searches within large databases in order to identify relationships and dependencies between individual data records, such as financial or business time-series, as well as trends and relationships within them. Two distinct approaches in which optics may be used are reviewed. In the first approach, the chapter reviews current research replacing copper connections in a conventional data storage system, such as a several terabyte RAID array of magnetic hard discs, by optical waveguides to achieve very high data rates with low crosstalk interference. In the second approach, the chapter reviews how high speed optical correlators with feedback can be used to realize artificial higher order neural networks using Fourier Transform free space optics and holographic database storage.
Chapter 21, “On Complex Artificial Higher Order Neural Networks: Dealing with Stochasticity, Jumps and Delays”, deals with the analysis problem of the global exponential stability for a general class of stochastic artificial higher order neural networks with multiple mixed time delays and Markovian jumping parameters. The mixed time delays under consideration comprise both the discrete time-varying delays and the distributed time-delays. The main purpose of this chapter is to establish easily verifiable conditions under which the delayed high-order stochastic jumping neural network is exponentially stable in the mean square in the presence of both the mixed time delays and Markovian switching. By employing a new Lyapunov-Krasovskii functional and conducting stochastic analysis, a linear matrix nequality (LMI) approach is developed to derive the criteria ensuring the exponential stability. Furthermore, the criteria are dependent on both the discrete time-delay and distributed time-delay, hence less conservative. The proposed criteria can be readily checked by using some standard numerical packages such as the Matlab LMI Toolbox. A simple example is provided to demonstrate the effectiveness and applicability of the proposed testing criteria.
Chapter 22, “Trigonometric Polynomial Higher Order Neural Network Group Models and Weighted Kernel Models for Financial Data Simulation and Prediction”, introduces trigonometric polynomial higher order neural network models. In the area of financial data simulation and prediction, there is no single neural network model that could handle the wide variety of data and perform well in the real world. A way of solving this difficulty is to develop a number of new models, with different algorithms. A wider variety of models would give financial operators more chances to find a suitable model when they process their data. That was the major motivation for this chapter. The theoretical principles of these improved models are presented and demonstrated and experiments are conducted by using real-life financial data.