A State-Based Sensitivity Analysis for Distinguishing the Global Importance of Predictor Variables in Artificial Neural Networks

Artificial neural networks (ANNs) are powerful empirical approaches used to model databases with a high degree of accuracy. Despite their recognition as universal approximators, many practitioners are skeptical about adopting their routine usage due to lack of model transparency. To improve the clarity of model prediction and correct the apparent lack of comprehension, researchers have utilized a variety of methodologies to extract the underlying variable relationships within ANNs, such as sensitivity analysis (SA). The theoretical basis of local SA (that predictors are independent and inputs other than variable of interest remain “fixed” at predefined values) is challenged in global SA, where, in addition to altering the attribute of interest, the remaining predictors are varied concurrently across their respective ranges. Here, a regression-based global methodology, state-based sensitivity analysis (SBSA), is proposed for measuring the importance of predictor variables upon a modeled response within ANNs. SBSA was applied to network models of a synthetic database having a defined structure and exhibiting multicollinearity. SBSA achieved the most accurate portrayal of predictor-response relationships (compared to local SA and Connected Weights Analysis), closely approximating the actual variability of the modeled system. From this, it is anticipated that skepticisms concerning the delineation of predictor influences and their uncertainty domains upon a modeled output within ANNs will be curtailed.

Modified STDP Triplet Rule Significantly Increases Neuron Training Stability in the Learning of Spatial Patterns

Spike-timing-dependent plasticity (STDP) is a set of Hebbian learning rules which are based firmly on biological evidence. STDP learning is capable of detecting spatiotemporal patterns highly obscured by noise. This feature appears attractive from the point of view of machine learning. In this paper three different additive STDP models of spike interactions were compared in respect to training performance when the neuron is exposed to a recurrent spatial pattern injected into Poisson noise. The STDP models compared were all-to-all interaction, nearest-neighbor interaction, and the nearest-neighbor triplet interaction. The parameters of the neuron model and STDP training rules were optimized for a range of spatial patterns of different sizes by the means of heuristic algorithm. The size of the pattern, that is, the number of synapses containing the pattern, was gradually decreased from what amounted to a relatively easy task down to a single synapse. Optimization was performed for each size of the pattern. The parameters were allowed to evolve freely. The triplet rule, in most cases, performed better by far than the other two rules, while the evolutionary algorithm immediately switched the polarity of the triplet update. The all-to-all rule achieved moderate results.

Artificial Neural Network Estimation of Thermal Insulation Value of Children’s School Wear in Kuwait Classroom

Artificial neural network (ANN) was utilized to predict the thermal insulation values of children’s school wear in Kuwait. The input thermal insulation data of the different children’s school wear used in Kuwait classrooms were obtained from study using thermal manikins. The lowest mean squared error (MSE) value for the validation data was 1.5 × 10−5 using one hidden layer of six neurons and one output layer. The R2 values for the training, validation, and testing data were almost equal to 1. The values from ANN prediction were compared with McCullough’s equation and the standard tables’ methods. Results suggested that the ANN is able to give more accurate prediction of the clothing thermal insulation values than the regression equation and the standard tables methods. The effect of the different input variables on the thermal insulation value was examined using Garson algorithm and sensitivity analysis and it was found that the cloths weight, the body surface area nude (BSA0), and body surface area covered by one layer of clothing (BSAC1) have the highest effect on the thermal insulation value with about 29%, 27%, and 23%, respectively.

Water Quality Modeling in Reservoirs Using Multivariate Linear Regression and Two Neural Network Models

In this study, two artificial neural network models (i.e., a radial basis function neural network, RBFN, and an adaptive neurofuzzy inference system approach, ANFIS) and a multilinear regression (MLR) model were developed to simulate the DO, TP, Chl a, and SD in the Mingder Reservoir of central Taiwan. The input variables of the neural network and the MLR models were determined using linear regression. The performances were evaluated using the RBFN, ANFIS, and MLR models based on statistical errors, including the mean absolute error, the root mean square error, and the correlation coefficient, computed from the measured and the model-simulated DO, TP, Chl a, and SD values. The results indicate that the performance of the ANFIS model is superior to those of the MLR and RBFN models. The study results show that the neural network using the ANFIS model is suitable for simulating the water quality variables with reasonable accuracy, suggesting that the ANFIS model can be used as a valuable tool for reservoir management in Taiwan.

Application of Neural Network Modeling to Identify Auditory Processing Disorders in School-Age Children

P300 Auditory Event-Related Potentials (P3AERPs) were recorded in nine school-age children with auditory processing disorders and nine age- and gender-matched controls in response to tone burst stimuli presented at varying rates (1/second or 3/second) under varying levels of competing noise (0 dB, 40 dB, or 60 dB SPL). Neural network modeling results indicated that speed of information processing and task-related demands significantly influenced P3AERP latency in children with auditory processing disorders. Competing noise and rapid stimulus rates influenced P3AERP amplitude in both groups.

Generalisation over Details: The Unsuitability of Supervised Backpropagation Networks for Tetris

We demonstrate the unsuitability of Artificial Neural Networks (ANNs) to the game of Tetris and show that their great strength, namely, their ability of generalization, is the ultimate cause. This work describes a variety of attempts at applying the Supervised Learning approach to Tetris and demonstrates that these approaches (resoundedly) fail to reach the level of performance of hand-crafted Tetris solving algorithms. We examine the reasons behind this failure and also demonstrate some interesting auxiliary results. We show that training a separate network for each Tetris piece tends to outperform the training of a single network for all pieces; training with randomly generated rows tends to increase the performance of the networks; networks trained on smaller board widths and then extended to play on bigger boards failed to show any evidence of learning, and we demonstrate that ANNs trained via Supervised Learning are ultimately ill-suited to Tetris.

Sensorless Direct Power Control of Induction Motor Drive Using Artificial Neural Network

This paper proposes the design of sensorless induction motor drive based on direct power control (DPC) technique. It is shown that DPC technique enjoys all advantages of pervious methods such as fast dynamic and ease of implementation, without having their problems. To reduce the cost of drive and enhance the reliability, an effective sensorless strategy based on artificial neural network (ANN) is developed to estimate rotor’s position and speed of induction motor. Developed sensorless scheme is a new model reference adaptive system (MRAS) speed observer for direct power control induction motor drives. The proposed MRAS speed observer uses the current model as an adaptive model. The neural network has been then designed and trained online by employing a back propagation network (BPN) algorithm. The estimator was designed and simulated in Simulink. Some simulations are carried out for the closed-loop speed control systems under various load conditions to verify the proposed methods. Simulation results confirm the performance of ANN based sensorless DPC induction motor drive in various conditions.

Stochastic Search Algorithms for Identification, Optimization, and Training of Artificial Neural Networks

This paper presents certain stochastic search algorithms (SSA) suitable for effective identification, optimization, and training of artificial neural networks (ANN). The modified algorithm of nonlinear stochastic search (MN-SDS) has been introduced by the author. Its basic objectives are to improve convergence property of the source defined nonlinear stochastic search (N-SDS) method as per Professor Rastrigin. Having in mind vast range of possible algorithms and procedures a so-called method of stochastic direct search (SDS) has been practiced (in the literature is called stochastic local search-SLS). The MN-SDS convergence property is rather advancing over N-SDS; namely it has even better convergence over range of gradient procedures of optimization. The SDS, that is, SLS, has not been practiced enough in the process of identification, optimization, and training of ANN. Their efficiency in some cases of pure nonlinear systems makes them suitable for optimization and training of ANN. The presented examples illustrate only partially operatively end efficiency of SDS, that is, MN-SDS. For comparative method backpropagation error (BPE) method was used.

Hybrid Feature Selection Based Weighted Least Squares Twin Support Vector Machine Approach for Diagnosing Breast Cancer, Hepatitis, and Diabetes

There is a necessity for analysis of a large amount of data in many fields such as healthcare, business, industries, and agriculture. Therefore, the need of the feature selection (FS) technique for the researchers is quite evident in many fields of science, especially in computer science. Furthermore, an effective FS technique that is best suited to a particular learning algorithm is of great help for the researchers. Hence, this paper proposes a hybrid feature selection (HFS) based efficient disease diagnostic model for Breast Cancer, Hepatitis, and Diabetes. A HFS is an efficient method that combines the positive aspects of both Filter and Wrapper FS approaches. The proposed model adopts weighted least squares twin support vector machine (WLSTSVM) as a classification approach, sequential forward selection (SFS) as a search strategy, and correlation feature selection (CFS) to evaluate the importance of each feature. This model not only selects relevant feature subset but also efficiently deals with the data imbalance problem. The effectiveness of the HFS based WLSTSVM approach is examined on three well-known disease datasets taken from UCI repository with the help of predictive accuracy, sensitivity, specificity, and geometric mean. The experiment confirms that our proposed HFS based WLSTSVM disease diagnostic model can result in positive outcomes.

Neural Virtual Sensors for Adaptive Magnetic Localization of Autonomous Dataloggers

The surging advance in micro- and nanotechnologies allied with neural learning systems allows the realization of miniaturized yet extremely powerful multisensor systems and networks for wide application fields, for example, in measurement, instrumentation, automation, and smart environments. Time and location context is particularly relevant to sensor swarms applied for distributed measurement in industrial environment, such as, for example, fermentation tanks. Common RF solutions face limits here, which can be overcome by magnetic systems. Previously, we have developed the electronic system for an integrated data logger swarm with magnetic localization and sensor node timebase synchronization. The focus of this work is on an approach to improving both localization accuracy and flexibility by the application of artificial neural networks applied as virtual sensors and classifiers in a hybrid dedicated learning system. Including also data from an industrial brewery environment, the best investigated neural virtual sensor approach has achieved an advance in localization accuracy of a factor of 4 compared to state-of-the-art numerical methods and, thus, results in the order of less than 5 cm meeting industrial expectations on a feasible solution for the presented integrated localization system solution.