scholarly journals Efficient Calculation of the Gauss-Newton Approximation of the Hessian Matrix in Neural Networks

2012 ◽  
Vol 24 (3) ◽  
pp. 607-610 ◽  
Author(s):  
Michael Fairbank ◽  
Eduardo Alonso
Keyword(s):  
Neural Networks ◽  
Learning Algorithm ◽  
Hessian Matrix ◽  
Matrix Multiplication ◽  
Running Time ◽  
The Neural Network ◽  
Efficient Calculation ◽  
Levenberg Marquardt ◽  
Time Critical ◽  
Lm Algorithm

The Levenberg-Marquardt (LM) learning algorithm is a popular algorithm for training neural networks; however, for large neural networks, it becomes prohibitively expensive in terms of running time and memory requirements. The most time-critical step of the algorithm is the calculation of the Gauss-Newton matrix, which is formed by multiplying two large Jacobian matrices together. We propose a method that uses backpropagation to reduce the time of this matrix-matrix multiplication. This reduces the overall asymptotic running time of the LM algorithm by a factor of the order of the number of output nodes in the neural network.

Study on Glass Furance Temperature Dual Control Based on Neural Networks Decoupling Control under WINCC

2013 ◽  
Vol 341-342 ◽  
pp. 856-860
Author(s):  
Hao Ming Yang ◽  
Lan Qing Zhang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Learning Algorithm ◽  
Decoupling Control ◽  
Dual Control ◽  
Script Language ◽  
Learning Speed ◽  
The Neural Network ◽  
Levenberg Marquardt ◽  
Experiment Control

Experiment control platform for the neural network decoupling control is constructed for the glass furnace taking heavy oil as fuel. By dual control, the improving Levenberg-Marquardt learning algorithm is discussed in order to improve the learning speed and to satisfy the real control. The neural network decoupling real control based on C-Script language and PLC S7-400 hard system under WINCC is realized with satisfying control results.

scholarly journals Use of a Multilayer Perceptron to Automate Terrain Assessment for the Needs of the Armed Forces

2018 ◽  
Vol 7 (11) ◽  
pp. 430 ◽  
Author(s):  
Krzysztof Pokonieczny
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Multilayer Perceptron ◽  
Learning Algorithm ◽  
Armed Forces ◽  
Shuttle Radar Topography Mission ◽  
Terrain Classification ◽  
The Neural Network ◽  
Data Configuration ◽  
Trained Neural Network

The classification of terrain in terms of passability plays a significant role in the process of military terrain assessment. It involves classifying selected terrain to specific classes (GO, SLOW-GO, NO-GO). In this article, the problem of terrain classification to the respective category of passability was solved by applying artificial neural networks (multilayer perceptron) to generate a continuous Index of Passability (IOP). The neural networks defined this factor for primary fields in two sizes (1000 × 1000 m and 100 × 100 m) based on the land cover elements obtained from Vector Smart Map (VMap) Level 2 and Shuttle Radar Topography Mission (SRTM). The work used a feedforward neural network consisting of three layers. The paper presents a comprehensive analysis of the reliability of the neural network parameters, taking into account the number of neurons, learning algorithm, activation functions and input data configuration. The studies and tests carried out have shown that a well-trained neural network can automate the process of terrain classification in terms of passability conditions.

scholarly journals Interpretation of Neural Networks Is Fragile

2019 ◽  
Vol 33 ◽  
pp. 3681-3688 ◽  
Author(s):  
Amirata Ghorbani ◽  
Abubakar Abid ◽  
James Zou
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Input Data ◽  
Learning Algorithm ◽  
Hessian Matrix ◽  
Machine Learning Algorithm ◽  
Feature Importance ◽  
Adversarial Attack ◽  
Measurement Biases

In order for machine learning to be trusted in many applications, it is critical to be able to reliably explain why the machine learning algorithm makes certain predictions. For this reason, a variety of methods have been developed recently to interpret neural network predictions by providing, for example, feature importance maps. For both scientific robustness and security reasons, it is important to know to what extent can the interpretations be altered by small systematic perturbations to the input data, which might be generated by adversaries or by measurement biases. In this paper, we demonstrate how to generate adversarial perturbations that produce perceptively indistinguishable inputs that are assigned the same predicted label, yet have very different interpretations. We systematically characterize the robustness of interpretations generated by several widely-used feature importance interpretation methods (feature importance maps, integrated gradients, and DeepLIFT) on ImageNet and CIFAR-10. In all cases, our experiments show that systematic perturbations can lead to dramatically different interpretations without changing the label. We extend these results to show that interpretations based on exemplars (e.g. influence functions) are similarly susceptible to adversarial attack. Our analysis of the geometry of the Hessian matrix gives insight on why robustness is a general challenge to current interpretation approaches.

scholarly journals Diagnosing Diabetes Using Artificial Neural Networks

2020 ◽  
Vol 5 (2) ◽  
pp. 221-224
Author(s):  
Joy Oyinye Orukwo ◽  
Ledisi Giok Kabari
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Success Rate ◽  
Medical Diagnosis ◽  
Learning Algorithm ◽  
Diabetes Diagnosis ◽  
Feed Forward Neural Network ◽  
The Neural Network ◽  
Artificial Neural

Diabetes has always been a silent killer and the number of people suffering from it has increased tremendously in the last few decades. More often than not, people continue with their normal lifestyle, unaware that their health is at severe risk and with each passing day diabetes goes undetected. Artificial Neural Networks have become extensively useful in medical diagnosis as it provides a powerful tool to help analyze, model and make sense of complex clinical data. This study developed a diabetes diagnosis system using feed-forward neural network with supervised learning algorithm. The neural network is systematically trained and tested and a success rate of 90% was achieved.

scholarly journals Image correction for cone-beam computed tomography simulator using neural network corrector

2017 ◽  
Vol 9 (2) ◽  
pp. 168781401769047
Author(s):  
Chin-Sheng Chen ◽  
Cheng-Yi Hsu ◽  
Shih-Kang Chen ◽  
Chih-Jer Lin ◽  
Ching-Hao Hsieh ◽  
...  
Keyword(s):  
Neural Network ◽  
Computed Tomography ◽  
Cone Beam Computed Tomography ◽  
Learning Algorithm ◽  
Central Point ◽  
Cone Beam ◽  
Measure Data ◽  
Image Correction ◽  
The Neural Network ◽  
Levenberg Marquardt

In this article, a neural network corrector is proposed to correct the image shift, yielding the degradation of three-dimensional image reconstruction, for each slice captured by cone-beam computed tomography simulator. There are 3 degrees of freedom in tube module of simulator; the central point of tube module should be aligned with the central point of detector module to guarantee the accurate image projection. However, the mechanism manufacturing and assembling tolerance will let the above aim cannot be met. Here, a standard kit is made to measure the image shift by 1° step from −10° to 10°. The measure data will be the input training data of proposed neural network corrector, and the corrected translation position will be the output of neural network corrector. The Levenberg–Marquardt learning algorithm adjusts the connected weights and biases of the neural network using a supervised gradient descent method, such that the defined error function can be minimized. To avoid the problem of overfitting and improve the generalized ability of the neural network, Bayesian regularization is added to the Levenberg–Marquardt learning algorithm. After the training of neural network corrector, the different target position commands are fed into the neural network corrector. Then, the corrected data from neural network corrector are fed to be the new position command to verify the image correction performance. Moreover, a phantom kit is made to check the corrected performance of the neural network corrector. Finally, the experimental results verify that the image shift can be reduced by the neural network corrector.

CHARACTERIZING ONE-LAYER ASSOCIATIVE NEURAL NETWORKS WITH OPTIMAL NOISE-REDUCTION ABILITY

1992 ◽  
Vol 06 (05) ◽  
pp. 1009-1025 ◽  
Author(s):  
TAO WANG ◽  
XIAOLIANG XING ◽  
XINHUA ZHUANG
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Cost Function ◽  
Noise Reduction ◽  
Gradient Descent ◽  
Storage Capacity ◽  
Learning Algorithm ◽  
Optimal Learning ◽  
The Neural Network ◽  
The Cost

In this paper, we describe an optimal learning algorithm for designing one-layer neural networks by means of global minimization. Taking the properties of a well-defined neural network into account, we derive a cost function to measure the goodness of the network quantitatively. The connection weights are determined by the gradient descent rule to minimize the cost function. The optimal learning algorithm is formed as either the unconstraint-based or the constraint-based minimization problem. It ensures the realization of each desired associative mapping with the best noise reduction ability in the sense of optimization. We also investigate the storage capacity of the neural network, the degree of noise reduction for a desired associative mapping, and the convergence of the learning algorithm in an analytic way. Finally, a large number of computer experimental results are presented.

scholarly journals Predicting Customer Turnover Using Recursive Neural Networks

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Abdullah Jafari Chashmi ◽  
Vahid Rahmati ◽  
Behrouz Rezasoroush ◽  
Masumeh Motevalli Alamoti ◽  
Mohsen Askari ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Learning Algorithm ◽  
Customer Relationship ◽  
Loyalty Programs ◽  
Recurrent Nerve ◽  
The Neural Network ◽  
Recursive Neural Networks ◽  
The Neural Networks ◽  
A Company

The most valuable asset for a company is its customers’ base. As a result, customer relationship management (CRM) is an important task that drives companies. By identifying and understanding the valuable customer segments, appropriate marketing strategies can be used to enhance customer satisfaction and maintain loyalty, as well as increase company retention. Predicting customer turnover is an important tool for companies to stay competitive in a fast-growing market. In this paper, we use the recurrent nerve sketch to predict rejection based on the time series of the lifetime of the customer. In anticipation, a key aspect of identifying key triggers is to turn off. To overcome the weakness of recurrent neural networks, the research model of the combination of LRFMP with the neural network has been used. In this paper, it was found that clustering by LRFMP can be used to perform a more comprehensive analysis of customers’ turnover. In this solution, LRFMP is used to execute customer segregation. The objective is to provide a new framework for LRFMP for macrodata and macrodata analysis in order to increase the problem of business problem solving and customer depreciation. The results of the research show that the neural networks are capable of predicting the LRFMP precursors of the customers in an effective way. This model can be used in advocacy systems for advertising and loyalty programs management. In the previous research, the LRFM and RFM algorithms along with the neural network and the machine learning algorithm, etc., have been used, and in the proposed solution, the use of the LRFMP algorithm increases the accuracy of the desired.

scholarly journals Automatic Defects Segmentation and Identification by Deep Learning Algorithm with Pulsed Thermography: Synthetic and Experimental Data

2021 ◽  
Vol 5 (1) ◽  
pp. 9
Author(s):  
Qiang Fang ◽  
Clemente Ibarra-Castanedo ◽  
Xavier Maldague
Keyword(s):  
Neural Network ◽  
Experimental Data ◽  
Neural Networks ◽  
Deep Learning ◽  
Infrared Thermography ◽  
Learning Algorithm ◽  
Synthetic Data ◽  
Training Data ◽  
Experimental Database ◽  
The Neural Network

In quality evaluation (QE) of the industrial production field, infrared thermography (IRT) is one of the most crucial techniques used for evaluating composite materials due to the properties of low cost, fast inspection of large surfaces, and safety. The application of deep neural networks tends to be a prominent direction in IRT Non-Destructive Testing (NDT). During the training of the neural network, the Achilles heel is the necessity of a large database. The collection of huge amounts of training data is the high expense task. In NDT with deep learning, synthetic data contributing to training in infrared thermography remains relatively unexplored. In this paper, synthetic data from the standard Finite Element Models are combined with experimental data to build repositories with Mask Region based Convolutional Neural Networks (Mask-RCNN) to strengthen the neural network, learning the essential features of objects of interest and achieving defect segmentation automatically. These results indicate the possibility of adapting inexpensive synthetic data merging with a certain amount of the experimental database for training the neural networks in order to achieve the compelling performance from a limited collection of the annotated experimental data of a real-world practical thermography experiment.

Sign in / Sign up

Export Citation Format

Share Document