CHARACTERIZING ONE-LAYER ASSOCIATIVE NEURAL NETWORKS WITH OPTIMAL NOISE-REDUCTION ABILITY

1992 ◽  
Vol 06 (05) ◽  
pp. 1009-1025 ◽  
Author(s):  
TAO WANG ◽  
XIAOLIANG XING ◽  
XINHUA ZHUANG
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Cost Function ◽  
Noise Reduction ◽  
Gradient Descent ◽  
Storage Capacity ◽  
Learning Algorithm ◽  
Optimal Learning ◽  
The Neural Network ◽  
The Cost

In this paper, we describe an optimal learning algorithm for designing one-layer neural networks by means of global minimization. Taking the properties of a well-defined neural network into account, we derive a cost function to measure the goodness of the network quantitatively. The connection weights are determined by the gradient descent rule to minimize the cost function. The optimal learning algorithm is formed as either the unconstraint-based or the constraint-based minimization problem. It ensures the realization of each desired associative mapping with the best noise reduction ability in the sense of optimization. We also investigate the storage capacity of the neural network, the degree of noise reduction for a desired associative mapping, and the convergence of the learning algorithm in an analytic way. Finally, a large number of computer experimental results are presented.

LEARNING OF THE HOPFIELD ASSOCIATIVE MEMORY BY GLOBAL MINIMIZATION

1994 ◽  
Vol 08 (01) ◽  
pp. 373-390
Author(s):  
TAO WANG
Keyword(s):  
Gradient Descent ◽  
Storage Capacity ◽  
Learning Algorithm ◽  
Basin Of Attraction ◽  
Basins Of Attraction ◽  
Connection Matrix ◽  
Global Minimization ◽  
Associative Memories ◽  
Optimal Learning ◽  
The Cost

In the paper, a learning algorithm for Hopfield associative memories (HAMs) is presented. According to the cost function that measures the goodness of the HAM, we determine the connection matrix using a global minimization, solved by a gradient descent rule. This optimal learning method can guarantee the storage of all training patterns with basins of attraction that are as large as possible. We also study the storage capacity of the HAM, the asymptotic stability of each training pattern and its basin of attraction. A large number of computer simulations have been conducted to show its performance.

scholarly journals Use of a Multilayer Perceptron to Automate Terrain Assessment for the Needs of the Armed Forces

2018 ◽  
Vol 7 (11) ◽  
pp. 430 ◽  
Author(s):  
Krzysztof Pokonieczny
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Multilayer Perceptron ◽  
Learning Algorithm ◽  
Armed Forces ◽  
Shuttle Radar Topography Mission ◽  
Terrain Classification ◽  
The Neural Network ◽  
Data Configuration ◽  
Trained Neural Network

The classification of terrain in terms of passability plays a significant role in the process of military terrain assessment. It involves classifying selected terrain to specific classes (GO, SLOW-GO, NO-GO). In this article, the problem of terrain classification to the respective category of passability was solved by applying artificial neural networks (multilayer perceptron) to generate a continuous Index of Passability (IOP). The neural networks defined this factor for primary fields in two sizes (1000 × 1000 m and 100 × 100 m) based on the land cover elements obtained from Vector Smart Map (VMap) Level 2 and Shuttle Radar Topography Mission (SRTM). The work used a feedforward neural network consisting of three layers. The paper presents a comprehensive analysis of the reliability of the neural network parameters, taking into account the number of neurons, learning algorithm, activation functions and input data configuration. The studies and tests carried out have shown that a well-trained neural network can automate the process of terrain classification in terms of passability conditions.

scholarly journals Diagnosing Diabetes Using Artificial Neural Networks

2020 ◽  
Vol 5 (2) ◽  
pp. 221-224
Author(s):  
Joy Oyinye Orukwo ◽  
Ledisi Giok Kabari
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Success Rate ◽  
Medical Diagnosis ◽  
Learning Algorithm ◽  
Diabetes Diagnosis ◽  
Feed Forward Neural Network ◽  
The Neural Network ◽  
Artificial Neural

Diabetes has always been a silent killer and the number of people suffering from it has increased tremendously in the last few decades. More often than not, people continue with their normal lifestyle, unaware that their health is at severe risk and with each passing day diabetes goes undetected. Artificial Neural Networks have become extensively useful in medical diagnosis as it provides a powerful tool to help analyze, model and make sense of complex clinical data. This study developed a diabetes diagnosis system using feed-forward neural network with supervised learning algorithm. The neural network is systematically trained and tested and a success rate of 90% was achieved.

Study on Glass Furance Temperature Dual Control Based on Neural Networks Decoupling Control under WINCC

2013 ◽  
Vol 341-342 ◽  
pp. 856-860
Author(s):  
Hao Ming Yang ◽  
Lan Qing Zhang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Learning Algorithm ◽  
Decoupling Control ◽  
Dual Control ◽  
Script Language ◽  
Learning Speed ◽  
The Neural Network ◽  
Levenberg Marquardt ◽  
Experiment Control

Experiment control platform for the neural network decoupling control is constructed for the glass furnace taking heavy oil as fuel. By dual control, the improving Levenberg-Marquardt learning algorithm is discussed in order to improve the learning speed and to satisfy the real control. The neural network decoupling real control based on C-Script language and PLC S7-400 hard system under WINCC is realized with satisfying control results.

scholarly journals Predicting Customer Turnover Using Recursive Neural Networks

2021 ◽  
Vol 2021 ◽  
pp. 1-11
Author(s):  
Abdullah Jafari Chashmi ◽  
Vahid Rahmati ◽  
Behrouz Rezasoroush ◽  
Masumeh Motevalli Alamoti ◽  
Mohsen Askari ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Learning Algorithm ◽  
Customer Relationship ◽  
Loyalty Programs ◽  
Recurrent Nerve ◽  
The Neural Network ◽  
Recursive Neural Networks ◽  
The Neural Networks ◽  
A Company

The most valuable asset for a company is its customers’ base. As a result, customer relationship management (CRM) is an important task that drives companies. By identifying and understanding the valuable customer segments, appropriate marketing strategies can be used to enhance customer satisfaction and maintain loyalty, as well as increase company retention. Predicting customer turnover is an important tool for companies to stay competitive in a fast-growing market. In this paper, we use the recurrent nerve sketch to predict rejection based on the time series of the lifetime of the customer. In anticipation, a key aspect of identifying key triggers is to turn off. To overcome the weakness of recurrent neural networks, the research model of the combination of LRFMP with the neural network has been used. In this paper, it was found that clustering by LRFMP can be used to perform a more comprehensive analysis of customers’ turnover. In this solution, LRFMP is used to execute customer segregation. The objective is to provide a new framework for LRFMP for macrodata and macrodata analysis in order to increase the problem of business problem solving and customer depreciation. The results of the research show that the neural networks are capable of predicting the LRFMP precursors of the customers in an effective way. This model can be used in advocacy systems for advertising and loyalty programs management. In the previous research, the LRFM and RFM algorithms along with the neural network and the machine learning algorithm, etc., have been used, and in the proposed solution, the use of the LRFMP algorithm increases the accuracy of the desired.

scholarly journals Estimation of Overtopping Discharges with Deep Neural Network(DNN) Method

2021 ◽  
Vol 8 (4) ◽  
pp. 229-236
Author(s):  
Changkyum Kim ◽  
Insik Chun ◽  
Byungcheol Oh
Keyword(s):  
Neural Network ◽  
Artificial Intelligence ◽  
Cost Function ◽  
Deep Neural Network ◽  
Multiple Linear Regression Model ◽  
Predictive Performance ◽  
Data Sets ◽  
Network Training ◽  
The Neural Network ◽  
The Cost

An Artificial Intelligence(AI) study was conducted to calculate overtopping discharges for various coastal structures. The Deep Neural Network(DNN), one of the artificial intelligence methods, was employed in the study. The neural network was trained, validated and tested using the EurOtop database containing the experimental data collected from all over the world. To improve the accuracy of the deep neural network results, all data were non-dimensionalized and max-min normalized as a preprocessing process. L2 regularization was also introduced in the cost function to secure the convergence of iterative learning, and the cost function was optimized using RMSProp and Adam techniques. In order to compare the performance of DNN, additional calculations based on the multiple linear regression model and EurOtop’s overtopping formulas were done as well, using the data sets which were not included in the network training. The results showed that the predictive performance of the AI technique was relatively superior to the two other methods.

NEURAL NETWORK CONTROL SYSTEMS THAT LEARN TO PERFORM APPROPRIATELY

2001 ◽  
Vol 11 (01) ◽  
pp. 79-88 ◽  
Author(s):  
JOHN A. BULLINARIA ◽  
PATRICIA M. RIDDELL
Keyword(s):  
Neural Network ◽  
Control Systems ◽  
Gradient Descent ◽  
Learning Algorithm ◽  
Network Control ◽  
Oculomotor Control ◽  
Evolution And Development ◽  
Learning Rates ◽  
Network Control Systems ◽  
The Cost

Setting up a neural network with a learning algorithm that determines how it can best operate is an efficient way to formulate control systems for many engineering applications, and is often much more feasible than direct programming. This paper examines three important aspects of this approach: the details of the cost function that is used with the gradient descent learning algorithm, how the resulting system depends on the initial pre-learning connection weights, and how the resulting system depends on the pattern of learning rates chosen for the different components of the system. We explore these issues by explicit simulations of a toy model that is a simplified abstraction of part of the human oculomotor control system. This allows us to compare our system with that produced by human evolution and development. We can then go on to consider how we might improve on the human system and apply what we have learnt to control systems that have no human analogue.

scholarly journals Automatic Defects Segmentation and Identification by Deep Learning Algorithm with Pulsed Thermography: Synthetic and Experimental Data

2021 ◽  
Vol 5 (1) ◽  
pp. 9
Author(s):  
Qiang Fang ◽  
Clemente Ibarra-Castanedo ◽  
Xavier Maldague
Keyword(s):  
Neural Network ◽  
Experimental Data ◽  
Neural Networks ◽  
Deep Learning ◽  
Infrared Thermography ◽  
Learning Algorithm ◽  
Synthetic Data ◽  
Training Data ◽  
Experimental Database ◽  
The Neural Network

In quality evaluation (QE) of the industrial production field, infrared thermography (IRT) is one of the most crucial techniques used for evaluating composite materials due to the properties of low cost, fast inspection of large surfaces, and safety. The application of deep neural networks tends to be a prominent direction in IRT Non-Destructive Testing (NDT). During the training of the neural network, the Achilles heel is the necessity of a large database. The collection of huge amounts of training data is the high expense task. In NDT with deep learning, synthetic data contributing to training in infrared thermography remains relatively unexplored. In this paper, synthetic data from the standard Finite Element Models are combined with experimental data to build repositories with Mask Region based Convolutional Neural Networks (Mask-RCNN) to strengthen the neural network, learning the essential features of objects of interest and achieving defect segmentation automatically. These results indicate the possibility of adapting inexpensive synthetic data merging with a certain amount of the experimental database for training the neural networks in order to achieve the compelling performance from a limited collection of the annotated experimental data of a real-world practical thermography experiment.

scholarly journals Using an extended Kalman filter learning algorithm for feed-forward neural networks to describe tracer correlations

2004 ◽  
Vol 4 (3) ◽  
pp. 3653-3667 ◽  
Author(s):  
D. J. Lary ◽  
H. Y. Mussa
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Kalman Filter ◽  
Extended Kalman Filter ◽  
Learning Algorithm ◽  
Feed Forward ◽  
The Neural Network ◽  
Fortran Code ◽  
Feed Forward Neural Networks ◽  
Time Of Year

Abstract. In this study a new extended Kalman filter (EKF) learning algorithm for feed-forward neural networks (FFN) is used. With the EKF approach, the training of the FFN can be seen as state estimation for a non-linear stationary process. The EKF method gives excellent convergence performances provided that there is enough computer core memory and that the machine precision is high. Neural networks are ideally suited to describe the spatial and temporal dependence of tracer-tracer correlations. The neural network performs well even in regions where the correlations are less compact and normally a family of correlation curves would be required. For example, the CH4-N2O correlation can be well described using a neural network trained with the latitude, pressure, time of year, and CH4 volume mixing ratio (v.m.r.). The neural network was able to reproduce the CH4-N2O correlation with a correlation coefficient between simulated and training values of 0.9997. The neural network Fortran code used is available for download.

scholarly journals Direct Quantization for Training Highly Accurate Low Bit-width Deep Neural Networks

2020 ◽  
Author(s):  
Tuan Hoang ◽  
Thanh-Toan Do ◽  
Tam V. Nguyen ◽  
Ngai-Man Cheung
Keyword(s):  
Neural Networks ◽  
Cost Function ◽  
Image Classification ◽  
Convolutional Neural Networks ◽  
Gradient Descent ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Deep Convolutional Neural Networks ◽  
Novel Method ◽  
The Cost

This paper proposes two novel techniques to train deep convolutional neural networks with low bit-width weights and activations. First, to obtain low bit-width weights, most existing methods obtain the quantized weights by performing quantization on the full-precision network weights. However, this approach would result in some mismatch: the gradient descent updates full-precision weights, but it does not update the quantized weights. To address this issue, we propose a novel method that enables direct updating of quantized weights with learnable quantization levels to minimize the cost function using gradient descent. Second, to obtain low bit-width activations, existing works consider all channels equally. However, the activation quantizers could be biased toward a few channels with high-variance. To address this issue, we propose a method to take into account the quantization errors of individual channels. With this approach, we can learn activation quantizers that minimize the quantization errors in the majority of channels. Experimental results demonstrate that our proposed method achieves state-of-the-art performance on the image classification task, using AlexNet, ResNet and MobileNetV2 architectures on CIFAR-100 and ImageNet datasets.

Sign in / Sign up

Export Citation Format

Share Document