A REGULARIZATION TERM TO AVOID THE SATURATION OF THE SIGMOIDS IN MULTILAYER NEURAL NETWORKS

1996 ◽  
Vol 07 (03) ◽  
pp. 257-262 ◽  
Author(s):  
LLUIS GARRIDO ◽  
SERGIO GÓMEZ ◽  
VICENS GAITÁN ◽  
MIQUEL SERRA-RICART
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Error Function ◽  
New Method ◽  
Quadratic Error ◽  
Regularization Term ◽  
Multilayer Neural Networks

In this paper we propose a new method to prevent the saturation of any set of hidden units of a multilayer neural network. This method is implemented by adding a regularization term to the standard quadratic error function, which is based on a repulsive action between pairs of patterns.

scholarly journals Feedforward Neural Networks with a Hidden Layer Regularization Method

Symmetry ◽  
2018 ◽  
Vol 10 (10) ◽  
pp. 525 ◽  
Author(s):  
Habtamu Alemu ◽  
Wei Wu ◽  
Junhong Zhao
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Structure ◽  
Regularization Method ◽  
Error Function ◽  
Feedforward Neural Networks ◽  
Training Process ◽  
Regularization Term ◽  
Neural Network Structure ◽  
Hidden Layer

In this paper, we propose a group Lasso regularization term as a hidden layer regularization method for feedforward neural networks. Adding a group Lasso regularization term into the standard error function as a hidden layer regularization term is a fruitful approach to eliminate the redundant or unnecessary hidden layer neurons from the feedforward neural network structure. As a comparison, a popular Lasso regularization method is introduced into standard error function of the network. Our novel hidden layer regularization method can force a group of outgoing weights to become smaller during the training process and can eventually be removed after the training process. This means it can simplify the neural network structure and it minimizes the computational cost. Numerical simulations are provided by using K-fold cross-validation method with K = 5 to avoid overtraining and to select the best learning parameters. The numerical results show that our proposed hidden layer regularization method prunes more redundant hidden layer neurons consistently for each benchmark dataset without loss of accuracy. In contrast, the existing Lasso regularization method prunes only the redundant weights of the network, but it cannot prune any redundant hidden layer neurons.

Effects of Nonlinear Synapses on the Performance of Multilayer Neural Networks

1996 ◽  
Vol 8 (5) ◽  
pp. 939-949 ◽  
Author(s):  
G. Dündar ◽  
F-C. Hsu ◽  
K. Rose
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Multilayer Neural Networks ◽  
Circuit Simulations ◽  
Multiplier Circuit

The problems arising from the use of nonlinear multipliers in multilayer neural network synapse structures are discussed. The errors arising from the neglect of nonlinearities are shown and the effect of training in eliminating these errors is discussed. A method for predicting the final errors resulting from nonlinearities is described. Our approximate results are compared with the results from circuit simulations of an actual multiplier circuit.

scholarly journals Increasing of Thermal Images Resolution Using Deep Learning Neural Networks

2021 ◽  
Vol 25 (3) ◽  
pp. 31-35
Author(s):  
Piotr Więcek ◽  
Dominik Sankowski
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Execution Time ◽  
High Accuracy ◽  
New Method ◽  
Residual Network ◽  
Thermal Images ◽  
The Neural Network

The article presents a new algorithm for increasing the resolution of thermal images. For this purpose, the residual network was integrated with the Kernel-Sharing Atrous Convolution (KSAC) image sub-sampling module. A significant reduction in the algorithm’s complexity and shortening the execution time while maintaining high accuracy were achieved. The neural network has been implemented in the PyTorch environment. The results of the proposed new method of increasing the resolution of thermal images with sizes 32 × 24, 160 × 120 and 640 × 480 for scales up to 6 are presented.

scholarly journals An Effective Method to Evaluate the Scientific Research Projects

2014 ◽  
Vol 39 (3) ◽  
pp. 175-188
Author(s):  
Xiaohui Hou ◽  
Lei Huang ◽  
Xuefei Li
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Linear Regression ◽  
Bp Neural Network ◽  
Error Function ◽  
Scientific Research ◽  
Research Projects ◽  
Neural Network Models ◽  
The Neural Networks ◽  
Linear Neural Network

Abstract The evaluation of the scientific research projects is an important procedure before the scientific research projects are approved. The BP neural network and linear neural network are adopted to evaluate the scientific research projects in this paper. The evaluation index system with 12 indexes is set up. The basic principle of the neural network is analyzed and then the BP neural network and linear neural network models are constructed and the output error function of the neural networks is introduced. The Matlab software is applied to set the parameters and calculate the neural networks. By computing a real-world example, the evaluation results of the scientific research projects are obtained and the results of the BP neural network, linear neural network and linear regression forecasting are compared. The analysis shows that the BP neural network has higher efficiency than the linear neural network and linear regression forecasting in the evaluation of the scientific research projects problem. The method proposed in this paper is an effective method to evaluate the scientific research projects.

Using Artificial Neural Networks for Fractal Interpolation Approach in 3D

2020 ◽  
Vol 12 (10) ◽  
pp. 1221-1225
Author(s):  
Rashad A. Al-Jawfi
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Theoretical Data ◽  
New Method ◽  
Fractal Interpolation ◽  
Great Precision ◽  
Interpolation Methods ◽  
Neural Network Algorithm ◽  
Iterated Functions Systems

In this paper, we introduce a new method for fractal interpolation, herein called Neural Network Algorithm (NNA), which is based on Iterated Functions Systems (IFS); proposed to self-affine signals interpolation with error of expected interpolation. Experiments on theoretical data show that the proposed interpolation schemes can obtain the expected point value and work with great precision in rebuilding the specified data profile, which leads to a significant advantage over other interpolation methods.

scholarly journals Modified Neural Network Algorithms for Predicting Trading Signals of Stock Market Indices

2009 ◽  
Vol 2009 ◽  
pp. 1-22 ◽  
Author(s):  
C. D. Tilakaratne ◽  
M. A. Mammadov ◽  
S. A. Morris
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Global Optimization ◽  
Stock Market ◽  
Historical Data ◽  
Error Function ◽  
Feedforward Neural Networks ◽  
Ordinary Least Squares ◽  
Global Optimization Algorithm ◽  
Network Algorithms

The aim of this paper is to present modified neural network algorithms to predict whether it is best to buy, hold, or sell shares (trading signals) of stock market indices. Most commonly used classification techniques are not successful in predicting trading signals when the distribution of the actual trading signals, among these three classes, is imbalanced. The modified network algorithms are based on the structure of feedforward neural networks and a modified Ordinary Least Squares (OLSs) error function. An adjustment relating to the contribution from the historical data used for training the networks and penalisation of incorrectly classified trading signals were accounted for, when modifying the OLS function. A global optimization algorithm was employed to train these networks. These algorithms were employed to predict the trading signals of the Australian All Ordinary Index. The algorithms with the modified error functions introduced by this study produced better predictions.

scholarly journals SOC and SOH Monitoring Algorithms for Lithium Batteries Using Multilayer Neural Networks

10.29007/m89x ◽  
2020 ◽  
Author(s):  
Jong Hyun Lee ◽  
Hyun Sil Kim ◽  
In Soo Lee
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Monitoring System ◽  
Lithium Batteries ◽  
Lithium Battery ◽  
Discharge Voltage ◽  
State Of Charge ◽  
State Of Health ◽  
Soc Estimation ◽  
Multilayer Neural Networks

This paper presents a battery monitoring system using a multilayer neural network (MNN) for state of charge (SOC) estimation and state of health (SOH) diagnosis. In this system, the MNN utilizes experimental discharge voltage data from lithium battery operation to estimate SOH and uses present and previous voltages for SOC estimation. From experimental results, we know that the proposed battery monitoring system performs SOC estimation and SOH diagnosis well.

scholarly journals A NEW HYBRID SYSTEM FOR RECOGNITION OF HANDWRITTEN-SCRIPT

2014 ◽  
pp. 50-57
Author(s):  
Khalid Saeed ◽  
Marek Tabedzki
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Object Recognition ◽  
Hybrid System ◽  
New Method ◽  
Recognition Process ◽  
Characteristic Points ◽  
Preparatory Stage ◽  
Work Method ◽  
Process Method

A new method for object recognition and classification is presented in this paper. It merges two well-known and tested methods: neural networks and method of minimal eigenvalues. Each of these methods answers for a different part of recognition process. Method of minimal eigenvalues makes preparatory stage of analysis – of coordinates of characteristic points we get the vector describing given image. Next, it is recognized and classified with neural network. Gathering of characteristic points we perform with our view-based algorithm, but other methods should also do. In this work, method was applied for words in Latin alphabet – handwritten and machine-printed. The obtained results are promising.

scholarly journals Support Spine Surgery by Information Technology

2020 ◽  
Vol 13 (3) ◽  
pp. 161-176
Author(s):  
Zoltan Tamas Kocsis
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Information Technology ◽  
Artificial Neural Networks ◽  
Spine Surgery ◽  
Surgical Procedure ◽  
New Method ◽  
The Neural Network ◽  
Artificial Neural

This paper presents a possible new method for supporting a specific spinal surgical procedure by artificial neural networks. The method should be based on the surgical demands and protocols used by surgeons in order to carry out successful operations. Considering these requirements, a plan for an algorithm that will be able to support surgeons in the preparation and the conduction of an operation is outlined. The aim is not to substitute the surgeon but to assist him. Furthermore, this paper demonstrates how the neural network to be designed can significantly reduce the possible surgical risks, thereby increasing surgery effectiveness.

Mechanical movement data acquisition method based on the multilayer neural networks and machine vision in a digital twin environment

Digital Twin ◽  
2021 ◽  
Vol 1 ◽  
pp. 6
Author(s):  
Hao Li ◽  
Gen Liu ◽  
Haoqi Wang ◽  
Xiaoyu Wen ◽  
Guizhong Xie ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Machine Vision ◽  
Mechanical Motion ◽  
Digital Twin ◽  
Movement Data ◽  
Multilayer Neural Networks ◽  
Visual Mark ◽  
Mechanical Movement ◽  
Acquisition Method

Background: Digital twin requires virtual reality mapping and optimization iteration between physical devices and virtual models. The mechanical movement data collection of physical equipment is essential for the implementation of accurate virtual and physical synchronization in a digital twin environment. However, the traditional approach relying on PLC (programmable logic control) fails to collect various mechanical motion state data. Additionally, few investigations have used machine visions for the virtual and physical synchronization of equipment. Thus, this paper presents a mechanical movement data acquisition method based on multilayer neural networks and machine vision. Methods: Firstly, various visual marks with different colors and shapes are designed for marking physical devices. Secondly, a recognition method based on the Hough transform and histogram feature is proposed to realize the recognition of shape and color features respectively. Then, the multilayer neural network model is introduced in the visual mark location. The neural network is trained by the dropout algorithm to realize the tracking and location of the visual mark. To test the proposed method, 1000 samples were selected. Results: The experiment results shows that when the size of the visual mark is larger than 6mm, the recognition success rate of the recognition algorithm can reach more than 95%. In the actual operation environment with multiple cameras, the identification points can be located more accurately. Moreover, the camera calibration process of binocular and multi-eye vision can be simplified by the multilayer neural networks. Conclusions: This study proposes an effective method in the collection of mechanical motion data of physical equipment in a digital twin environment. Further studies are needed to perceive posture and shape data of physical entities under the multi-camera redundant shooting.

Sign in / Sign up

Export Citation Format

Share Document