scholarly journals IMPROVED BACKPROPAGATION LEARNING IN NEURAL NETWORKS WITH WINDOWED MOMENTUM

2002 ◽  
Vol 12 (03n04) ◽  
pp. 303-318 ◽  
Author(s):  
ERNEST ISTOOK ◽  
TONY MARTINEZ
Keyword(s):  
Neural Network ◽  
Standard Technique ◽  
Presentation Order ◽  
Convergence Time ◽  
Data Sets ◽  
Data Set ◽  
Acceptable Solution ◽  
Additional Information ◽  
Network Training ◽  
Improved Accuracy

Backpropagation, which is frequently used in Neural Network training, often takes a great deal of time to converge on an acceptable solution. Momentum is a standard technique that is used to speed up convergence and maintain generalization performance. In this paper we present the Windowed Momentum algorithm, which increases speedup over Standard Momentum. Windowed Momentum is designed to use a fixed width history of recent weight updates for each connection in a neural network. By using this additional information, Windowed Momentum gives significant speedup over a set of applications with same or improved accuracy. Windowed Momentum achieved an average speedup of 32% in convergence time on 15 data sets, including a large OCR data set with over 500,000 samples. In addition to this speedup, we present the consequences of sample presentation order. We show that Windowed Momentum is able to overcome these effects that can occur with poor presentation order and still maintain its speedup advantages.

scholarly journals Prediction of yarn unevenness based on BMNN

2021 ◽  
Vol 16 ◽  
pp. 155892502110379
Author(s):  
Hao Jiang ◽  
Jiuxiang Song ◽  
Baowei Zhang ◽  
Suna Zhao ◽  
Yonghua Wang
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Learning System ◽  
Training Data ◽  
Data Sets ◽  
Data Set ◽  
Training Time ◽  
Network Training ◽  
Yarn Quality ◽  
Neural Network Structure

With the continuous development of deep learning, due to the complexity of the deep neural network structure and the limitation of training time, some scholars have proposed broad learning, the Broad Learning System (BLS). However, BLS currently only verifies that it has excellent effects on some of the network training data sets, and it does not necessarily have excellent effects on some actual data sets. In response to this, this paper uses the effect of BLS in predicting the unevenness of yarn quality in the yarn data set, and proposes a BLS-based multi-layer neural network (MNN) for the problems, which is called Broad Multilayer Neural Network (BMNN).

A Comparison Study of Mahalanobis-Taguchi System and Neural Network for Multivariate Pattern Recognition

2005 ◽  
Author(s):  
Jungeui Hong ◽  
Elizabeth A. Cudney ◽  
Genichi Taguchi ◽  
Rajesh Jugulum ◽  
Kioumars Paryani ◽  
...  
Keyword(s):  
Neural Network ◽  
Small Data ◽  
Data Sets ◽  
Comparison Study ◽  
Data Set ◽  
Set Size ◽  
Breast Cancer Study ◽  
Discriminant Ability ◽  
Small Data Sets ◽  
Multivariate Pattern

The Mahalanobis-Taguchi System is a diagnosis and predictive method for analyzing patterns in multivariate cases. The goal of this study is to compare the ability of the Mahalanobis-Taguchi System and a neural network to discriminate using small data sets. We examine the discriminant ability as a function of data set size using an application area where reliable data is publicly available. The study uses the Wisconsin Breast Cancer study with nine attributes and one class.

scholarly journals Comparison between XBT data and TOPEX/Poseidon satellite altimetry in the Ligurian-Tyrrhenian area

2003 ◽  
Vol 21 (1) ◽  
pp. 123-135 ◽  
Author(s):  
S. Vignudelli ◽  
P. Cipollini ◽  
F. Reseghetti ◽  
G. Fusco ◽  
G. P. Gasparini ◽  
...  
Keyword(s):  
Pilot Project ◽  
Atlantic Water ◽  
Data Sets ◽  
Tyrrhenian Sea ◽  
Satellite Altimeter ◽  
Ligurian Sea ◽  
Data Set ◽  
Additional Information ◽  
Forecasting System ◽  
Computational Errors

Abstract. From September 1999 to December 2000, eXpendable Bathy-Thermograph (XBT) profiles were collected along the Genova-Palermo shipping route in the framework of the Mediterranean Forecasting System Pilot Project (MFSPP). The route is virtually coincident with track 0044 of the TOPEX/Poseidon satellite altimeter, crossing the Ligurian and Tyrrhenian basins in an approximate N–S direction. This allows a direct comparison between XBT and altimetry, whose findings are presented in this paper. XBT sections reveal the presence of the major features of the regional circulation, namely the eastern boundary of the Ligurian gyre, the Bonifacio gyre and the Modified Atlantic Water inflow along the Sicily coast. Twenty-two comparisons of steric heights derived from the XBT data set with concurrent realizations of single-pass altimetric heights are made. The overall correlation is around 0.55 with an RMS difference of less than 3 cm. In the Tyrrhenian Sea the spectra are remarkably similar in shape, but in general the altimetric heights contain more energy. This difference is explained in terms of oceanographic signals, which are captured with a different intensity by the satellite altimeter and XBTs, as well as computational errors. On scales larger than 100 km, the data sets are also significantly coherent, with increasing coherence values at longer wavelengths. The XBTs were dropped every 18–20 km along the track: as a consequence, the spacing scale was unable to resolve adequately the internal radius of deformation (< 20 km). Furthermore, few XBT drops were carried out in the Ligurian Sea, due to the limited north-south extent of this basin, so the comparison is problematic there. On the contrary, the major features observed in the XBT data in the Tyrrhenian Sea are also detected by TOPEX/Poseidon. The manuscript is completed by a discussion on how to integrate the two data sets, in order to extract additional information. In particular, the results emphasize their complementariety in providing a dynamically complete description of the observed structures. Key words. Oceanography: general (descriptive and regional oceanography) Oceanography: physical (sea level variations; instruments and techniques)

The Application of Probabilistic Neural Network in Speech Recognition Based on Partition Clustering

2012 ◽  
Vol 263-266 ◽  
pp. 2173-2178
Author(s):  
Xin Guang Li ◽  
Min Feng Yao ◽  
Li Rui Jian ◽  
Zhen Jiang Li
Keyword(s):  
Neural Network ◽  
Speech Recognition ◽  
Clustering Algorithm ◽  
Probabilistic Neural Network ◽  
Back Propagation ◽  
Back Propagation Neural Network ◽  
Data Sets ◽  
Data Set ◽  
Proposed Model ◽  
Partition Clustering

A probabilistic neural network (PNN) speech recognition model based on the partition clustering algorithm is proposed in this paper. The most important advantage of PNN is that training is easy and instantaneous. Therefore, PNN is capable of dealing with real time speech recognition. Besides, in order to increase the performance of PNN, the selection of data set is one of the most important issues. In this paper, using the partition clustering algorithm to select data is proposed. The proposed model is tested on two data sets from the field of spoken Arabic numbers, with promising results. The performance of the proposed model is compared to single back propagation neural network and integrated back propagation neural network. The final comparison result shows that the proposed model performs better than the other two neural networks, and has an accuracy rate of 92.41%.

scholarly journals Ambulatory and Laboratory Stress Detection Based on Raw Electrocardiogram Signals Using a Convolutional Neural Network

Sensors ◽  
2019 ◽  
Vol 19 (20) ◽  
pp. 4408 ◽  
Author(s):  
Hyun-Myung Cho ◽  
Heesu Park ◽  
Suh-Yeon Dong ◽  
Inchan Youn
Keyword(s):  
Neural Network ◽  
Transfer Learning ◽  
Mental Arithmetic ◽  
Small Data ◽  
Data Sets ◽  
Learning Method ◽  
Data Set ◽  
Electrocardiogram Signals ◽  
Proposed Model ◽  
Small Data Set

The goals of this study are the suggestion of a better classification method for detecting stressed states based on raw electrocardiogram (ECG) data and a method for training a deep neural network (DNN) with a smaller data set. We suggest an end-to-end architecture to detect stress using raw ECGs. The architecture consists of successive stages that contain convolutional layers. In this study, two kinds of data sets are used to train and validate the model: A driving data set and a mental arithmetic data set, which smaller than the driving data set. We apply a transfer learning method to train a model with a small data set. The proposed model shows better performance, based on receiver operating curves, than conventional methods. Compared with other DNN methods using raw ECGs, the proposed model improves the accuracy from 87.39% to 90.19%. The transfer learning method improves accuracy by 12.01% and 10.06% when 10 s and 60 s of ECG signals, respectively, are used in the model. In conclusion, our model outperforms previous models using raw ECGs from a small data set and, so, we believe that our model can significantly contribute to mobile healthcare for stress management in daily life.

EFFECTS OF NONSINGULAR PREPROCESSING ON FEEDFORWARD NETWORK TRAINING

2005 ◽  
Vol 19 (02) ◽  
pp. 217-247 ◽  
Author(s):  
CHANGHUA YU ◽  
MICHAEL T. MANRY ◽  
JIANG LI
Keyword(s):  
Back Propagation ◽  
Original Data ◽  
Data Sets ◽  
Training Algorithm ◽  
Feedforward Network ◽  
Data Set ◽  
Network Training ◽  
The Neural Network ◽  
Hidden Layer ◽  
Theoretical Analyses

In the neural network literature, many preprocessing techniques, such as feature de-correlation, input unbiasing and normalization, are suggested to accelerate multilayer perceptron training. In this paper, we show that a network trained with an original data set and one trained with a linear transformation of the original data will go through the same training dynamics, as long as they start from equivalent states. Thus preprocessing techniques may not be helpful and are merely equivalent to using a different weight set to initialize the network. Theoretical analyses of such preprocessing approaches are given for conjugate gradient, back propagation and the Newton method. In addition, an efficient Newton-like training algorithm is proposed for hidden layer training. Experiments on various data sets confirm the theoretical analyses and verify the improvement of the new algorithm.

Neuronet-Based Approach for Assessing Liquefaction Potential of Soils

1998 ◽  
Vol 1633 (1) ◽  
pp. 3-8 ◽  
Author(s):  
Hossam Eldin Ali ◽  
Yacoub M. Najjar
Keyword(s):  
Neural Network ◽  
Numerical Approach ◽  
Soil Liquefaction ◽  
Data Sets ◽  
Ann Model ◽  
Neural Network Approach ◽  
Liquefaction Potential ◽  
Network Training ◽  
Artificial Neural Network Ann ◽  
Hidden Layer

A backpropagation artificial neural network (ANN) algorithm with one hidden layer was used as a new numerical approach to characterize the soil liquefaction potential. For this purpose, 61 field data sets representing various earthquake sites from around the world were used. To develop the most accurate prediction model for liquefaction potential, alternating combinations of input parameters were used during the training and testing phases of the developed network. The accuracy of the designed network was validated against an additional 44 records not used previously in either the network training or testing stages. The prediction accuracy of the neural network approach–based model is compared with predictions obtained by using fuzzy logic and statistically based approaches. Overall, the ANN model outperformed all other investigated approaches.

scholarly journals BraggNet: integrating Bragg peaks using neural networks

2019 ◽  
Vol 52 (4) ◽  
pp. 854-863 ◽  
Author(s):  
Brendan Sullivan ◽  
Rick Archibald ◽  
Jahaun Azadmanesh ◽  
Venu Gopal Vandavasi ◽  
Patricia S. Langan ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Negative Control ◽  
Protein Crystal ◽  
Data Sets ◽  
Data Set ◽  
X Ray ◽  
X Ray Crystallography ◽  
Integrated Intensities ◽  
Neutron Crystallography

Neutron crystallography offers enormous potential to complement structures from X-ray crystallography by clarifying the positions of low-Z elements, namely hydrogen. Macromolecular neutron crystallography, however, remains limited, in part owing to the challenge of integrating peak shapes from pulsed-source experiments. To advance existing software, this article demonstrates the use of machine learning to refine peak locations, predict peak shapes and yield more accurate integrated intensities when applied to whole data sets from a protein crystal. The artificial neural network, based on the U-Net architecture commonly used for image segmentation, is trained using about 100 000 simulated training peaks derived from strong peaks. After 100 training epochs (a round of training over the whole data set broken into smaller batches), training converges and achieves a Dice coefficient of around 65%, in contrast to just 15% for negative control data sets. Integrating whole peak sets using the neural network yields improved intensity statistics compared with other integration methods, including k-nearest neighbours. These results demonstrate, for the first time, that neural networks can learn peak shapes and be used to integrate Bragg peaks. It is expected that integration using neural networks can be further developed to increase the quality of neutron, electron and X-ray crystallography data.

scholarly journals SYNTHESIS OF SELF-ORGANIZING MAP AND FEEDFORWARD NEURAL NETWORK FOR BETTER FORECASTING

2014 ◽  
pp. 68-75
Author(s):  
Oles Hodych ◽  
Yuriy Shcherbyna ◽  
Michael Zylan
Keyword(s):  
Neural Network ◽  
Feedforward Neural Network ◽  
Share Price ◽  
Data Sets ◽  
Self Organizing Map ◽  
Data Set ◽  
Price Fluctuation ◽  
Price Data ◽  
Comparison Results ◽  
Self Organizing

In this article the authors propose an approach to forecasting the direction of the share price fluctuation, which is based on utilization of the Feedforward Neural Network in conjunction with Self-Organizing Map. It is proposed to use the Self-Organizing Map for filtration of the share price data set, whereas the Feedforward Neural Network is used to forecast the direction of the share price fluctuation based on the filtered data set. The comparison results are presented for filtered and non-filtered share price data sets.

scholarly journals Core looseness fault identification model based on Mel spectrogram-CNN

2021 ◽  
Vol 2137 (1) ◽  
pp. 012060
Author(s):  
Ping He ◽  
Yong Li ◽  
Shoulong Chen ◽  
Hoghua Xu ◽  
Lei Zhu ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Convolution Neural Network ◽  
Data Sets ◽  
Data Set ◽  
Recognition Model ◽  
Model Based ◽  
Fault Recognition ◽  
Transformer Core ◽  
Identification Model

Abstract In order to realize transformer voiceprint recognition, a transformer voiceprint recognition model based on Mel spectrum convolution neural network is proposed. Firstly, the transformer core looseness fault is simulated by setting different preloads, and the sound signals under different preloads are collected; Secondly, the sound signal is converted into a spectrogram that can be trained by convolutional neural network, and then the dimension is reduced by Mel filter bank to draw Mel spectrogram, which can generate spectrogram data sets under different preloads in batch; Finally, the data set is introduced into convolutional neural network for training, and the transformer voiceprint fault recognition model is obtained. The results show that the training accuracy of the proposed Mel spectrum convolution neural network transformer identification model is 99.91%, which can well identify the core loosening faults.

Sign in / Sign up

Export Citation Format

Share Document