A Modification to the Nguyen–Widrow Weight Initialization Method

2019 ◽  
pp. 141-153 ◽  
Author(s):  
Apeksha Mittal ◽  
Amit Prakash Singh ◽  
Pravin Chandra
Keyword(s):  
Weight Initialization

scholarly journals Deep ConvNet: Non-Random Weight Initialization for Repeatable Determinism, Examined with FSGM

Sensors ◽  
2021 ◽  
Vol 21 (14) ◽  
pp. 4772
Author(s):  
Richard N. M. Rudd-Orthner ◽  
Lyudmila Mihaylova
Keyword(s):  
Cross Validation ◽  
Color Image ◽  
Validation Dataset ◽  
Number Sequence ◽  
Random Weight ◽  
Fast Gradient ◽  
Weight Initialization ◽  
Fitting In ◽  
Sequence Substitution ◽  
Sign Method

A repeatable and deterministic non-random weight initialization method in convolutional layers of neural networks examined with the Fast Gradient Sign Method (FSGM). Using the FSGM approach as a technique to measure the initialization effect with controlled distortions in transferred learning, varying the dataset numerical similarity. The focus is on convolutional layers with induced earlier learning through the use of striped forms for image classification. Which provided a higher performing accuracy in the first epoch, with improvements of between 3–5% in a well known benchmark model, and also ~10% in a color image dataset (MTARSI2), using a dissimilar model architecture. The proposed method is robust to limit optimization approaches like Glorot/Xavier and He initialization. Arguably the approach is within a new category of weight initialization methods, as a number sequence substitution of random numbers, without a tether to the dataset. When examined under the FGSM approach with transferred learning, the proposed method when used with higher distortions (numerically dissimilar datasets), is less compromised against the original cross-validation dataset, at ~31% accuracy instead of ~9%. This is an indication of higher retention of the original fitting in transferred learning.

An Experimental Study of Weight Initialization and Lamarckian Inheritance on Neuroevolution

2021 ◽  
pp. 584-600
Author(s):  
Zimeng Lyu ◽  
AbdElRahman ElSaid ◽  
Joshua Karns ◽  
Mohamed Mkaouer ◽  
Travis Desell
Keyword(s):  
Experimental Study ◽  
Lamarckian Inheritance ◽  
Weight Initialization

Weight initialization techniques

1998 ◽  
pp. 87-121 ◽  
Author(s):  
Mikko Lehtokangas ◽  
Jukka Saarinen ◽  
Petri Salmela ◽  
Kimmo Kaski
Keyword(s):  
Weight Initialization
Sign in / Sign up

Export Citation Format

Share Document