Neuro-Scientific Analysis of Weights in Neural Networks

Weight Distribution ◽

Network Architecture ◽

Network Training ◽

Quantitative Metrics ◽

Benchmark Datasets ◽

Weight Initialization ◽

Deep learning is a popular topic among machine learning researchers nowadays, with great strides being made in recent years to develop robust artificial neural networks for faster convergence to a reasonable accuracy. Network architecture and hyperparameters of the model are fundamental aspects of model convergence. One such important parameter is the initial values of weights, also known as weight initialization. In this paper, we perform two research tasks concerned with the weights of neural networks. First, we develop three novel weight initialization algorithms inspired by the neuroscientific construction of the mammalian brains and then test them on benchmark datasets against other algorithms to compare and assess their performance. We call these algorithms the lognormal weight initialization, modified lognormal weight initialization, and skewed weight initialization. We observe from our results that these initialization algorithms provide state-of-the-art results on all of the benchmark datasets. Second, we analyze the influence of training an artificial neural network on its weight distribution by measuring the correlation between the quantitative metrics of skewness and kurtosis against the model accuracy using linear regression for different weight initializations. Results indicate a positive correlation between network accuracy and skewness of the weight distribution but no affirmative relation between accuracy and kurtosis. This analysis provides further insight into understanding the inner mechanism of neural network training using the shape of weight distribution. Overall, the works in this paper are the first of their kind in incorporating neuroscientific knowledge into the domain of artificial neural network weights.

Analisis Jaringan Syaraf Tiruan untuk Memprediksi Jumlah Narapidana pada Lembaga Pemasyarakatan Simalungun dengan Metode Backpropagation

Prosiding Seminar Nasional Riset Information Science (SENARIS) ◽

10.30645/senaris.v1i0.82 ◽

2019 ◽

Vol 1 ◽

pp. 762

Author(s):

Vicky Adriani ◽

Irfan Sudahri Damanik ◽

Jaya Tata Hardinata

Keyword(s):

Neural Network ◽

Neural Networks ◽

Human Brain ◽

Network Architecture ◽

Accuracy Rate ◽

Backpropagation Algorithm ◽

Artificial Neural ◽

Detention Facilities

The author has conducted research at the Simalungun District Prosecutor's Office and found the problem of prison rooms that did not match the number of prisoners which caused a lack of security and a lack of detention facilities and risked inmates to flee. Artificial Neural Network which is one of the artificial representations of the human brain that always tries to simulate the learning process of the human brain. The application uses the Backpropagation algorithm where the data entered is the number of prisoners. Then Artificial Neural Networks are formed by determining the number of units per layer. Once formed, training is carried out from the data that has been grouped. Experiments are carried out with a network architecture consisting of input units, hidden units, and output units. Testing using Matlab software. For now, the number of prisoners continues to increase. Predictions with the best accuracy use the 12-3-1 architecture with an accuracy rate of 75% and the lowest level of accuracy using 12-4-1 architecture with an accuracy rate of 25%.

Genetic Neural Networks: an artificial neural network architecture for capturing gene expression relationships

Bioinformatics ◽

10.1093/bioinformatics/bty945 ◽

2018 ◽

Vol 35 (13) ◽

pp. 2226-2234 ◽

Cited By ~ 5

Author(s):

Ameen Eetemadi ◽

Ilias Tagkopoulos

Keyword(s):

Neural Network ◽

Gene Expression ◽

Neural Networks ◽

Network Architecture ◽

Gene Networks ◽

Supplementary Information ◽

Genome Wide ◽

Transcriptomics Data ◽

Abstract Motivation Gene expression prediction is one of the grand challenges in computational biology. The availability of transcriptomics data combined with recent advances in artificial neural networks provide an unprecedented opportunity to create predictive models of gene expression with far reaching applications. Results We present the Genetic Neural Network (GNN), an artificial neural network for predicting genome-wide gene expression given gene knockouts and master regulator perturbations. In its core, the GNN maps existing gene regulatory information in its architecture and it uses cell nodes that have been specifically designed to capture the dependencies and non-linear dynamics that exist in gene networks. These two key features make the GNN architecture capable to capture complex relationships without the need of large training datasets. As a result, GNNs were 40% more accurate on average than competing architectures (MLP, RNN, BiRNN) when compared on hundreds of curated and inferred transcription modules. Our results argue that GNNs can become the architecture of choice when building predictors of gene expression from exponentially growing corpus of genome-wide transcriptomics data. Availability and implementation https://github.com/IBPA/GNN Supplementary information Supplementary data are available at Bioinformatics online.

Use of Binary Sigmoid Function And Linear Identity In Artificial Neural Networks For Forecasting Population Density

IJISTECH (International Journal Of Information System & Technology) ◽

10.30645/ijistech.v1i1.6 ◽

2017 ◽

Vol 1 (1) ◽

pp. 43 ◽

Cited By ~ 4

Author(s):

Anjar Wanto ◽

Agus Perdana Windarto ◽

Dedy Hartama ◽

Iin Parlina

Keyword(s):

Neural Network ◽

Neural Networks ◽

Population Density ◽

Network Architecture ◽

Activation Function ◽

Sigmoid Function ◽

Backpropagation Algorithm ◽

Architecture Model ◽

Artificial Neural Network (ANN) is often used to solve forecasting cases. As in this study. The artificial neural network used is with backpropagation algorithm. The study focused on cases concerning overcrowding forecasting based District in Simalungun in Indonesia in 2010-2015. The data source comes from the Central Bureau of Statistics of Simalungun Regency. The population density forecasting its future will be processed using backpropagation algorithm focused on binary sigmoid function (logsig) and a linear function of identity (purelin) with 5 network architecture model used the 3-5-1, 3-10-1, 3-5 -10-1, 3-5-15-1 and 3-10-15-1. Results from 5 to architectural models using Neural Networks Backpropagation with binary sigmoid function and identity functions vary greatly, but the best is 3-5-1 models with an accuracy of 94%, MSE, and the epoch 0.0025448 6843 iterations. Thus, the use of binary sigmoid activation function (logsig) and the identity function (purelin) on Backpropagation Neural Networks for forecasting the population density is very good, as evidenced by the high accuracy results achieved.

Implementasi Metode Backpropogation dengan Inisialisasi Bobot Nguyen Widrow untuk Peramalan Harga Saham

Jurnal Teknologi Informasi dan Ilmu Komputer ◽

10.25126/jtiik.201961904 ◽

2019 ◽

Vol 6 (1) ◽

pp. 49

Author(s):

Eliv Kurniawan ◽

Hari Wibawanto ◽

Djoko Adi Widodo

Keyword(s):

Neural Network ◽

Neural Networks ◽

Stock Prices ◽

Stock Price ◽

Daily Data ◽

Backpropagation Method ◽

Weight Initialization ◽

Jaringan saraf tiruan merupakan suatu ilmu yang terus berkembang pesat hingga saat ini. Jaringan saraf tiruan merupakan suatu ilmu komputasi yang didasarkan dan terinspirasi dari cara kerja sistem saraf manusia. Sama halnya dengan sistem saraf manusia, jaringan saraf tiruan bekerja melalui proses pembelajaran terhadap data-data yang sudah ada untuk memformulakan keluaran dari data-data baru. Jaringan saraf tiruan dengan metode backpropagation mampu melakukan peramalan untuk data nonlinear seperti bentuk data harian harga saham. Salah satu algoritma inisialisasi bobot yang dapat meningkatkan waktu eksekusi adalah nguyen-widrow. Pada penelitian ini akan dilakukan implementasi metode backpropagation dengan inisialisasi bobot nguyen widrow untuk meramalkan harga saham. Proses implementasi melalui 3 tahapan, yaitu preprosesing data, pelatihan jaringan, dan pengujian jaringan. Hasil dari penelitian ini menunjukkan bahwa pelatihan jaringan saraf tiruan dengan jumlah dataset yang banyak membutuhkan perhitungan yang kompleks, sehingga jaringan saraf tiruan dengan arsitektur jaringan yang sederhana kurang efektif dan dapat terjebak pada titik lokal minimum. Hasil peramalan untuk harga close saham BBCA.JK memiliki nilai MAPE 0,85% dan untuk harga close saham AALI.JK memiliki nilai MAPE sebesar 1,84%.AbstractArtificial neural network is a hot topic and invite a lot of admiration in the last decade. Artificial Neural Network is one of the artificial representations of the humans brain who always try to simulate the learning process of the humans brain. Artificial neural network with backpropagation method is able to forecast nonlinear data such as daily data form stock price. One of the weight initialization algorithms that can be increase the execution time is nguyen-widrow. In this research will be implemented backpropagation method with nguyen widrow weight initialization to forecast stock prices. The process of implementation through 3 stages, that is preprosesing data, training, and testing or simulate. The results of this research indicate that the training of artificial neural networks with many datasets required a complex calculations, so the artificial neural network with simple architectures is less effective and can get stuck at minimum local points. The results forecasting for the close price of BBCA.JK have a MAPE value 0.85% and for the close price of AALI.JK have 1.84% of MAPE value

Optimizing Artificial Neural Networks For The Evaluation Of Asphalt Pavement Structural Performance

The Baltic Journal of Road and Bridge Engineering ◽

10.7250/bjrbe.2019-14.433 ◽

2019 ◽

Vol 14 (1) ◽

pp. 58-79 ◽

Cited By ~ 1

Author(s):

Gaetano Bosurgi ◽

Orazio Pellegrino ◽

Giuseppe Sollazzo

Keyword(s):

Neural Network ◽

Neural Networks ◽

Performance Evaluation ◽

Network Architecture ◽

Training Data ◽

Structural Performance ◽

Reliable Technique ◽

Artificial Neural Networks represent useful tools for several engineering issues. Although they were adopted in several pavement-engineering problems for performance evaluation, their application on pavement structural performance evaluation appears to be remarkable. It is conceivable that defining a proper Artificial Neural Network for estimating structural performance in asphalt pavements from measurements performed through quick and economic surveys produces significant savings for road agencies and improves maintenance planning. However, the architecture of such an Artificial Neural Network must be optimised, to improve the final accuracy and provide a reliable technique for enriching decision-making tools. In this paper, the influence on the final quality of different features conditioning the network architecture has been examined, for maximising the resulting quality and, consequently, the final benefits of the methodology. In particular, input factor quality (structural, traffic, climatic), “homogeneity” of training data records and the actual net topology have been investigated. Finally, these results further prove the approach efficiency, for improving Pavement Management Systems and reducing deflection survey frequency, with remarkable savings for road agencies.

Prediction of Motor Failure Time Using An Artificial Neural Network

Sensors ◽

10.3390/s19194342 ◽

2019 ◽

Vol 19 (19) ◽

pp. 4342 ◽

Cited By ~ 6

Author(s):

Gustavo Scalabrini Sampaio ◽

Arnaldo Rabello de Aguiar Vallim Filho ◽

Leilton Santos da Silva ◽

Leandro Augusto da Silva

Keyword(s):

Neural Network ◽

Neural Networks ◽

Failure Time ◽

Machine Learning Techniques ◽

Network Training ◽

Model Generalization ◽

The Neural Network ◽

Learning Techniques ◽

Industry is constantly seeking ways to avoid corrective maintenance so as to reduce costs. Performing regular scheduled maintenance can help to mitigate this problem, but not necessarily in the most efficient way. In the context of condition-based maintenance, the main contributions of this work were to propose a methodology to treat and transform the collected data from a vibration system that simulated a motor and to build a dataset to train and test an Artificial Neural Network capable of predicting the future condition of the equipment, pointing out when a failure can happen. To achieve this goal, a device model was built to simulate typical motor vibrations, consisting of a computer cooler fan and several magnets. Measurements were made using an accelerometer, and the data were collected and processed to produce a structured dataset. The neural network training with this dataset converged quickly and stably, while the tests performed, k-fold cross-validation and model generalization, presented excellent performance. The same tests were performed with other machine learning techniques, to demonstrate the effectiveness of neural networks mainly in their generalizability. The results of the work confirm that it is possible to use neural networks to perform predictive tasks in relation to the conditions of industrial equipment. This is an important area of study that helps to support the growth of smart industries.

A Study on the Image Classification Techniques Based on Wavelet Artificial Neural Network Algorithm

Applied Mechanics and Materials ◽

10.4028/www.scientific.net/amm.602-605.3512 ◽

2014 ◽

Vol 602-605 ◽

pp. 3512-3514

Author(s):

Xue Ding ◽

Hong Hong Yang

Keyword(s):

Neural Network ◽

Neural Networks ◽

Artificial Neural Network Training

Training Methods ◽

Neural Network Training ◽

Target Vector ◽

Network Training ◽

Incremental Training ◽

Artificial Neural ◽

With the ever-changing education information technology, it is a big problem for the universities and college that how to classify the thousands of copies of the image during the art examination marking process. This paper is to explore the application of artificial intelligence techniques, and to do accurate classification of a large number of images within a limited time and under the help of computer. It is can be seen that the proposed method is feasible through the application of the results of the actual work. Artificial neural network training Artificial neural network training methods have two mainly style, which are Incremental Training and Batch Training, and take the amount of different network training mission as the distinction standard. First, to introduce the Incremental Training [1], that means whenever the network receives the input vector and target vector, it have to adjust once the connection weights and thresholds. It is an online learning method. The other one is Batch Training [2], that means no longer adjust the connection and immediately, but perform bulk adjustment, and after a given volume of the input vector and target vector. Both training methods can be applied, whether it is static or dynamic neural network. Different results will be obtained by artificial neural network for the use of different training methods. When using artificial neural networks to solve specific problems, learning methods, training methods and artificial neural network function should be selected according to the expected results of question type and its specific requirements [3-4]. The selection of parameters of wavelet neural networks and adaptive learning

Memprediksi Jumlah Siswa Baru Menggunakan Metode Backpropagation (STUDI KASUS: SMK HARAPAN BANGSA KUALA)

JUKI : Jurnal Komputer dan Informatika ◽

10.53842/juki.v3i1.40 ◽

2021 ◽

Vol 3 (1) ◽

pp. 10-16

Author(s):

Kris Jayanti ◽

Katen Lumbanbatu ◽

Suci Ramadani

Keyword(s):

Neural Network ◽

Neural Networks ◽

Network Architecture ◽

Time Series Data ◽

Real Data ◽

Activation Function ◽

Series Data ◽

Base Function ◽

Artificial Neural Network (ANN) and time series data can be used for forecasting methods well. Artificial Neural Network is a method whose working principle is adapted from a mathematical model in humans or biological nerves. Neural networks are characterized by; (1) the pattern of connections between neurons (called architecture), (2) determining the weight of the connection (called training or learning), and (3) the activation function. The research objective was to obtain the best artificial neural network architecture, comparing the two methods of Backpropogation Neural Networks with the Radial Base Function Artificial Neural Network (RBF) method. This research is a research using real data (true experimental). This research was conducted at SMK Harapan Bangsa Kuala, which was obtained from 2015 to 2019. The results showed that for one iteration using the backpropagation method the result was 0,378197657 with a squared error 0.143033468, then the results achieved were not in accordance with the target.

Jaringan Syaraf Tiruan Backpropogation dalam Memprediksi Jumlah Tilang di Kejaksaan Negeri Simalungun

Prosiding Seminar Nasional Riset Information Science (SENARIS) ◽

10.30645/senaris.v1i0.76 ◽

2019 ◽

Vol 1 ◽

pp. 697

Author(s):

Sony Irwanda ◽

Jaya Tata Hardinata ◽

Irfan Sudahri Damanik

Keyword(s):

Neural Network ◽

Neural Networks ◽

Network Architecture ◽

Matlab Software ◽

This study predicts the number of ticketing by applying Artificial Neural Networks. The application uses the Backpropogation algorithm where the data entered is the number of tickets. Then an Artificial Neural Network is formed by determining the number of units per layer. After the network is formed training is carried out from the data that has been grouped. Experiments are carried out with a network architecture consisting of input units, hidden units, output units and network architecture. Testing is done with Matlab software.

Classification of asteroid families with artificial neural networks

Serbian Astronomical Journal ◽

10.2298/saj2001039v ◽

2020 ◽

pp. 39-48

Author(s):

D. Vujicic ◽

R. Pavlovic ◽

D. Milosevic ◽

B. Djordjevic ◽

S. Randjic ◽

...

Keyword(s):

Neural Network ◽

Neural Networks ◽

Neural Network Training ◽

Network Training ◽

Artificial Neural ◽

Artificial Neural Network Training ◽

Asteroid Families

This paper describes an artificial neural network for classification of asteroids into families. The data used for artificial neural network training and testing were obtained by the Hierarchical Clustering Method (HCM). We have shown that an artificial neural networks can be used as a validation method for the HCM on families with a large number of members.