The Impact of Learning Rate Decay and Periodical Learning Rate Restart on Artificial Neural Network

2021 ◽  
Author(s):  
Yimin Ding
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Learning Rate ◽  
Artificial Neural ◽  
The Impact ◽  
Rate Decay

Analyzing the Impact of Learning Rate in development of the Prediction Model using Artificial Neural Network

2021 ◽  
Vol 23 (07) ◽  
pp. 121-135
Author(s):  
Anil Kumar Bisht ◽  
◽  
Ravendra Singh ◽  
Rakesh Bhutiani ◽  
Ashutosh Bhatt ◽  
...  
Keyword(s):  
Neural Network ◽  
Water Quality ◽  
Artificial Neural Network ◽  
Mean Squared Error ◽  
Learning Rate ◽  
Ganges River ◽  
Squared Error ◽  
Artificial Neural ◽  
The Ganges ◽  
The Impact

Water Quality (WQ) modeling and forecasting are very challenging for water management bodies due to the complex and nonlinear relationship between the parameters responsible for determining water quality. The main focus of this paper is the water quality prediction of the Ganges River by analyzing the impact of one of the critical configuration parameters of a neural network known as the learning rate. The proposed prediction model based on an artificial neural network (ANN) consists of different sets of experiments performed by comparing twelve different training functions against the variation in learning rates. A total of 360 experiments have been conducted on the dataset collected over the period 2001 to 2015 with five stations along the Ganges River in the state of Uttarakhand, India. All experiments have been conducted in MATLAB software. The ANN-based program is written in Matlab’s NN-Toolbox. As input parameters, we have used temperature, pH, dissolved oxygen (DO), biochemical oxygen demand (BOD), and total coliform. The water quality standard set by the Central Pollution Control Board of India has been used. The performance of the developed model has been calculated based on Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Percentage Error (MAPE). Trail training function-based artificial neural network models indicate higher predictive accuracy when compared to other models developed using the remaining eleven training functions when the learning rate is set to 0.04. In conclusion, ANN has the ability to efficiently predict the water quality of rivers and the learning rate has a greater impact on the development of such predictive models. So, it is required to be tuned very carefully.

scholarly journals Artificial neural network, predictor variables and sensitivity threshold for DNA methylation-based age prediction using blood samples

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Zhonghui Thong ◽  
Jolena Ying Ying Tan ◽  
Eileen Shuzhen Loo ◽  
Yu Wei Phua ◽  
Xavier Liang Shun Chan ◽  
...  
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Regression Model ◽  
Sensitivity Threshold ◽  
Ann Model ◽  
Blood Samples ◽  
Ann Models ◽  
Artificial Neural ◽  
The Impact ◽  
Age Prediction

AbstractRegression models are often used to predict age of an individual based on methylation patterns. Artificial neural network (ANN) however was recently shown to be more accurate for age prediction. Additionally, the impact of ethnicity and sex on our previous regression model have not been studied. Furthermore, there is currently no age prediction study investigating the lower limit of input DNA at the bisulfite treatment stage prior to pyrosequencing. Herein, we evaluated both regression and ANN models, and the impact of ethnicity and sex on age prediction for 333 local blood samples using three loci on the pyrosequencing platform. Subsequently, we trained a one locus-based ANN model to reduce the amount of DNA used. We demonstrated that the ANN model has a higher accuracy of age prediction than the regression model. Additionally, we showed that ethnicity did not affect age prediction among local Chinese, Malays and Indians. Although the predicted age of males were marginally overestimated, sex did not impact the accuracy of age prediction. Lastly, we present a one locus, dual CpG model using 25 ng of input DNA that is sufficient for forensic age prediction. In conclusion, the two ANN models validated would be useful for age prediction to provide forensic intelligence leads.

Prognosis of water levels in a moor groundwater system influenced by hydrology and water extraction using an artificial neural network 

2021 ◽  
Author(s):  
Sascha Flaig ◽  
Timothy Praditia ◽  
Alexander Kissinger ◽  
Ulrich Lang ◽  
Sergey Oladyshkin ◽  
...  
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Mean Squared Error ◽  
Meteorological Data ◽  
Water Levels ◽  
Typical Structure ◽  
Water Abstraction ◽  
Physical Knowledge ◽  
Artificial Neural ◽  
The Impact

<p>In order to prevent possible negative impacts of water abstraction in an ecologically sensitive moor south of Munich (Germany), a “predictive control” scheme is in place. We design an artificial neural network (ANN) to provide predictions of moor water levels and to separate hydrological from anthropogenic effects. As the moor is a dynamic system, we adopt the „Long short-term memory“ architecture.</p><p>To find the best LSTM setup, we train, test and compare LSTMs with two different structures: (1) the non-recurrent one-to-one structure, where the series of inputs are accumulated and fed into the LSTM; and (2) the recurrent many-to-many structure, where inputs gradually enter the LSTM (including LSTM forecasts from previous forecast time steps). The outputs of our LSTMs then feed into a readout layer that converts the hidden states into water level predictions. We hypothesize that the recurrent structure is the better structure because it better resembles the typical structure of differential equations for dynamic systems, as they would usually be used for hydro(geo)logical systems. We evaluate the comparison with the mean squared error as test metric, and conclude that the recurrent many-to-many LSTM performs better for the analyzed complex situations. It also produces plausible predictions with reasonable accuracy for seven days prediction horizon.</p><p>Furthermore, we analyze the impact of preprocessing meteorological data to evapotranspiration data using typical ETA models. Inserting knowledge into the LSTM in the form of ETA models (rather than implicitly having the LSTM learn the ETA relations) leads to superior prediction results. This finding aligns well with current ideas on physically-inspired machine learning.</p><p>As an additional validation step, we investigate whether our ANN is able to correctly identify both anthropogenic and natural influences and their interaction. To this end, we investigate two comparable pumping events under different meteorological conditions. Results indicate that all individual and combined influences of input parameters on water levels can be represented well. The neural networks recognize correctly that the predominant precipitation and lower evapotranspiration during one pumping event leads to a lower decrease of the hydrograph.</p><p>To further demonstrate the capability of the trained neural network, scenarios of pumping events are created and simulated.</p><p>In conclusion, we show that more robust and accurate predictions of moor water levels can be obtained if available physical knowledge of the modeled system is used to design and train the neural network. The artificial neural network can be a useful instrument to assess the impact of water abstraction by quantifying the anthropogenic influence.</p>

scholarly journals Pengaruh Cahaya dan Kualitas Citra dalam Klasifikasi Kematangan Pisang Cavendish Berdasarkan Ciri Warna Menggunakan Artificial Neural Network

2021 ◽  
Vol 21 (1) ◽  
pp. 215-228
Author(s):  
Aditya Dwi Putro ◽  
Arief Hermawan
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Learning Rate ◽  
Artificial Neural

Buah pisang merupakan komoditas yang memberikan kontribusi besar terhadap angka produksi buah nasional maupun internasional. Pemerintah melalui Badan Standarisasi Nasional menetapkan standar untuk buah pisang, menjaga mutu buah pisang. Tujuan dari penelitian ini adalah menganalisa pengaruh cahaya dan kualitas citra dalam mengklasifikasikan tingkat kematangan buah pisang berdasarkan ciri warna buah pisang di Kebun Pisang Cavendish kabupaten banyumas jawa tengah sesuai dengan SNI 7422:2009[1]. Pisang yang terdapat di Kebun Pisang Cavendish ini beraneka ragam kualitas, sebagai buah lokal yang memiliki nilai ekonomi tinggi dan memiliki potensi pasar yang masih terbuka luas, pisang menjadi salah satu komoditas buah-buahan yang dapat diandalkan. Permasalahan yang sering ditemukan selain resource dan ketelitian yakni kurang tepatnya dan kurang pengetahuannya karyawan dalam membedakan tingkat kematangan pisang terutama karyawan baru. Artificial Neural Network digunakan sebagai metode dalam proses pengklasifikasian. Dataset pada penelitian ini adalah 80 citra buah pisang yang diambil per tandan terdiri dari 40 tandan citra pisang Cavendish yang diambil di pagi hari dengan kualitas citra bagus 20 dan kualitas citra tidak bagus 20, 40 tandan citra pisang Cavendish yang diambil di sore hari dengan kualitas citra bagus 20 dan kualitas citra tidak bagus 20. Tingkat kematangan pisang pada penelitian ini yaitu mentah dan matang. pengujian menghasilkan Akurasi tertinggi dalam proses klasifikasi kategori buah pisang cavendish menggunakan epoch 5000, goal 0.0001 dan learning rate 0.1 dengan jumlah akurasi sebesar 100% dengan model trainlm dan waktu 1.6 detik.

The use of genetic algorithm and self-updating artificial neural network for the inverse design of cabin environment

2016 ◽  
Vol 26 (3) ◽  
pp. 347-354 ◽  
Author(s):  
Tian-hu Zhang ◽  
Xue-yi You
Keyword(s):  
Neural Network ◽  
Fluid Dynamics ◽  
Genetic Algorithm ◽  
Computational Fluid Dynamics ◽  
Artificial Neural Network ◽  
Computational Cost ◽  
The Self ◽  
Inverse Design ◽  
Artificial Neural ◽  
The Impact

The inverse process of computational fluid dynamics was used to explore the expected indoor environment with the preset objectives. An inverse design method integrating genetic algorithm and self-updating artificial neural network is presented. To reduce the computational cost and eliminate the impact of prediction error of artificial neural network, a self-updating artificial neural network is proposed to realize the self-adaption of computational fluid dynamics database, where all the design objectives of solutions are obtained by computational fluid dynamics instead of artificial neural network. The proposed method was applied to the inverse design of an MD-82 aircraft cabin. The result shows that the performance of artificial neural network is improved with the increase of computational fluid dynamics database. When the number of computational fluid dynamics cases is more than 80, the success rate of artificial neural network increases to more than 40%. Comparing to genetic algorithm and computational fluid dynamics, the proposed hybrid method reduces about 53% of the computational cost. The pseudo solutions are avoided when the self-updating artificial neural network is adopted. In addition, the number of computational fluid dynamics cases is determined automatically, and the requirement of human adjustment is avoided.

scholarly journals SEIRDQ: A COVID-19 case projection modeling framework using ANN to model quarantine

2021 ◽  
Author(s):  
Harish Chandra ◽  
Xianwei Meng ◽  
Arman Margaryan
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Data Sources ◽  
Modeling Framework ◽  
Novel Approach ◽  
Out Of Sample ◽  
Artificial Neural ◽  
Model Versus ◽  
Artificial Neural Network Ann ◽  
The Impact

We propose and implement a novel approach to model the evolution of COVID-19 pandemic and predict the daily COVID-19 cases (infected, recovered and dead). Our model builds on the classical SEIR-based framework by adding additional compartments to capture recovered, dead and quarantined cases. Quarantine impacts are modeled using an Artificial Neural Network (ANN), leveraging alternative data sources such as the Google mobility reports. Since our model captures the impact of lockdown policies through the quarantine functions we designed, it is able to model and predict future waves of COVID-19 cases. We also benchmark out-of-sample predictions from our model versus those from other popular COVID-19 case projection models.

scholarly journals Klasifikasi Tanaman Obat Berdasarkan Ekstraksi Fitur Morfologi Daun Menggunakan Jaringan Syaraf Tiruan

2018 ◽  
Vol 5 (2) ◽  
pp. 169-174
Author(s):  
Kana Saputra S ◽  
Mochammad Iswan Perangin-Angin
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Feature Extraction ◽  
Medicinal Plants ◽  
Medicinal Plant ◽  
Leaf Morphology ◽  
Recognition Rate ◽  
Learning Rate ◽  
Artificial Neural ◽  
Backpropagation Artificial Neural Network

Abstrak Indonesia telah lama mengenal dan menggunakan tanaman yang berkhasiat sebagai obat. Dari banyaknya tanaman obat yang ada di dunia, 80% tanaman obat tumbuh di hutan tropika yang berada di Indonesia. Sekitar 28.000 spesies tanaman tumbuh dan 1.000 spesies diantaranya telah digunakan sebagai  tanaman obat. Dengan banyaknya spesies tanaman obat dan tingkat kemiripan yang tinggi dapat menyebabkan kesalahan dalam proses identifikasi jenis tanaman obat. Sehingga dibutuhkan bantuan komputer untuk mengenali jenis tanaman obat tersebut. Tujuan dari penelitian ini adalah untuk mengidentifikasi jenis tanaman obat menggunakan jaringan syaraf tiruan backpropagation berdasarkan ekstraksi fitur morfologi daun. Hasilnya menujukkan bahwa perubahan nilai learning rate mempengaruhi hasil identifikasi jenis tanaman obat berdasarkan fitur morfologi daun. Hasil perhitungan rata-rata nilai recognition rate sebesar 90% untuk data training dan 75,56% untuk data testing terjadi saat learning rate 0,01. Nilai learning rate terbaik untuk identifikasi jenis tanaman obat adalah 0,01 dengan jumlah rata-rata epoch sebesar 11,67 dan MSE sebesar 0,13. Ini menunjukkan bahwa metode ekstraksi fitur morfologi daun dan algoritma jaringan syaraf tiruan backpropagation sangat baik digunakan untuk mengidentifkasi jenis tanaman obat. Kata Kunci: Ekstraksi Fitur, Jaringan Syaraf Tiruan Backpropagation, Morfologi Daun, Tanaman Obat Abstract Indonesia has known and used a nutritious plant as a medicine. most of the medicinal plants in the world that is 80% of medicinal plants grown in tropical forests in Indonesia. the plant grows about 28,000 species and 1,000 species of which have been used as medicinal plants. Many species of medicinal plants with a high degree of similarity can cause errors in the process of identifying medicinal plants. Because the problem was needed computer assistance to recognize the types of medicinal plants. This research proposed to identify species of medicinal plants using backpropagation artificial neural network based on leaf morphological feature extraction. The results showed that changes in the value of learning rate influence the identification of medicinal plant species based on leaf morphology features. The calculation average of recognition rate is 90% for training data and 75.56% for data testing occurs at learning rate 0.01. The best learning rate for plant species identification is 0.01 with epoch average is 11.67 and MSE is 0.13. The results of this research concluded that the leaf morphology feature extraction method and backpropagation artificial neural network algorithm are very well used to identify the types of medicinal plants. Keywords: Backpropagation Artificial Neural Network, Feature Extraction, Leaf Morphology, Medicinal Plant

scholarly journals Thermal field prediction in DED manufacturing process using Artificial Neural Network

2021 ◽  
Author(s):  
Seifallah Fetni ◽  
Quy Duc Thinh Pham ◽  
Van Xuan Tran ◽  
Laurent Duchêne ◽  
Hoang Son Tran ◽  
...  
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Artificial Neural Network ◽  
Finite Element ◽  
Manufacturing Process ◽  
Thermal Field ◽  
Activation Function ◽  
Tungsten Carbides ◽  
Artificial Neural ◽  
The Impact

In the last decade, machine learning is increasingly attracting researchers in several scientific areas and, in particular, in the additive manufacturing field. Meanwhile, this technique remains as a black box technique for many researchers. Indeed, it allows obtaining novel insights to overcome the limitation of classical methods, such as the finite element method, and to take into account multi-physical complex phenomena occurring during the manufacturing process. This work presents a comprehensive study for implementing a machine learning technique (artificial neural network) to predict the thermal field evolution during the direct energy deposition of 316L stainless steel and tungsten carbides. The framework consists of a finite element thermal model and a neural network. The influence of the number of hidden layers and the number of nodes in each layer was also investigated. The results showed that an architecture based on 3 or 4 hidden layers and the rectified linear unit as the activation function lead to obtaining a high fidelity prediction with an accuracy exceeding 99%. The impact of the chosen architecture on the model accuracy and CPU usage was also highlighted. The proposed framework can be used to predict the thermal field when simulating multi-layer deposition.

Sign in / Sign up

Export Citation Format

Share Document