Polynomial Learning Rate Policy with Warm Restart for Deep Neural Network

2019 ◽  
Author(s):  
Purnendu Mishra ◽  
Kishor Sarawadekar
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Learning Rate ◽  
Rate Policy

scholarly journals Analisis Sentimen Twitter untuk Menilai Opini Terhadap Perusahaan Publik Menggunakan Algoritma Deep Neural Network

2021 ◽  
Vol 7 (2) ◽  
pp. 108-118
Author(s):  
Erwin Yudi Hidayat ◽  
Raindy Wicaksana Hardiansyah ◽  
Affandy Affandy
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Application Programming Interface ◽  
Learning Rate ◽  
Application Programming ◽  
Hidden Layer ◽  
Programming Interface

Dalam menaikkan kinerja serta mengevaluasi kualitas, perusahaan publik membutuhkan feedback dari masyarakat / konsumen yang bisa didapat melalui media sosial. Sebagai pengguna media sosial Twitter terbesar ketiga di dunia, tweet yang beredar di Indonesia memiliki potensi meningkatkan reputasi dan citra perusahaan. Dengan memanfaatkan algoritma Deep Neural Network (DNN), neural network yang tersusun dari layer yang jumlahnya lebih dari satu, didapati hasil analisa sentimen pada Twitter berbahasa Indonesia menjadi lebih baik dibanding dengan metode lainnya. Penelitian ini menganalisa sentimen melalui tweet dari masyarakat Indonesia terhadap sejumlah perusahaan publik dengan menggunakan DNN. Data Tweet sebanyak 5504 record didapat dengan melakukan crawling melalui Application Programming Interface (API) Twitter yang selanjutnya dilakukan preprocessing (cleansing, case folding, formalisasi, stemming, dan tokenisasi). Proses labeling dilakukan untuk 3902 record dengan memanfaatkan aplikasi Sentiment Strength Detection. Tahap pelatihan model dilakukan menggunakan algoritma DNN dengan variasi jumlah hidden layer, susunan node, dan nilai learning rate. Eksperimen dengan proporsi data training dan testing sebesar 90:10 memberikan hasil performa terbaik. Model tersusun dengan 3 hidden layer dengan susunan node tiap layer pada model tersebut yaitu 128, 256, 128 node dan menggunakan learning rate sebesar 0.005, model mampu menghasilkan nilai akurasi mencapai 88.72%. 

Hyperbolic Tangent Polynomial Parity Cyclic Learning Rate for Deep Neural Network

2021 ◽  
pp. 448-460
Author(s):  
Hong Lin ◽  
Xiaodong Yang ◽  
Binyan Wu ◽  
Ruyan Xiong
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Learning Rate ◽  
Hyperbolic Tangent

scholarly journals Customer Decision Prediction Using Deep Neural Network on Telco Customer Churn Data

2021 ◽  
Vol 21 (2) ◽  
pp. 122
Author(s):  
Hiya Nalatissifa ◽  
Hilman Ferdinandus Pardede
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Random Search ◽  
Telecommunications Industry ◽  
Learning Rate ◽  
Gradient Algorithm ◽  
Stochastic Gradient Descent ◽  
Business World ◽  
Customer Churn ◽  
Hidden Layer

Customer churn is the most important problem in the business world, especially in the telecommunications industry, because it greatly influences company profits. Getting new customers for a company is much more difficult and expensive than retaining existing customers. Machine learning, part of data mining, is a sub-field of artificial intelligence widely used to make predictions, including predicting customer churn. Deep neural network (DNN) has been used for churn prediction, but selecting hyperparameters in modeling requires more time and effort, making the process more challenging for the researcher. Therefore, the purpose of this study is to propose a better architecture for the DNN algorithm by using a hard tuner to obtain more optimal hyperparameters. The tuning hyperparameter used is random search in determining the number of nodes in each hidden layer, dropout, and learning rate. In addition, this study also uses three variations of the number of hidden layers, two variations of the activation function, namely rectified linear unit (ReLu) and Sigmoid, then uses five variations of the optimizer (stochastic gradient descent (SGD), adaptive moment estimation (Adam), adaptive gradient algorithm (Adagrad), Adadelta, and root mean square propagation (RMSprop)). Experiments show that the DNN algorithm using hyperparameter tuning random search produces a performance value of 83.09 % accuracy using three hidden layers, the number of nodes in each hidden layer is [20, 35, 15], using the RMSprop optimizer, dropout 0.1, the learning rate is 0.01, with the fastest tuning time of 21 seconds. Better than modeling using k-nearest neighbor (K-NN), random forest (RF), and decision tree (DT) as comparison algorithms.

scholarly journals PREDICTION OF HOTEL BOOKING CANCELLATION USING DEEP NEURAL NETWORK AND LOGISTIC REGRESSION ALGORITHM

2021 ◽  
Vol 18 (1) ◽  
pp. 1-8
Author(s):  
Nugroho Adi Putro ◽  
Rendi Septian ◽  
Widiastuti Widiastuti ◽  
Mawadatul Maulidah ◽  
Hilman Ferdinandus Pardede
Keyword(s):  
Neural Network ◽  
Logistic Regression ◽  
Deep Neural Network ◽  
Demand Management ◽  
The State ◽  
Learning Rate ◽  
Optimal Value ◽  
Reservation System ◽  
Logistic Regression Algorithm ◽  
Hotel Booking

Booking cancellation is a key aspect of hotel revenue management as it affects the room reservation system. Booking cancellation has a significant effect on revenue which has a significant impact on demand management decisions in the hotel industry. In order to reduce the cancellation effect, the hotel applies the cancellation model as the key to addressing this problem with the machine learning-based system developed. In this study, using a data collection from the Kaggle website with the name hotel-booking-demand dataset. The research objective was to see the performance of the deep neural network method which has two classification classes, namely cancel and not. Then optimized with optimizers and learning rate. And to see which attribute has the most role in determining the level of accuracy using the Logistic Regression algorithm. The results obtained are the Encoder-Decoder Layer by adamax optimizer which is higher than that of the Decoder-Encoder by adadelta optimizer. After adding the learning rate, the adamax accuracy for the encoders and encoders decreased for a learning rate of 0.001. The results of the top three ranks of each neural network after adding the learning rate show that the smaller the learning rate, the higher the accuracy, but we don't know what the optimal value for the learning rate is. By using the Logistic Regression algorithm by eliminating several attributes, the most influential level of accuracy is the state attribute and total_of_special_requests, where accuracy increases when the state attribute is removed because there are 177 variations in these attributes

scholarly journals PACL: Piecewise Arc Cotangent Decay Learning Rate for Deep Neural Network Training

IEEE Access ◽  
2020 ◽  
Vol 8 ◽  
pp. 112805-112813 ◽  
Author(s):  
Haixu Yang ◽  
Jihong Liu ◽  
Hongwei Sun ◽  
Henggui Zhang
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Learning Rate ◽  
Neural Network Training ◽  
Network Training

scholarly journals Deep Neural Network untuk Prediksi Stroke

2021 ◽  
Vol 7 (3) ◽  
pp. 443
Author(s):  
Anas Faisal ◽  
Agus Subekti
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Learning Rate ◽  
Hidden Layer

Pada Tahun 2019 Organisasi Kesehatan Dunia (WHO) mendudukkan stroke sebagai tujuh dari sepuluh penyebab utama kematian. Kementerian Kesehatan menggolongkan stroke sebagai penyakit katastropik karena dampaknya luas secara ekonomi dan sosial. Oleh karena itu, diperlukan peran dari teknologi informasi untuk memprediksi stroke guna pencegahan dan perawatan dini. Analisis data yang memiliki kelas tidak seimbang mengakibatkan ketidakakuratan dalam memprediksi stroke. Penelitian ini membandingkan tiga teknik oversampling untuk mendapatkan model prediksi yang lebih baik. Data kelas yang sudah diseimbangkan diuji menggunakan tiga model Arsitektur Deep Neural Network (DNN) dengan melakukan optimasi pada beberapa parameter yaitu optimizer, learning rate dan epoch. Hasil paling baik didapatkan teknik oversampling SMOTETomek dan Arsitektur DNN dengan lima hidden layer, optimasi Adam, learning rate 0.001 dan jumlah epoch 500. Skor akurasi, presisi, recall, dan f1-score masing-masing mendapatkan 0.96, 0.9614, 0.9608 dan 0.9611.

Sign in / Sign up

Export Citation Format

Share Document