Challenging the Recognition of Facial Expression via Deep Learning

2014 ◽  
Vol 571-572 ◽  
pp. 717-720
Author(s):  
De Kun Hu ◽  
Yong Hong Liu ◽  
Li Zhang ◽  
Gui Duo Duan
Keyword(s):  
Neural Network ◽  
Facial Expression ◽  
Gradient Descent ◽  
Deep Neural Network ◽  
Back Propagation ◽  
Stochastic Gradient Descent ◽  
Excellent Performance ◽  
Proposed Model ◽  
Input Layer ◽  
Fully Connected

A deep Neural Network model was trained to classify the facial expression in unconstrained images, which comprises nine layers, including input layer, convolutional layer, pooling layer, fully connected layers and output layer. In order to optimize the model, rectified linear units for the nonlinear transformation, weights sharing for reducing the complexity, “mean” and “max” pooling for subsample, “dropout” for sparsity are applied in the forward processing. With large amounts of hard training faces, the model was trained via back propagation method with stochastic gradient descent. The results of shows the proposed model achieves excellent performance.

scholarly journals Neural ODEs as the deep limit of ResNets with constant weights

2020 ◽  
pp. 1-41 ◽  
Author(s):  
Benny Avelin ◽  
Kaj Nyström
Keyword(s):  
Neural Network ◽  
Gradient Descent ◽  
Deep Neural Network ◽  
Theoretical Foundation ◽  
Stochastic Gradient ◽  
Loss Functions ◽  
Stochastic Gradient Descent ◽  
Decay Estimates ◽  
Value Loss ◽  
Fokker Planck Equations

In this paper, we prove that, in the deep limit, the stochastic gradient descent on a ResNet type deep neural network, where each layer shares the same weight matrix, converges to the stochastic gradient descent for a Neural ODE and that the corresponding value/loss functions converge. Our result gives, in the context of minimization by stochastic gradient descent, a theoretical foundation for considering Neural ODEs as the deep limit of ResNets. Our proof is based on certain decay estimates for associated Fokker–Planck equations.

scholarly journals A 10-Year Probability Deep Neural Network Prediction Model for Lung Cancer

Cancers ◽  
2021 ◽  
Vol 13 (4) ◽  
pp. 928
Author(s):  
Hsiu-An Lee ◽  
Louis R. Chao ◽  
Chien-Yeh Hsu
Keyword(s):  
Neural Network ◽  
Lung Cancer ◽  
Prediction Model ◽  
Deep Neural Network ◽  
Clinical Decision Making ◽  
Stochastic Gradient Descent ◽  
Research Database ◽  
Proposed Model ◽  
Neural Network Prediction ◽  
Network Prediction

Cancer is the leading cause of death in Taiwan. According to the Cancer Registration Report of Taiwan’s Ministry of Health and Welfare, a total of 13,488 people suffered from lung cancer in 2016, making it the second-most common cancer and the leading cancer in men. Compared with other types of cancer, the incidence of lung cancer is high. In this study, the National Health Insurance Research Database (NHIRDB) was used to determine the diseases and symptoms associated with lung cancer, and a 10-year probability deep neural network prediction model for lung cancer was developed. The proposed model could allow patients with a high risk of lung cancer to receive an earlier diagnosis and support the physicians’ clinical decision-making. The study was designed as a cohort study. The subjects were patients who were diagnosed with lung cancer between 2000 and 2009, and the patients’ disease histories were back-tracked for a period, extending to ten years before the diagnosis of lung cancer. As a result, a total of 13 diseases were selected as the predicting factors. A nine layers deep neural network model was created to predict the probability of lung cancer, depending on the different pre-diagnosed diseases, and to benefit the earlier detection of lung cancer in potential patients. The model is trained 1000 times, the batch size is set to 100, the SGD (Stochastic gradient descent) optimizer is used, the learning rate is set to 0.1, and the momentum is set to 0.1. The proposed model showed an accuracy of 85.4%, a sensitivity of 72.4% and a specificity of 85%, as well as an 87.4% area under ROC (AUROC) (95%, 0.8604–0.8885) model precision. Based on data analysis and deep learning, our prediction model discovered some features that had not been previously identified by clinical knowledge. This study tracks a decade of clinical diagnostic records to identify possible symptoms and comorbidities of lung cancer, allows early prediction of the disease, and assists more patients with early diagnosis.

Wireless Brain Wave Classification for Alzheimer’s Patients via Efficient Neural Network Computation

2018 ◽  
Vol 10 (03) ◽  
pp. 1850004
Author(s):  
Grant Sheen
Keyword(s):  
Neural Network ◽  
Gradient Descent ◽  
Descent Method ◽  
Stochastic Gradient Descent ◽  
Normal Person ◽  
Gradient Descent Method ◽  
Step Size ◽  
Brain Wave ◽  
Proposed Model ◽  
Wireless Recording

Wireless recording and real time classification of brain waves are essential steps towards future wearable devices to assist Alzheimer’s patients in conveying their thoughts. This work is concerned with efficient computation of a dimension-reduced neural network (NN) model on Alzheimer’s patient data recorded by a wireless headset. Due to much fewer sensors in wireless recording than the number of electrodes in a traditional wired cap and shorter attention span of an Alzheimer’s patient than a normal person, the data is much more restrictive than is typical in neural robotics and mind-controlled games. To overcome this challenge, an alternating minimization (AM) method is developed for network training. AM minimizes a nonsmooth and nonconvex objective function one variable at a time while fixing the rest. The sub-problem for each variable is piecewise convex with a finite number of minima. The overall iterative AM method is descending and free of step size (learning parameter) in the standard gradient descent method. The proposed model, trained by the AM method, significantly outperforms the standard NN model trained by the stochastic gradient descent method in classifying four daily thoughts, reaching accuracies around 90% for Alzheimer’s patient. Curved decision boundaries of the proposed model with multiple hidden neurons are found analytically to establish the nonlinear nature of the classification.

scholarly journals Perancangan Model Jaringan Syaraf Tiruan Propagasi Balik Untuk Mengestimasi Curah Hujan

2012 ◽  
Vol 9 (2) ◽  
Author(s):  
Elohansen Padang
Keyword(s):  
Neural Network ◽  
Gradient Descent ◽  
Back Propagation ◽  
Activation Function ◽  
Back Propagation Neural Network ◽  
Rainfall Data ◽  
Training Method ◽  
Output Layer ◽  
Input Layer ◽  
Hidden Layer

This research was conducted to investigate the ability of backpropagation artificial neural network in estimating rainfall. Neural network used consists of input layer, 2 hidden layers and output layer. Input layer consists of 12 neurons that represent each input; first hidden layer consists of 12 neurons with activation function tansig, while the second hidden layer consists of 24 neurons with activation function logsig. Output layer consists of 1 neuron with activation function purelin. Training method used is the method of gradient descent with momentum. Training method used is the method of gradient descent with momentum. Learning rate and momentum parameters defined respectively by 0.1 and 0.5. To evaluate the performance of the network model to recognize patterns of rainfall data is used in Biak city rainfall data from January 1997 - December 2008 (12 years). This data is divided into 2 parts, namely training and testing data using rainfall data from January 1997-December 2005 and data estimation using rainfall data from January 2006-December 2008. From the results of this study concluded that rainfall patterns Biak town can be recognized quite well by the model of back propagation neural network. The test results and estimates of the model results testing the value of R = 0.8119, R estimate = 0.53801, MAPE test = 0.1629, and MAPE estimate = 0.6813.

scholarly journals MonolithNet: Training monolithic deep neural networks via a partitioned training strategy

2018 ◽  
Vol 4 (1) ◽  
pp. 3
Author(s):  
Rene Bidart ◽  
Alexander Wong
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Gradient Descent ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Neural Net ◽  
Training Strategy ◽  
Effective Manner

In this study, we explore the training of monolithic deep neural net-works in an effective manner. One of the biggest challenges withtraining such networks to the desired level of accuracy is the dif-ficulty in converging to a good solution using iterative optimizationmethods such as stochastic gradient descent due to the enormousnumber of parameters that need to be learned. To achieve this,we introduce a partitioned training strategy, where proxy layersare connected to different partitions of a deep neural network toenable isolated training of a much smaller number of parametersto convergence. To illustrate the efficacy of this training strategy,we introduce MonolithNet, a massive residual deep neural networkconsisting of 437 million parameters. The trained MonolithNet wasable to achieve a top-1 accuracy of 97% on the CIFAR10 imageclassification dataset, which demonstrates the feasibility of the pro-posed training strategy for training monolithic deep neural networksto high accuracies.

KLASIFIKASI CITRA GENUS PANTHERA MENGGUNAKAN METODE CONVOLUTIONAL NEURAL NETWORK (CNN)

2019 ◽  
Vol 24 (3) ◽  
pp. 220-228
Author(s):  
Gusti Alfahmi Anwar ◽  
Desti Riminarsih
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Input Layer ◽  
Fully Connected

Panthera merupakan genus dari keluarga kucing yang memiliki empat spesies popular yaitu, harimau, jaguar, macan tutul, singa. Singa memiliki warna keemasan dan tidak memilki motif, harimau memiliki motif loreng dengan garis-garis panjang, jaguar memiliki tubuh yang lebih besar dari pada macan tutul serta memiliki motif tutul yang lebih lebar, sedangkan macan tutul memiliki tubuh yang sedikit lebih ramping dari pada jaguar dan memiliki tutul yang tidak terlalu lebar. Pada penelitian ini dilakukan klasifikasi genus panther yaitu harimau, jaguar, macan tutul, dan singa menggunakan metode Convolutional Neural Network. Model Convolutional Neural Network yang digunakan memiliki 1 input layer, 5 convolution layer, dan 2 fully connected layer. Dataset yang digunakan berupa citra harimau, jaguar, macan tutul, dan singa. Data training terdiri dari 3840 citra, data validasi sebanyak 960 citra, dan data testing sebanyak 800 citra. Hasil akurasi dari pelatihan model untuk training yaitu 92,31% dan validasi yaitu 81,88%, pengujian model menggunakan dataset testing mendapatan hasil 68%. Hasil akurasi prediksi didapatkan dari nilai F1-Score pada pengujian didapatkan sebesar 78% untuk harimau, 70% untuk jaguar, 37% untuk macan tutul, 74% untuk singa. Macan tutul mendapatkan akurasi terendah dibandingkan 3 hewan lainnya tetapi lebih baik dibandingkan hasil penelitian sebelumnya.

Sign in / Sign up

Export Citation Format

Share Document