scholarly journals Evaluation of Data Augmentation Techniques for Facial Expression Recognition Systems

Electronics ◽  
2020 ◽  
Vol 9 (11) ◽  
pp. 1892
Author(s):  
Simone Porcu ◽  
Alessandro Floris ◽  
Luigi Atzori
Keyword(s):  
Facial Expression ◽  
Facial Expression Recognition ◽  
Data Augmentation ◽  
Generative Adversarial Networks ◽  
Geometric Transformation ◽  
Expression Recognition ◽  
Learning Approaches ◽  
Geometric Transformations ◽  
Augmentation Techniques ◽  
The Impact

Most Facial Expression Recognition (FER) systems rely on machine learning approaches that require large databases for an effective training. As these are not easily available, a good solution is to augment the databases with appropriate data augmentation (DA) techniques, which are typically based on either geometric transformation or oversampling augmentations (e.g., generative adversarial networks (GANs)). However, it is not always easy to understand which DA technique may be more convenient for FER systems because most state-of-the-art experiments use different settings which makes the impact of DA techniques not comparable. To advance in this respect, in this paper, we evaluate and compare the impact of using well-established DA techniques on the emotion recognition accuracy of a FER system based on the well-known VGG16 convolutional neural network (CNN). In particular, we consider both geometric transformations and GAN to increase the amount of training images. We performed cross-database evaluations: training with the "augmented" KDEF database and testing with two different databases (CK+ and ExpW). The best results were obtained combining horizontal reflection, translation and GAN, bringing an accuracy increase of approximately 30%. This outperforms alternative approaches, except for the one technique that could however rely on a quite bigger database.

scholarly journals eXnet: An Efficient Approach for Emotion Recognition in the Wild

Sensors ◽  
2020 ◽  
Vol 20 (4) ◽  
pp. 1087
Author(s):  
Muhammad Naveed Riaz ◽  
Yao Shen ◽  
Muhammad Sohail ◽  
Minyi Guo
Keyword(s):  
Facial Expression ◽  
Emotion Recognition ◽  
Facial Expression Recognition ◽  
Data Augmentation ◽  
Raspberry Pi ◽  
Expression Recognition ◽  
Extensive Evaluation ◽  
Benchmark Datasets ◽  
Augmentation Techniques ◽  
In The Wild

Facial expression recognition has been well studied for its great importance in the areas of human–computer interaction and social sciences. With the evolution of deep learning, there have been significant advances in this area that also surpass human-level accuracy. Although these methods have achieved good accuracy, they are still suffering from two constraints (high computational power and memory), which are incredibly critical for small hardware-constrained devices. To alleviate this issue, we propose a new Convolutional Neural Network (CNN) architecture eXnet (Expression Net) based on parallel feature extraction which surpasses current methods in accuracy and contains a much smaller number of parameters (eXnet: 4.57 million, VGG19: 14.72 million), making it more efficient and lightweight for real-time systems. Several modern data augmentation techniques are applied for generalization of eXnet; these techniques improve the accuracy of the network by overcoming the problem of overfitting while containing the same size. We provide an extensive evaluation of our network against key methods on Facial Expression Recognition 2013 (FER-2013), Extended Cohn-Kanade Dataset (CK+), and Real-world Affective Faces Database (RAF-DB) benchmark datasets. We also perform ablation evaluation to show the importance of different components of our architecture. To evaluate the efficiency of eXnet on embedded systems, we deploy it on Raspberry Pi 4B. All these evaluations show the superiority of eXnet for emotion recognition in the wild in terms of accuracy, the number of parameters, and size on disk.

scholarly journals Facial Expression Recognition of Instructor Using Deep Features and Extreme Learning Machine

2021 ◽  
Vol 2021 ◽  
pp. 1-17
Author(s):  
Yusra Khalid Bhatti ◽  
Afshan Jamil ◽  
Nudrat Nida ◽  
Muhammad Haroon Yousaf ◽  
Serestina Viriri ◽  
...  
Keyword(s):  
Facial Expression ◽  
Extreme Learning Machine ◽  
Facial Expressions ◽  
Classroom Environment ◽  
Facial Expression Recognition ◽  
Parameter Tuning ◽  
Expression Recognition ◽  
Fast Learning ◽  
Learning Machine ◽  
The Impact

Classroom communication involves teacher’s behavior and student’s responses. Extensive research has been done on the analysis of student’s facial expressions, but the impact of instructor’s facial expressions is yet an unexplored area of research. Facial expression recognition has the potential to predict the impact of teacher’s emotions in a classroom environment. Intelligent assessment of instructor behavior during lecture delivery not only might improve the learning environment but also could save time and resources utilized in manual assessment strategies. To address the issue of manual assessment, we propose an instructor’s facial expression recognition approach within a classroom using a feedforward learning model. First, the face is detected from the acquired lecture videos and key frames are selected, discarding all the redundant frames for effective high-level feature extraction. Then, deep features are extracted using multiple convolution neural networks along with parameter tuning which are then fed to a classifier. For fast learning and good generalization of the algorithm, a regularized extreme learning machine (RELM) classifier is employed which classifies five different expressions of the instructor within the classroom. Experiments are conducted on a newly created instructor’s facial expression dataset in classroom environments plus three benchmark facial datasets, i.e., Cohn–Kanade, the Japanese Female Facial Expression (JAFFE) dataset, and the Facial Expression Recognition 2013 (FER2013) dataset. Furthermore, the proposed method is compared with state-of-the-art techniques, traditional classifiers, and convolutional neural models. Experimentation results indicate significant performance gain on parameters such as accuracy, F1-score, and recall.

scholarly journals A Review on Finding Efficient Approach to Detect Customer Emotion Analysis using Deep Learning Analysis

2021 ◽  
Vol 3 (2) ◽  
pp. 95-113
Author(s):  
Kottilingam Kottursamy
Keyword(s):  
Deep Learning ◽  
Facial Expression Recognition ◽  
Data Augmentation ◽  
Expression Recognition ◽  
Generalization Error ◽  
Augmentation Techniques ◽  
Parallel Feature ◽  
Improved Accuracy ◽  
Learning Analysis

The role of facial expression recognition in social science and human-computer interaction has received a lot of attention. Deep learning advancements have resulted in advances in this field, which go beyond human-level accuracy. This article discusses various common deep learning algorithms for emotion recognition, all while utilising the eXnet library for achieving improved accuracy. Memory and computation, on the other hand, have yet to be overcome. Overfitting is an issue with large models. One solution to this challenge is to reduce the generalization error. We employ a novel Convolutional Neural Network (CNN) named eXnet to construct a new CNN model utilising parallel feature extraction. The most recent eXnet (Expression Net) model improves on the previous model's inaccuracy while having many fewer parameters. Data augmentation techniques that have been in use for decades are being utilized with the generalized eXnet. It employs effective ways to reduce overfitting while maintaining overall size under control.

Efficient facial expression recognition based on convolutional neural network

2021 ◽  
Vol 25 (1) ◽  
pp. 139-154
Author(s):  
Yongxiang Cai ◽  
Jingwen Gao ◽  
Gen Zhang ◽  
Yuangang Liu
Keyword(s):  
Neural Network ◽  
Facial Expression ◽  
Convolutional Neural Network ◽  
Facial Expression Recognition ◽  
Data Augmentation ◽  
Recognition Rate ◽  
Classification Performance ◽  
Driving Safety ◽  
Expression Recognition ◽  
Before And After

The goal of research in Facial Expression Recognition (FER) is to build a robust and strong recognizability model. In this paper, we propose a new scheme for FER systems based on convolutional neural network. Part of the regular convolution operation is replaced by depthwise separable convolution to reduce the number of parameters and the computational workload; the self-adaption joint loss function is adopted to improve the classification performance. In addition, we balance our train set through data augmentation, and we preprocess the input images through illumination processing, face detection, and other methods, effectively maximizing the expression recognition rate. Experiments to validate our methods are conducted based on the TensorFlow platform and Fer2013 dataset. We analyze the experimental results before and after train set balancing and network model modification, and we compare our results with those of other researchers. The results show that our method is effective at increasing the expression recognition rate under the same experiment conditions. We further conduct an experiment on our own expression dataset relevant to driving safety, and it yields similar results.

Sign in / Sign up

Export Citation Format

Share Document