scholarly journals Facial Expression Recognition Using Computer Vision: A Systematic Review

2019 ◽  
Vol 9 (21) ◽  
pp. 4678 ◽  
Author(s):  
Daniel Canedo ◽  
António J. R. Neves
Keyword(s):  
Systematic Review ◽  
Computer Vision ◽  
Facial Expression ◽  
Emotion Recognition ◽  
Facial Expression Recognition ◽  
Principal Component ◽  
Local Binary Patterns ◽  
Future Research ◽  
Expression Recognition ◽  
Contemporary World

Emotion recognition has attracted major attention in numerous fields because of its relevant applications in the contemporary world: marketing, psychology, surveillance, and entertainment are some examples. It is possible to recognize an emotion through several ways; however, this paper focuses on facial expressions, presenting a systematic review on the matter. In addition, 112 papers published in ACM, IEEE, BASE and Springer between January 2006 and April 2019 regarding this topic were extensively reviewed. Their most used methods and algorithms will be firstly introduced and summarized for a better understanding, such as face detection, smoothing, Principal Component Analysis (PCA), Local Binary Patterns (LBP), Optical Flow (OF), Gabor filters, among others. This review identified a clear difficulty in translating the high facial expression recognition (FER) accuracy in controlled environments to uncontrolled and pose-variant environments. The future efforts in the FER field should be put into multimodal systems that are robust enough to face the adversities of real world scenarios. A thorough analysis on the research done on FER in Computer Vision based on the selected papers is presented. This review aims to not only become a reference for future research on emotion recognition, but also to provide an overview of the work done in this topic for potential readers.

Facial Expression Recognition Based on the Texture Features of Global Principal Component and Local Boundary

2014 ◽  
Vol 548-549 ◽  
pp. 1110-1117 ◽  
Author(s):  
Wei Hao Zheng ◽  
Wei Wang ◽  
Yi De Ma
Keyword(s):  
Facial Expression ◽  
Facial Expression Recognition ◽  
Principal Component ◽  
Texture Features ◽  
Local Binary Patterns ◽  
Expression Recognition ◽  
Human Machine Interaction ◽  
Local Boundary ◽  
Facial Expression Analysis ◽  
Global And Local

Facial expression recognition is a key ingredient to either emotion analysis or pattern recognition, which is also an important component in human-machine interaction. In facial expression analysis, one of the well-known methods to obtain the texture of expressions is local binary patterns (LBP) which compares pixels in local region and encodes the comparison result in forms of histogram. However, we argue that the textures of expressions are not accurate and still contain some irrelevant information, especially in the region between eyes and mouth. In this paper, we propose a compound method to recognize expressions by applying local binary patterns to global and local images processed by bidirectional principal component analysis (BDPCA) reconstruction and morphologic preprocess, respectively. It proves that our method can be applied for recognizing expressions by using texture features of global principal component and local boundary and achieves a considerable high accuracy.

A Hybrid Approach for Facial Expression Recognition Using Extended Local Binary Patterns and Principal Component Analysis

2021 ◽  
pp. 517-544
Author(s):  
Gopal Krishan Prajapat ◽  
Rakesh Kumar
Keyword(s):  
Principal Component Analysis ◽  
Feature Extraction ◽  
Facial Expression ◽  
Facial Expression Recognition ◽  
Local Binary Pattern ◽  
Recognition Rate ◽  
Principal Component ◽  
Local Binary Patterns ◽  
Component Analysis ◽  
Expression Recognition

Facial feature extraction and recognition plays a prominent role in human non-verbal interaction and it is one of the crucial factors among pose, speech, facial expression, behaviour and actions which are used in conveying information about the intentions and emotions of a human being. In this article an extended local binary pattern is used for the feature extraction process and a principal component analysis (PCA) is used for dimensionality reduction. The projections of the sample and model images are calculated and compared by Euclidean distance method. The combination of extended local binary pattern and PCA (ELBP+PCA) improves the accuracy of the recognition rate and also diminishes the evaluation complexity. The evaluation of proposed facial expression recognition approach will focus on the performance of the recognition rate. A series of tests are performed for the validation of algorithms and to compare the accuracy of the methods on the JAFFE, Extended Cohn-Kanade images database.

scholarly journals PROSES EKSTRAKSI DAN KLASIFIKASI CITRA EMOSI MENGGUNAKAN METODE PCA DAN CNN

JOUTICA ◽  
2021 ◽  
Vol 6 (2) ◽  
pp. 484
Author(s):  
Resty Wulanningrum ◽  
Anggi Nur Fadzila ◽  
Danar Putra Pamungkas
Keyword(s):  
Neural Network ◽  
Principal Component Analysis ◽  
Facial Expression ◽  
Convolutional Neural Network ◽  
Facial Expression Recognition ◽  
Principal Component ◽  
Component Analysis ◽  
Expression Recognition

Manusia secara alami menggunakan ekspresi wajah untuk berkomunikasi dan menunjukan emosi mereka dalam berinteraksi sosial. Ekspresi wajah termasuk kedalam komunikasi non-verbal yang dapat menyampaikan keadaan emosi seseorang kepada orang yang telah mengamatinya. Penelitian ini menggunakan metode Principal Component Analysis (PCA) untuk proses ekstraksi ciri pada citra ekspresi dan metode Convolutional Neural Network (CNN) sebagai prosesi klasifikasi emosi, dengan menggunakan data Facial Expression Recognition-2013 (FER-2013) dilakukan proses training dan testing untuk menghasilkan nilai akurasi dan pengenalan emosi wajah. Hasil pengujian akhir mendapatkan nilai akurasi pada metode PCA sebesar 59,375% dan nilai akurasi pada pengujian metode CNN sebesar 59,386%.

From Face to Facial Expression

2010 ◽  
pp. 217-238
Author(s):  
Zakia Hammal ◽  
Zakia Hammal
Keyword(s):  
Computer Vision ◽  
Facial Expression ◽  
Everyday Life ◽  
Facial Expression Recognition ◽  
Multidisciplinary Approach ◽  
Expression Recognition ◽  
Future Challenges ◽  
Facial Expression Classification ◽  
Expression Classification ◽  
Processing Steps

This chapter addresses recent advances in computer vision for facial expression classification. The authors present the different processing steps of the problem of automatic facial expression recognition. They describe the advances of each stage of the problem and review the future challenges towards the application of such systems to everyday life situations. The authors also introduce the importance of taking advantage of the human strategy by reviewing advances of research in psychology towards multidisciplinary approach for facial expression classification. Finally, the authors describe one contribution which aims at dealing with some of the discussed challenges.

A Hybrid Approach for Facial Expression Recognition Using Extended Local Binary Patterns and Principal Component Analysis

2019 ◽  
Vol 8 (2) ◽  
pp. 1-25
Author(s):  
Gopal Krishan Prajapat ◽  
Rakesh Kumar
Keyword(s):  
Principal Component Analysis ◽  
Feature Extraction ◽  
Facial Expression ◽  
Facial Expression Recognition ◽  
Local Binary Pattern ◽  
Recognition Rate ◽  
Principal Component ◽  
Component Analysis ◽  
Expression Recognition ◽  
Distance Method

Facial feature extraction and recognition plays a prominent role in human non-verbal interaction and it is one of the crucial factors among pose, speech, facial expression, behaviour and actions which are used in conveying information about the intentions and emotions of a human being. In this article an extended local binary pattern is used for the feature extraction process and a principal component analysis (PCA) is used for dimensionality reduction. The projections of the sample and model images are calculated and compared by Euclidean distance method. The combination of extended local binary pattern and PCA (ELBP+PCA) improves the accuracy of the recognition rate and also diminishes the evaluation complexity. The evaluation of proposed facial expression recognition approach will focus on the performance of the recognition rate. A series of tests are performed for the validation of algorithms and to compare the accuracy of the methods on the JAFFE, Extended Cohn-Kanade images database.

scholarly journals eXnet: An Efficient Approach for Emotion Recognition in the Wild

Sensors ◽  
2020 ◽  
Vol 20 (4) ◽  
pp. 1087
Author(s):  
Muhammad Naveed Riaz ◽  
Yao Shen ◽  
Muhammad Sohail ◽  
Minyi Guo
Keyword(s):  
Facial Expression ◽  
Emotion Recognition ◽  
Facial Expression Recognition ◽  
Data Augmentation ◽  
Raspberry Pi ◽  
Expression Recognition ◽  
Extensive Evaluation ◽  
Benchmark Datasets ◽  
Augmentation Techniques ◽  
In The Wild

Facial expression recognition has been well studied for its great importance in the areas of human–computer interaction and social sciences. With the evolution of deep learning, there have been significant advances in this area that also surpass human-level accuracy. Although these methods have achieved good accuracy, they are still suffering from two constraints (high computational power and memory), which are incredibly critical for small hardware-constrained devices. To alleviate this issue, we propose a new Convolutional Neural Network (CNN) architecture eXnet (Expression Net) based on parallel feature extraction which surpasses current methods in accuracy and contains a much smaller number of parameters (eXnet: 4.57 million, VGG19: 14.72 million), making it more efficient and lightweight for real-time systems. Several modern data augmentation techniques are applied for generalization of eXnet; these techniques improve the accuracy of the network by overcoming the problem of overfitting while containing the same size. We provide an extensive evaluation of our network against key methods on Facial Expression Recognition 2013 (FER-2013), Extended Cohn-Kanade Dataset (CK+), and Real-world Affective Faces Database (RAF-DB) benchmark datasets. We also perform ablation evaluation to show the importance of different components of our architecture. To evaluate the efficiency of eXnet on embedded systems, we deploy it on Raspberry Pi 4B. All these evaluations show the superiority of eXnet for emotion recognition in the wild in terms of accuracy, the number of parameters, and size on disk.

scholarly journals Are affective factors related to individual differences in facial expression recognition?

2020 ◽  
Vol 7 (9) ◽  
pp. 190699
Author(s):  
Sarah A. H. Alharbi ◽  
Katherine Button ◽  
Lingshan Zhang ◽  
Kieran J. O'Shea ◽  
Vanessa Fasolt ◽  
...  
Keyword(s):  
Individual Differences ◽  
Facial Expression ◽  
Emotion Recognition ◽  
Facial Expression Recognition ◽  
Recognition Performance ◽  
Expression Recognition ◽  
General Anxiety ◽  
Affective Factors ◽  
Multimodal Emotion Recognition ◽  
Anxiety Subscale

Evidence that affective factors (e.g. anxiety, depression, affect) are significantly related to individual differences in emotion recognition is mixed. Palermo et al . (Palermo et al . 2018 J. Exp. Psychol. Hum. Percept. Perform. 44 , 503–517) reported that individuals who scored lower in anxiety performed significantly better on two measures of facial-expression recognition (emotion-matching and emotion-labelling tasks), but not a third measure (the multimodal emotion recognition test). By contrast, facial-expression recognition was not significantly correlated with measures of depression, positive or negative affect, empathy, or autistic-like traits. Because the range of affective factors considered in this study and its use of multiple expression-recognition tasks mean that it is a relatively comprehensive investigation of the role of affective factors in facial expression recognition, we carried out a direct replication. In common with Palermo et al . (Palermo et al . 2018 J. Exp. Psychol. Hum. Percept. Perform. 44 , 503–517), scores on the DASS anxiety subscale negatively predicted performance on the emotion recognition tasks across multiple analyses, although these correlations were only consistently significant for performance on the emotion-labelling task. However, and by contrast with Palermo et al . (Palermo et al . 2018 J. Exp. Psychol. Hum. Percept. Perform. 44 , 503–517), other affective factors (e.g. those related to empathy) often also significantly predicted emotion-recognition performance. Collectively, these results support the proposal that affective factors predict individual differences in emotion recognition, but that these correlations are not necessarily specific to measures of general anxiety, such as the DASS anxiety subscale.

Real-time facial expression recognition using smoothed deep neural network ensemble

2020 ◽  
Vol 28 (1) ◽  
pp. 97-111
Author(s):  
Nadir Kamel Benamara ◽  
Mikel Val-Calvo ◽  
Jose Ramón Álvarez-Sánchez ◽  
Alejandro Díaz-Morcillo ◽  
Jose Manuel Ferrández-Vicente ◽  
...  
Keyword(s):  
Neural Network ◽  
Facial Expression ◽  
Emotion Recognition ◽  
Facial Expression Recognition ◽  
Facial Emotion Recognition ◽  
Training Data ◽  
Facial Emotion ◽  
Processing Unit ◽  
Expression Recognition ◽  
Neural Network Ensemble

Facial emotion recognition (FER) has been extensively researched over the past two decades due to its direct impact in the computer vision and affective robotics fields. However, the available datasets to train these models include often miss-labelled data due to the labellers bias that drives the model to learn incorrect features. In this paper, a facial emotion recognition system is proposed, addressing automatic face detection and facial expression recognition separately, the latter is performed by a set of only four deep convolutional neural network respect to an ensembling approach, while a label smoothing technique is applied to deal with the miss-labelled training data. The proposed system takes only 13.48 ms using a dedicated graphics processing unit (GPU) and 141.97 ms using a CPU to recognize facial emotions and reaches the current state-of-the-art performances regarding the challenging databases, FER2013, SFEW 2.0, and ExpW, giving recognition accuracies of 72.72%, 51.97%, and 71.82% respectively.

Sign in / Sign up

Export Citation Format

Share Document