scholarly journals Emotion Monitoring for Preschool Children Based on Face Recognition and Emotion Recognition Algorithms

Complexity ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Guiping Yu
Keyword(s):  
Face Recognition ◽  
Preschool Children ◽  
Facial Expression ◽  
Emotion Recognition ◽  
Experimental Results ◽  
Geometric Feature ◽  
Feature Descriptor ◽  
Expression Recognition ◽  
Facial Contour ◽  
Recognition Algorithms

In this paper, we study the face recognition and emotion recognition algorithms to monitor the emotions of preschool children. For previous emotion recognition focusing on faces, we propose to obtain more comprehensive information from faces, gestures, and contexts. Using the deep learning approach, we design a more lightweight network structure to reduce the number of parameters and save computational resources. There are not only innovations in applications, but also algorithmic enhancements. And face annotation is performed on the dataset, while a hierarchical sampling method is designed to alleviate the data imbalance phenomenon that exists in the dataset. A new feature descriptor, called “oriented gradient histogram from three orthogonal planes,” is proposed to characterize facial appearance variations. A new efficient geometric feature is also proposed to capture facial contour variations, and the role of audio methods in emotion recognition is explored. Multifeature fusion can be used to optimally combine different features. The experimental results show that the method is very effective compared to other recent methods in dealing with facial expression recognition problems about videos in both laboratory-controlled environments and outdoor environments. The method performed experiments on expression detection in a facial expression database. The experimental results are compared with data from previous studies and demonstrate the effectiveness of the proposed new method.

scholarly journals On-Road Driver Emotion Recognition Using Facial Expression

2022 ◽  
Vol 12 (2) ◽  
pp. 807
Author(s):  
Huafei Xiao ◽  
Wenbo Li ◽  
Guanzhong Zeng ◽  
Yingzhang Wu ◽  
Jiyong Xue ◽  
...  
Keyword(s):  
Facial Expression ◽  
Emotion Recognition ◽  
Data Augmentation ◽  
Recognition Task ◽  
Superior Performance ◽  
Expression Recognition ◽  
Emotion Detection ◽  
Backbone Networks ◽  
Driving Task ◽  
Machine Systems

With the development of intelligent automotive human-machine systems, driver emotion detection and recognition has become an emerging research topic. Facial expression-based emotion recognition approaches have achieved outstanding results on laboratory-controlled data. However, these studies cannot represent the environment of real driving situations. In order to address this, this paper proposes a facial expression-based on-road driver emotion recognition network called FERDERnet. This method divides the on-road driver facial expression recognition task into three modules: a face detection module that detects the driver’s face, an augmentation-based resampling module that performs data augmentation and resampling, and an emotion recognition module that adopts a deep convolutional neural network pre-trained on FER and CK+ datasets and then fine-tuned as a backbone for driver emotion recognition. This method adopts five different backbone networks as well as an ensemble method. Furthermore, to evaluate the proposed method, this paper collected an on-road driver facial expression dataset, which contains various road scenarios and the corresponding driver’s facial expression during the driving task. Experiments were performed on the on-road driver facial expression dataset that this paper collected. Based on efficiency and accuracy, the proposed FERDERnet with Xception backbone was effective in identifying on-road driver facial expressions and obtained superior performance compared to the baseline networks and some state-of-the-art networks.

scholarly journals eXnet: An Efficient Approach for Emotion Recognition in the Wild

Sensors ◽  
2020 ◽  
Vol 20 (4) ◽  
pp. 1087
Author(s):  
Muhammad Naveed Riaz ◽  
Yao Shen ◽  
Muhammad Sohail ◽  
Minyi Guo
Keyword(s):  
Facial Expression ◽  
Emotion Recognition ◽  
Facial Expression Recognition ◽  
Data Augmentation ◽  
Raspberry Pi ◽  
Expression Recognition ◽  
Extensive Evaluation ◽  
Benchmark Datasets ◽  
Augmentation Techniques ◽  
In The Wild

Facial expression recognition has been well studied for its great importance in the areas of human–computer interaction and social sciences. With the evolution of deep learning, there have been significant advances in this area that also surpass human-level accuracy. Although these methods have achieved good accuracy, they are still suffering from two constraints (high computational power and memory), which are incredibly critical for small hardware-constrained devices. To alleviate this issue, we propose a new Convolutional Neural Network (CNN) architecture eXnet (Expression Net) based on parallel feature extraction which surpasses current methods in accuracy and contains a much smaller number of parameters (eXnet: 4.57 million, VGG19: 14.72 million), making it more efficient and lightweight for real-time systems. Several modern data augmentation techniques are applied for generalization of eXnet; these techniques improve the accuracy of the network by overcoming the problem of overfitting while containing the same size. We provide an extensive evaluation of our network against key methods on Facial Expression Recognition 2013 (FER-2013), Extended Cohn-Kanade Dataset (CK+), and Real-world Affective Faces Database (RAF-DB) benchmark datasets. We also perform ablation evaluation to show the importance of different components of our architecture. To evaluate the efficiency of eXnet on embedded systems, we deploy it on Raspberry Pi 4B. All these evaluations show the superiority of eXnet for emotion recognition in the wild in terms of accuracy, the number of parameters, and size on disk.

scholarly journals Are affective factors related to individual differences in facial expression recognition?

2020 ◽  
Vol 7 (9) ◽  
pp. 190699
Author(s):  
Sarah A. H. Alharbi ◽  
Katherine Button ◽  
Lingshan Zhang ◽  
Kieran J. O'Shea ◽  
Vanessa Fasolt ◽  
...  
Keyword(s):  
Individual Differences ◽  
Facial Expression ◽  
Emotion Recognition ◽  
Facial Expression Recognition ◽  
Recognition Performance ◽  
Expression Recognition ◽  
General Anxiety ◽  
Affective Factors ◽  
Multimodal Emotion Recognition ◽  
Anxiety Subscale

Evidence that affective factors (e.g. anxiety, depression, affect) are significantly related to individual differences in emotion recognition is mixed. Palermo et al . (Palermo et al . 2018 J. Exp. Psychol. Hum. Percept. Perform. 44 , 503–517) reported that individuals who scored lower in anxiety performed significantly better on two measures of facial-expression recognition (emotion-matching and emotion-labelling tasks), but not a third measure (the multimodal emotion recognition test). By contrast, facial-expression recognition was not significantly correlated with measures of depression, positive or negative affect, empathy, or autistic-like traits. Because the range of affective factors considered in this study and its use of multiple expression-recognition tasks mean that it is a relatively comprehensive investigation of the role of affective factors in facial expression recognition, we carried out a direct replication. In common with Palermo et al . (Palermo et al . 2018 J. Exp. Psychol. Hum. Percept. Perform. 44 , 503–517), scores on the DASS anxiety subscale negatively predicted performance on the emotion recognition tasks across multiple analyses, although these correlations were only consistently significant for performance on the emotion-labelling task. However, and by contrast with Palermo et al . (Palermo et al . 2018 J. Exp. Psychol. Hum. Percept. Perform. 44 , 503–517), other affective factors (e.g. those related to empathy) often also significantly predicted emotion-recognition performance. Collectively, these results support the proposal that affective factors predict individual differences in emotion recognition, but that these correlations are not necessarily specific to measures of general anxiety, such as the DASS anxiety subscale.

Real-time facial expression recognition using smoothed deep neural network ensemble

2020 ◽  
Vol 28 (1) ◽  
pp. 97-111
Author(s):  
Nadir Kamel Benamara ◽  
Mikel Val-Calvo ◽  
Jose Ramón Álvarez-Sánchez ◽  
Alejandro Díaz-Morcillo ◽  
Jose Manuel Ferrández-Vicente ◽  
...  
Keyword(s):  
Neural Network ◽  
Facial Expression ◽  
Emotion Recognition ◽  
Facial Expression Recognition ◽  
Facial Emotion Recognition ◽  
Training Data ◽  
Facial Emotion ◽  
Processing Unit ◽  
Expression Recognition ◽  
Neural Network Ensemble

Facial emotion recognition (FER) has been extensively researched over the past two decades due to its direct impact in the computer vision and affective robotics fields. However, the available datasets to train these models include often miss-labelled data due to the labellers bias that drives the model to learn incorrect features. In this paper, a facial emotion recognition system is proposed, addressing automatic face detection and facial expression recognition separately, the latter is performed by a set of only four deep convolutional neural network respect to an ensembling approach, while a label smoothing technique is applied to deal with the miss-labelled training data. The proposed system takes only 13.48 ms using a dedicated graphics processing unit (GPU) and 141.97 ms using a CPU to recognize facial emotions and reaches the current state-of-the-art performances regarding the challenging databases, FER2013, SFEW 2.0, and ExpW, giving recognition accuracies of 72.72%, 51.97%, and 71.82% respectively.

scholarly journals Facial Expression Recognition Using Computer Vision: A Systematic Review

2019 ◽  
Vol 9 (21) ◽  
pp. 4678 ◽  
Author(s):  
Daniel Canedo ◽  
António J. R. Neves
Keyword(s):  
Systematic Review ◽  
Computer Vision ◽  
Facial Expression ◽  
Emotion Recognition ◽  
Facial Expression Recognition ◽  
Principal Component ◽  
Local Binary Patterns ◽  
Future Research ◽  
Expression Recognition ◽  
Contemporary World

Emotion recognition has attracted major attention in numerous fields because of its relevant applications in the contemporary world: marketing, psychology, surveillance, and entertainment are some examples. It is possible to recognize an emotion through several ways; however, this paper focuses on facial expressions, presenting a systematic review on the matter. In addition, 112 papers published in ACM, IEEE, BASE and Springer between January 2006 and April 2019 regarding this topic were extensively reviewed. Their most used methods and algorithms will be firstly introduced and summarized for a better understanding, such as face detection, smoothing, Principal Component Analysis (PCA), Local Binary Patterns (LBP), Optical Flow (OF), Gabor filters, among others. This review identified a clear difficulty in translating the high facial expression recognition (FER) accuracy in controlled environments to uncontrolled and pose-variant environments. The future efforts in the FER field should be put into multimodal systems that are robust enough to face the adversities of real world scenarios. A thorough analysis on the research done on FER in Computer Vision based on the selected papers is presented. This review aims to not only become a reference for future research on emotion recognition, but also to provide an overview of the work done in this topic for potential readers.

scholarly journals Automated Facial Expression Recognition Using Gradient-Based Ternary Texture Patterns

2013 ◽  
Vol 2013 ◽  
pp. 1-8 ◽  
Author(s):  
Faisal Ahmed ◽  
Emam Hossain
Keyword(s):  
Facial Expression ◽  
Recognition Performance ◽  
Texture Feature ◽  
Recognition Task ◽  
Facial Feature ◽  
Recognition System ◽  
Feature Descriptor ◽  
Expression Recognition ◽  
Local Texture ◽  
Face Expression

Recognition of human expression from facial image is an interesting research area, which has received increasing attention in the recent years. A robust and effective facial feature descriptor is the key to designing a successful expression recognition system. Although much progress has been made, deriving a face feature descriptor that can perform consistently under changing environment is still a difficult and challenging task. In this paper, we present the gradient local ternary pattern (GLTP)—a discriminative local texture feature for representing facial expression. The proposed GLTP operator encodes the local texture of an image by computing the gradient magnitudes of the local neighborhood and quantizing those values in three discrimination levels. The location and occurrence information of the resulting micropatterns is then used as the face feature descriptor. The performance of the proposed method has been evaluated for the person-independent face expression recognition task. Experiments with prototypic expression images from the Cohn-Kanade (CK) face expression database validate that the GLTP feature descriptor can effectively encode the facial texture and thus achieves improved recognition performance than some well-known appearance-based facial features.

scholarly journals Toward Continuous Social Phenotyping: Analyzing Gaze Patterns in an Emotion Recognition Task for Children With Autism Through Wearable Smart Glasses

10.2196/13810 ◽  
2020 ◽  
Vol 22 (4) ◽  
pp. e13810 ◽  
Author(s):  
Anish Nag ◽  
Nick Haber ◽  
Catalin Voss ◽  
Serena Tamura ◽  
Jena Daniels ◽  
...  
Keyword(s):  
Facial Expression ◽  
Emotion Recognition ◽  
Facial Expression Recognition ◽  
Eye Gaze ◽  
Recognition Task ◽  
Children With Autism ◽  
Expression Recognition ◽  
Eye Tracker ◽  
Gaze Tracking ◽  
Smart Glasses

Background Several studies have shown that facial attention differs in children with autism. Measuring eye gaze and emotion recognition in children with autism is challenging, as standard clinical assessments must be delivered in clinical settings by a trained clinician. Wearable technologies may be able to bring eye gaze and emotion recognition into natural social interactions and settings. Objective This study aimed to test: (1) the feasibility of tracking gaze using wearable smart glasses during a facial expression recognition task and (2) the ability of these gaze-tracking data, together with facial expression recognition responses, to distinguish children with autism from neurotypical controls (NCs). Methods We compared the eye gaze and emotion recognition patterns of 16 children with autism spectrum disorder (ASD) and 17 children without ASD via wearable smart glasses fitted with a custom eye tracker. Children identified static facial expressions of images presented on a computer screen along with nonsocial distractors while wearing Google Glass and the eye tracker. Faces were presented in three trials, during one of which children received feedback in the form of the correct classification. We employed hybrid human-labeling and computer vision–enabled methods for pupil tracking and world–gaze translation calibration. We analyzed the impact of gaze and emotion recognition features in a prediction task aiming to distinguish children with ASD from NC participants. Results Gaze and emotion recognition patterns enabled the training of a classifier that distinguished ASD and NC groups. However, it was unable to significantly outperform other classifiers that used only age and gender features, suggesting that further work is necessary to disentangle these effects. Conclusions Although wearable smart glasses show promise in identifying subtle differences in gaze tracking and emotion recognition patterns in children with and without ASD, the present form factor and data do not allow for these differences to be reliably exploited by machine learning systems. Resolving these challenges will be an important step toward continuous tracking of the ASD phenotype.

Facial Emotion Recognition Using an Ensemble of Multi-Level Convolutional Neural Networks

2019 ◽  
Vol 33 (11) ◽  
pp. 1940015 ◽  
Author(s):  
Hai-Duong Nguyen ◽  
Soonja Yeom ◽  
Guee-Sang Lee ◽  
Hyung-Jeong Yang ◽  
In-Seop Na ◽  
...  
Keyword(s):  
Facial Expression ◽  
Emotion Recognition ◽  
Facial Expression Recognition ◽  
Research Area ◽  
Expression Recognition ◽  
Human Machine Interaction ◽  
Network Connections ◽  
Multi Level ◽  
Machine Interaction ◽  
Made In

Emotion recognition plays an indispensable role in human–machine interaction system. The process includes finding interesting facial regions in images and classifying them into one of seven classes: angry, disgust, fear, happy, neutral, sad, and surprise. Although many breakthroughs have been made in image classification, especially in facial expression recognition, this research area is still challenging in terms of wild sampling environment. In this paper, we used multi-level features in a convolutional neural network for facial expression recognition. Based on our observations, we introduced various network connections to improve the classification task. By combining the proposed network connections, our method achieved competitive results compared to state-of-the-art methods on the FER2013 dataset.

Facial Expression Recognition Method Based on Improved VGG Convolutional Neural Network

2019 ◽  
Vol 34 (07) ◽  
pp. 2056003
Author(s):  
Shuo Cheng ◽  
Guohui Zhou
Keyword(s):  
Neural Network ◽  
Facial Expression ◽  
Convolutional Neural Network ◽  
Network Structure ◽  
Facial Expression Recognition ◽  
Network Models ◽  
Experimental Results ◽  
Expression Recognition ◽  
Classification Problems ◽  
Nonlinear Network

Because the shallow neural network has limited ability to represent complex functions with limited samples and calculation units, its generalization ability will be limited when it comes to complex classification problems. The essence of deep learning is to learn a nonlinear network structure, to represent input data distributed representation and demonstrate a powerful ability to learn deeper features of data from a small set of samples. In order to realize the accurate classification of expression images under normal conditions, this paper proposes an expression recognition model of improved Visual Geometry Group (VGG) deep convolutional neural network (CNN). Based on the VGG-19, the model optimizes network structure and network parameters. Most expression databases are unable to train the entire network from the start due to lack of sufficient data. This paper uses migration learning techniques to overcome the shortage of image training samples. Shallow CNN, Alex-Net and improved VGG-19 deep CNN are used to train and analyze the facial expression data on the Extended Cohn–Kanade expression database, and compare the experimental results obtained. The experimental results indicate that the improved VGG-19 network model can achieve 96% accuracy in facial expression recognition, which is obviously superior to the results of other network models.

Sign in / Sign up

Export Citation Format

Share Document