A Descriptive Survey on Face Emotion Recognition Techniques

Recognition of natural emotion from human faces has applications in Human–Computer Interaction, image and video retrieval, automated tutoring systems, smart environment as well as driver warning systems. It is also a significant indication of nonverbal communication among the individuals. The assignment of Face Emotion Recognition (FER) is predominantly complex for two reasons. The first reason is the nonexistence of a large database of training images, and the second one is about classifying the emotions, which can be complex based on the static input image. In addition, robust unbiased FER in real time remains the foremost challenge for various supervised learning-based techniques. This survey analyzes diverse techniques regarding the FER systems. It reviews a bunch of research papers and performs a significant analysis. Initially, the analysis depicts various techniques that are contributed in different research papers. In addition, this paper offers a comprehensive study regarding the chronological review and performance achievements in each contribution. The analytical review is also concerned about the measures for which the maximum performance was achieved in several contributions. Finally, the survey is extended with various research issues and gaps that can be useful for the researchers to promote improved future works on the FER models.

Automatic Face Emotion Recognition With the Aid of Probability-Based Bird Swarm-Trained Neural Network

This paper intends to develop a novel FER model, which consists of four stages: (1) face detection, (2) feature extraction, (3) dimension reduction, and (4) classification. In this context, the face detection is done using Viola Jones method (VJ). It is the first object recognition model to offer better recognition rates in real-time. Further, features extraction techniques like local binary pattern (LBP) and discrete wavelet transform (DWT) are used for extracting the features from face detected images. Moreover, the dimension reduction of features is done using principal component analysis (PCA), which is an arithmetical process that exploits an orthogonal transformation to exchange a group of annotations of probably interrelated constraints. The classification procedure is performed using neural network (NN), with the new training algorithm called bird swarm algorithm, which is modified based on probability and hence termed as probability-based BSA (P-BSA).

Music Recommendation Based on Face Emotion Recognition

We propose a new approach for playing music automatically using facial emotion. Most of the existing approaches involve playing music manually, using wearable computing devices, or classifying based on audio features. Instead, we propose to change the manual sorting and playing. We have used a Convolutional Neural Network for emotion detection. For music recommendations, Pygame & Tkinter are used. Our proposed system tends to reduce the computational time involved in obtaining the results and the overall cost of the designed system, thereby increasing the system’s overall accuracy. Testing of the system is done on the FER2013 dataset. Facial expressions are captured using an inbuilt camera. Feature extraction is performed on input face images to detect emotions such as happy, angry, sad, surprise, and neutral. Automatically music playlist is generated by identifying the current emotion of the user. It yields better performance in terms of computational time, as compared to the algorithm in the existing literature.

Disease-specific contribution of pulvinar dysfunction to impaired emotion recognition in schizophrenia

One important aspect for managing social interactions is the ability to rapidly and accurately perceive and respond to facial expressions, which is highly dependent upon intact processing within both cortical and subcortical components of the early visual pathways. Social cognitive deficits, including face emotion recognition (FER) deficits, are characteristic of several neuropsychiatric disorders, including schizophrenia (Sz) and autism spectrum disorders (ASD). Here, we investigated potential visual sensory contributions to FER deficits in Sz (n=28) and adult ASD (n=20) participants compared to neurotypical (n=30) controls using task-based fMRI during an implicit static/dynamic FER task. Compared to neurotypical controls, both Sz and ASD participants had significantly lower FER scores which interrelated with diminished activation of the superior temporal sulcus (STS). In Sz, STS deficits were predicted by reduced activation of both early visual regions and the pulvinar nucleus of the thalamus, along with impaired cortico-pulvinar interaction. By contrast, ASD participants showed patterns of increased early visual cortical and pulvinar activation. Large effect-size structural and histological abnormalities of pulvinar have previously been documented in Sz. Moreover, we have recently demonstrated impaired pulvinar activation to simple visual stimuli in Sz. Here, we provide the first demonstration of a disease-specific contribution of impaired pulvinar activation to social cognitive impairment in Sz.

HOG-ESRs Face Emotion Recognition Algorithm Based on HOG Feature and ESRs Method

As we all know, there are many ways to express emotions. Among them, facial emotion recognition, which is widely used in human–computer interaction, psychoanalysis of mental patients, multimedia retrieval, and other fields, is still a challenging task. At present, although convolutional neural network has achieved great success in face emotion recognition algorithms, it has a rising space in effective feature extraction and recognition accuracy. According to a large number of literature studies, histogram of oriented gradient (HOG) can effectively extract face features, and ensemble methods can effectively improve the accuracy and robustness of the algorithm. Therefore, this paper proposes a new algorithm, HOG-ESRs, which improves the traditional ensemble methods to the ensembles with shared representations (ESRs) method, effectively reducing the residual generalization error, and then combining HOG features with ESRs. The experimental results on the FER2013 dataset show that the new algorithm can not only effectively extract features and reduce the residual generalization error, but also improve the accuracy and robustness of the algorithm, the purpose of the study being achieved. The application of HOG-ESRs in facial emotion recognition is helpful to solve the symmetry of edge detection and the deficiency of related methods in an outdoor lighting environment.

Positive Shifts in Emotion Evaluation Following Mindfulness-Based Cognitive Therapy (MBCT) in Remitted Depressed Participants

Objectives A combination of negatively biased information processing and a reduced ability to experience positive emotions can persist into remission from major depression (rMDD). Studies have shown that mindfulness-based cognitive therapy (MBCT) can increase self-reported positive emotions in rMDD participants; similar changes using neuropsychological tasks have not been shown. In this study, we investigated neuropsychological change in emotional processing following MBCT in rMDD participants. Methods Seventy-three rMDD participants, 40 of whom received MBCT and 33 of whom continued with treatment as usual (TAU), and 42 never depressed participants took part; neither the TAU nor never depressed participants received MBCT. All were assessed at baseline and immediately following MBCT or after an 8-week gap for those without active intervention. Participants completed emotion evaluation and face emotion recognition tasks with self-report measures (mood, mindfulness) at each session. Results Results showed an MBCT-specific shift in ratings from less negative to more positive emotion evaluations, which correlated with mindfulness practice and self-report mindfulness change. Both the MBCT and TAU groups showed a small increase in overall face emotion recognition accuracy compared with no change in never depressed participants. Conclusions These findings support a specific role for MBCT in encouraging more positive evaluations of life situations in those with previous depression rather than influencing lower-level processing of emotions. Results should be interpreted cautiously given that this was a non-randomised, preference choice trial. Trial Registration NCT02226042