A REVIEW ON FACIAL EMOTION RECOGNITION THAT USES MACHINE LEARNING ALGORITHMS

For a computer, identification of human emotion from a still image of the human face is a complex, challenging, and heavily calculative task. Classification of human emotion is done by using a different combination of convolutional neural networks (CNN) that task is known as Facial Emotion Recognition (FER). CNN model is achieved by training and testing on lots of same categorical images from the dataset using different hyperparameter tuning. The main contribution of this work is to look for various CNN architectures, hyperparameter tuning and compare the performance of those CNN models based on accuracy and loss while training and testing on Facial Emotion Recognition. This study shall help to provide a guide for the selection of an appropriate CNN model and tuning parameter according to the needs of the applicant.

Download Full-text

Facial Emotion Recognition Using Hybrid Features

Informatics ◽

10.3390/informatics7010006 ◽

2020 ◽

Vol 7 (1) ◽

pp. 6 ◽

Cited By ~ 1

Author(s):

Abdulrahman Alreshidi ◽

Mohib Ullah

Keyword(s):

Emotion Recognition ◽

Facial Expressions ◽

Skin Color ◽

Modular Design ◽

Machine Learning Algorithms ◽

Facial Emotion Recognition ◽

Facial Emotion ◽

Hybrid Features ◽

Cascade Classifiers ◽

Reference Methods

Facial emotion recognition is a crucial task for human-computer interaction, autonomous vehicles, and a multitude of multimedia applications. In this paper, we propose a modular framework for human facial emotions’ recognition. The framework consists of two machine learning algorithms (for detection and classification) that could be trained offline for real-time applications. Initially, we detect faces in the images by exploring the AdaBoost cascade classifiers. We then extract neighborhood difference features (NDF), which represent the features of a face based on localized appearance information. The NDF models different patterns based on the relationships between neighboring regions themselves instead of considering only intensity information. The study is focused on the seven most important facial expressions that are extensively used in day-to-day life. However, due to the modular design of the framework, it can be extended to classify N number of facial expressions. For facial expression classification, we train a random forest classifier with a latent emotional state that takes care of the mis-/false detection. Additionally, the proposed method is independent of gender and facial skin color for emotion recognition. Moreover, due to the intrinsic design of NDF, the proposed method is illumination and orientation invariant. We evaluate our method on different benchmark datasets and compare it with five reference methods. In terms of accuracy, the proposed method gives 13% and 24% better results than the reference methods on the static facial expressions in the wild (SFEW) and real-world affective faces (RAF) datasets, respectively.

Download Full-text

Two-Stage Recognition and beyond for Compound Facial Emotion Recognition

Electronics ◽

10.3390/electronics10222847 ◽

2021 ◽

Vol 10 (22) ◽

pp. 2847

Author(s):

Dorota Kamińska ◽

Kadir Aktas ◽

Davit Rizhinashvili ◽

Danila Kuklyanov ◽

Abdallah Hussein Sham ◽

...

Keyword(s):

Emotion Recognition ◽

Complex Problem ◽

Facial Emotion Recognition ◽

Facial Emotion ◽

Facial Features ◽

Recognition Method ◽

Two Stage ◽

Facial Images ◽

Emotion Labels

Facial emotion recognition is an inherently complex problem due to individual diversity in facial features and racial and cultural differences. Moreover, facial expressions typically reflect the mixture of people’s emotional statuses, which can be expressed using compound emotions. Compound facial emotion recognition makes the problem even more difficult because the discrimination between dominant and complementary emotions is usually weak. We have created a database that includes 31,250 facial images with different emotions of 115 subjects whose gender distribution is almost uniform to address compound emotion recognition. In addition, we have organized a competition based on the proposed dataset, held at FG workshop 2020. This paper analyzes the winner’s approach—a two-stage recognition method (1st stage, coarse recognition; 2nd stage, fine recognition), which enhances the classification of symmetrical emotion labels.

Download Full-text

Recognition of Emotion Intensities Using Machine Learning Algorithms: A Comparative Study

Sensors ◽

10.3390/s19081897 ◽

2019 ◽

Vol 19 (8) ◽

pp. 1897 ◽

Cited By ~ 5

Author(s):

Dhwani Mehta ◽

Mohammad Faridul Haque Siddiqui ◽

Ahmad Y. Javaid

Keyword(s):

Comparative Study ◽

Emotion Recognition ◽

Nearest Neighbor ◽

Machine Learning Algorithms ◽

Facial Emotion Recognition ◽

Facial Emotion ◽

Support Vector ◽

Human Machine Interaction ◽

Intensity Estimation ◽

Intensity Of Emotion

Over the past two decades, automatic facial emotion recognition has received enormous attention. This is due to the increase in the need for behavioral biometric systems and human–machine interaction where the facial emotion recognition and the intensity of emotion play vital roles. The existing works usually do not encode the intensity of the observed facial emotion and even less involve modeling the multi-class facial behavior data jointly. Our work involves recognizing the emotion along with the respective intensities of those emotions. The algorithms used in this comparative study are Gabor filters, a Histogram of Oriented Gradients (HOG), and Local Binary Pattern (LBP) for feature extraction. For classification, we have used Support Vector Machine (SVM), Random Forest (RF), and Nearest Neighbor Algorithm (kNN). This attains emotion recognition and intensity estimation of each recognized emotion. This is a comparative study of classifiers used for facial emotion recognition along with the intensity estimation of those emotions for databases. The results verified that the comparative study could be further used in real-time behavioral facial emotion and intensity of emotion recognition.

Download Full-text

Recognition of Face Emotion using Convolutional Neural Network

International Journal of Innovative Technology and Exploring Engineering - Special Issue ◽

10.35940/ijitee.i7033.079920 ◽

2020 ◽

Vol 9 (9) ◽

pp. 364-376

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Emotion Recognition ◽

Real Time ◽

Research Work ◽

Recognition System ◽

Video Stream ◽

Machine Learning Algorithms ◽

Facial Emotion Recognition ◽

Facial Emotion

Recognition of face emotion has been a challenging task for many years. This work uses machine learning algorithms for both, a real-time image or a stored database image in the area of facial emotion recognition system. So it is very clear that, deep learning technology becomes important for Human-computer interaction (HCI) applications. The proposed system has two parts, real-time based facial emotion recognition system and also the image based facial emotion recognition system. A Convolutional Neural Network (CNN) model is used to train and test different facial emotion images in this research work. This work was executed successfully using Python 3.7.6 platform. The input Face image of a person was taken using the webcam video stream or from the standard database available for research. The five different facial emotions considered in this work are happy, surprise, angry, sad and neutral. The best recognition accuracy with the proposed system for the webcam video stream is found to be 91.2%, whereas for the input database images is found to be 90.08%.

Download Full-text

Classification of prefeature extracted images with deep convolutional neural network in facial emotion recognition of vehicle driver

10.1049/pbpc042e_ch10 ◽

2021 ◽

pp. 223-251

Author(s):

Ganesan Kaliyaperumal ◽

Manikandan N.S.

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Emotion Recognition ◽

Deep Convolutional Neural Network ◽

Facial Emotion Recognition ◽

Facial Emotion

Download Full-text

Skybiometry and AffectNet on Facial Emotion Recognition Using Supervised Machine Learning Algorithms

Proceedings of the 2018 International Conference on Control and Computer Vision - ICCCV '18 ◽

10.1145/3232651.3232665 ◽

2018 ◽

Cited By ~ 2

Author(s):

Mirafe R. Prospero ◽

Edson B. Lagamayo ◽

Anndee Christian L. Tumulak ◽

Arman Bernard G. Santos ◽

Bryan G. Dadiz

Keyword(s):

Machine Learning ◽

Emotion Recognition ◽

Learning Algorithms ◽

Machine Learning Algorithms ◽

Facial Emotion Recognition ◽

Supervised Machine Learning ◽

Facial Emotion

Download Full-text

Beeinträchtigtes Erkennen emotionaler Gesichtsausdrücke durch psychiatrische Patientinnen bei Alexithymie

Zeitschrift für Psychiatrie Psychologie und Psychotherapie ◽

10.1024/1661-4747/a000135 ◽

2013 ◽

Vol 61 (1) ◽

pp. 7-15 ◽

Cited By ~ 1

Author(s):

Daniel Dittrich ◽

Gregor Domes ◽

Susi Loebel ◽

Christoph Berger ◽

Carsten Spitzer ◽

...

Keyword(s):

Emotion Recognition ◽

Emotional Expression ◽

Depression Scale ◽

Facial Emotion Recognition ◽

Facial Emotion ◽

Check List ◽

Psychosomatische Erkrankungen ◽

Psychiatrische Patienten ◽

Symptom Check List ◽

Tas 20

Die vorliegende Studie untersucht die Hypothese eines mit Alexithymie assoziierten Defizits beim Erkennen emotionaler Gesichtsaudrücke an einer klinischen Population. Darüber hinaus werden Hypothesen zur Bedeutung spezifischer Emotionsqualitäten sowie zu Gender-Unterschieden getestet. 68 ambulante und stationäre psychiatrische Patienten (44 Frauen und 24 Männer) wurden mit der Toronto-Alexithymie-Skala (TAS-20), der Montgomery-Åsberg Depression Scale (MADRS), der Symptom-Check-List (SCL-90-R) und der Emotional Expression Multimorph Task (EEMT) untersucht. Als Stimuli des Gesichtererkennungsparadigmas dienten Gesichtsausdrücke von Basisemotionen nach Ekman und Friesen, die zu Sequenzen mit sich graduell steigernder Ausdrucksstärke angeordnet waren. Mittels multipler Regressionsanalyse untersuchten wir die Assoziation von TAS-20 Punktzahl und facial emotion recognition (FER). Während sich für die Gesamtstichprobe und den männlichen Stichprobenteil kein signifikanter Zusammenhang zwischen TAS-20-Punktzahl und FER zeigte, sahen wir im weiblichen Stichprobenteil durch die TAS-20 Punktzahl eine signifikante Prädiktion der Gesamtfehlerzahl (β = .38, t = 2.055, p < 0.05) und den Fehlern im Erkennen der Emotionen Wut und Ekel (Wut: β = .40, t = 2.240, p < 0.05, Ekel: β = .41, t = 2.214, p < 0.05). Für wütende Gesichter betrug die Varianzaufklärung durch die TAS-20-Punktzahl 13.3 %, für angeekelte Gesichter 19.7 %. Kein Zusammenhang bestand zwischen der Zeit, nach der die Probanden die emotionalen Sequenzen stoppten, um ihre Bewertung abzugeben (Antwortlatenz) und Alexithymie. Die Ergebnisse der Arbeit unterstützen das Vorliegen eines mit Alexithymie assoziierten Defizits im Erkennen emotionaler Gesichtsausdrücke bei weiblchen Probanden in einer heterogenen, klinischen Stichprobe. Dieses Defizit könnte die Schwierigkeiten Hochalexithymer im Bereich sozialer Interaktionen zumindest teilweise begründen und so eine Prädisposition für psychische sowie psychosomatische Erkrankungen erklären.

Download Full-text