scholarly journals Towards Affective TV with Facial Expression Recognition

2021 ◽  
Author(s):  
Pedro A. Valentim ◽  
Fábio Barreto ◽  
Débora C. Muchaluat-Saade
Keyword(s):  
Facial Expression ◽  
Latin American ◽  
Facial Expression Recognition ◽  
Affective Computing ◽  
Interactive Television ◽  
Digital Tv ◽  
Point Of View ◽  
Expression Recognition ◽  
Wide Range ◽  
Latin American Countries

Facial recognition techniques, fantasized in fiction movie classics, have already become reality. Such technology opens up a wide range of possibilities for different kinds of systems. From the point of view of interactive applications, facial expression as input data may be more immediate and more trustworthy to the user’s sentiment than the click of a button. For interactive television, facial expression recognition could be used for bringing broadcasters and viewers closer, enabling TV content to be personalized by the user sentiment. In fact, not only facial expression recognition, but any interaction that enables affective computing. In this work, we call this concept Affective TV. In order to support it, this work proposes facial expression recognition for digital TV applications. Our proposal is implemented and evaluated in the Ginga-NCL middleware, a digital TV standard used in several Latin American countries.

scholarly journals Improving Human–Robot Interaction by Enhancing NAO Robot Awareness of Human Facial Expression

Sensors ◽  
2021 ◽  
Vol 21 (19) ◽  
pp. 6438
Author(s):  
Chiara Filippini ◽  
David Perpetuini ◽  
Daniela Cardone ◽  
Arcangelo Merla
Keyword(s):  
Facial Expression ◽  
Facial Expressions ◽  
Facial Expression Recognition ◽  
Affective Computing ◽  
Human Robot Interaction ◽  
Human Interaction ◽  
Expression Recognition ◽  
Robot Interaction ◽  
Interaction Field ◽  
Nao Robot

An intriguing challenge in the human–robot interaction field is the prospect of endowing robots with emotional intelligence to make the interaction more genuine, intuitive, and natural. A crucial aspect in achieving this goal is the robot’s capability to infer and interpret human emotions. Thanks to its design and open programming platform, the NAO humanoid robot is one of the most widely used agents for human interaction. As with person-to-person communication, facial expressions are the privileged channel for recognizing the interlocutor’s emotional expressions. Although NAO is equipped with a facial expression recognition module, specific use cases may require additional features and affective computing capabilities that are not currently available. This study proposes a highly accurate convolutional-neural-network-based facial expression recognition model that is able to further enhance the NAO robot’ awareness of human facial expressions and provide the robot with an interlocutor’s arousal level detection capability. Indeed, the model tested during human–robot interactions was 91% and 90% accurate in recognizing happy and sad facial expressions, respectively; 75% accurate in recognizing surprised and scared expressions; and less accurate in recognizing neutral and angry expressions. Finally, the model was successfully integrated into the NAO SDK, thus allowing for high-performing facial expression classification with an inference time of 0.34 ± 0.04 s.

scholarly journals Multilabel convolution neural network for facial expression recognition and ordinal intensity estimation

2021 ◽  
Vol 7 ◽  
pp. e736
Author(s):  
Olufisayo Ekundayo ◽  
Serestina Viriri
Keyword(s):  
Neural Network ◽  
Facial Expression ◽  
Facial Expression Recognition ◽  
Affective Computing ◽  
Convolution Neural Network ◽  
Superior Performance ◽  
Expression Recognition ◽  
Intensity Estimation ◽  
Expression Intensity ◽  
Intensity Of Emotion

Facial Expression Recognition (FER) has gained considerable attention in affective computing due to its vast area of applications. Diverse approaches and methods have been considered for a robust FER in the field, but only a few works considered the intensity of emotion embedded in the expression. Even the available studies on expression intensity estimation successfully assigned a nominal/regression value or classified emotion in a range of intervals. Most of the available works on facial expression intensity estimation successfully present only the emotion intensity estimation. At the same time, others proposed methods that predict emotion and its intensity in different channels. These multiclass approaches and extensions do not conform to man heuristic manner of recognising emotion and its intensity estimation. This work presents a Multilabel Convolution Neural Network (ML-CNN)-based model, which could simultaneously recognise emotion and provide ordinal metrics as the intensity estimation of the emotion. The proposed ML-CNN is enhanced with the aggregation of Binary Cross-Entropy (BCE) loss and Island Loss (IL) functions to minimise intraclass and interclass variations. Also, ML-CNN model is pre-trained with Visual Geometric Group (VGG-16) to control overfitting. In the experiments conducted on Binghampton University 3D Facial Expression (BU-3DFE) and Cohn Kanade extension (CK+) datasets, we evaluate ML-CNN’s performance based on accuracy and loss. We also carried out a comparative study of our model with some popularly used multilabel algorithms using standard multilabel metrics. ML-CNN model simultaneously predicts emotion and intensity estimation using ordinal metrics. The model also shows appreciable and superior performance over four standard multilabel algorithms: Chain Classifier (CC), distinct Random K label set (RAKEL), Multilabel K Nearest Neighbour (MLKNN) and Multilabel ARAM (MLARAM).

scholarly journals Deep Joint Spatiotemporal Network (DJSTN) for Efficient Facial Expression Recognition

Sensors ◽  
2020 ◽  
Vol 20 (7) ◽  
pp. 1936 ◽  
Author(s):  
Dami Jeong ◽  
Byung-Gyu Kim ◽  
Suh-Yeon Dong
Keyword(s):  
Facial Expression ◽  
Facial Expression Recognition ◽  
Affective Computing ◽  
Recognition Accuracy ◽  
Three Dimensional ◽  
Expression Recognition ◽  
Facial Muscle ◽  
Important Process ◽  
Emotional Status ◽  
Temporal Features

Understanding a person’s feelings is a very important process for the affective computing. People express their emotions in various ways. Among them, facial expression is the most effective way to present human emotional status. We propose efficient deep joint spatiotemporal features for facial expression recognition based on the deep appearance and geometric neural networks. We apply three-dimensional (3D) convolution to extract spatial and temporal features at the same time. For the geometric network, 23 dominant facial landmarks are selected to express the movement of facial muscle through the analysis of energy distribution of whole facial landmarks.We combine these features by the designed joint fusion classifier to complement each other. From the experimental results, we verify the recognition accuracy of 99.21%, 87.88%, and 91.83% for CK+, MMI, and FERA datasets, respectively. Through the comparative analysis, we show that the proposed scheme is able to improve the recognition accuracy by 4% at least.

scholarly journals Region based Facial Expression Recognition using Gradient Directions

2020 ◽  
Vol 9 (4) ◽  
pp. 2771-2776
Keyword(s):  
Facial Expression ◽  
Template Matching ◽  
Facial Expression Recognition ◽  
Affective Computing ◽  
Recognition System ◽  
Research Field ◽  
Expression Recognition ◽  
Emotion Classification ◽  
Classification Result ◽  
Basic Emotions

Facial Expression Recognition (FER) has gained significant importance in the research field of Affective Computing in different extents. As a part of the different dimensional thinking, aiming at improving the accuracy of the recognition system and reducing the computational load, region based FER is proposed in this paper. The system is an emotion identifying system among the basic emotions, through subject independent template matching based on gradient directions. The model designed is tested on the Enhanced Cohn-Kanade (CK+) dataset. Another important contribution of the work is using only eye (including eyebrows and the nose portion near eyes) and mouth regions in the emotion recognition. The emotion classification result is 94.3% (CK+ dataset) for 6-class FER.

Sign in / Sign up

Export Citation Format

Share Document