scholarly journals CNN-Based Facial Expression Recognition from Annotated RGB-D Images for Human–Robot Interaction

2019 ◽  
Vol 16 (04) ◽  
pp. 1941002 ◽  
Author(s):  
Jing Li ◽  
Yang Mi ◽  
Gongfa Li ◽  
Zhaojie Ju
Keyword(s):  
Facial Expression ◽  
Facial Expression Recognition ◽  
Recognition Task ◽  
Recognition System ◽  
Human Robot Interaction ◽  
Microsoft Kinect ◽  
Depth Information ◽  
Expression Recognition ◽  
Stream Network ◽  
Depth Images

Facial expression recognition has been widely used in human computer interaction (HCI) systems. Over the years, researchers have proposed different feature descriptors, implemented different classification methods, and carried out a number of experiments on various datasets for automatic facial expression recognition. However, most of them used 2D static images or 2D video sequences for the recognition task. The main limitations of 2D-based analysis are problems associated with variations in pose and illumination, which reduce the recognition accuracy. Therefore, an alternative way is to incorporate depth information acquired by 3D sensor, because it is invariant in both pose and illumination. In this paper, we present a two-stream convolutional neural network (CNN)-based facial expression recognition system and test it on our own RGB-D facial expression dataset collected by Microsoft Kinect for XBOX in unspontaneous scenarios since Kinect is an inexpensive and portable device to capture both RGB and depth information. Our fully annotated dataset includes seven expressions (i.e., neutral, sadness, disgust, fear, happiness, anger, and surprise) for 15 subjects (9 males and 6 females) aged from 20 to 25. The two individual CNNs are identical in architecture but do not share parameters. To combine the detection results produced by these two CNNs, we propose the late fusion approach. The experimental results demonstrate that the proposed two-stream network using RGB-D images is superior to that of using only RGB images or depth images.

scholarly journals Real Time Facial Expression Recognition System Based on Deep Learning

2019 ◽  
Vol 8 (2S11) ◽  
pp. 4047-4051
Keyword(s):  
Deep Learning ◽  
Facial Expression ◽  
Real Time ◽  
Facial Expression Recognition ◽  
Recognition System ◽  
Expression Recognition ◽  
Or Education ◽  
Face Expression ◽  
Active Research ◽  
Main Components

The automatic detection of facial expressions is an active research topic, since its wide fields of applications in human-computer interaction, games, security or education. However, the latest studies have been made in controlled laboratory environments, which is not according to real world scenarios. For that reason, a real time Facial Expression Recognition System (FERS) is proposed in this paper, in which a deep learning approach is applied to enhance the detection of six basic emotions: happiness, sadness, anger, disgust, fear and surprise in a real-time video streaming. This system is composed of three main components: face detection, face preparation and face expression classification. The results of proposed FERS achieve a 65% of accuracy, trained over 35558 face images..

scholarly journals Facial Expression Recognition using SVM with CNN and Handcrafted Features

2019 ◽  
Vol 8 (4) ◽  
pp. 3570-3574
Keyword(s):  
Facial Expression ◽  
Facial Expression Recognition ◽  
Recognition System ◽  
Vital Role ◽  
Working Environment ◽  
Support Vector ◽  
Expression Recognition ◽  
Shopping Malls ◽  
Benchmark Datasets ◽  
Mind Set

The facial expression recognition system is playing vital role in many organizations, institutes, shopping malls to know about their stakeholders’ need and mind set. It comes under the broad category of computer vision. Facial expression can easily explain the true intention of a person without any kind of conversation. The main objective of this work is to improve the performance of facial expression recognition in the benchmark datasets like CK+, JAFFE. In order to achieve the needed accuracy metrics, the convolution neural network was constructed to extract the facial expression features automatically and combined with the handcrafted features extracted using Histogram of Gradients (HoG) and Local Binary Pattern (LBP) methods. Linear Support Vector Machine (SVM) is built to predict the emotions using the combined features. The proposed method produces promising results as compared to the recent work in [1].This is mainly needed in the working environment, shopping malls and other public places to effectively understand the likeliness of the stakeholders at that moment.

scholarly journals Improving Human–Robot Interaction by Enhancing NAO Robot Awareness of Human Facial Expression

Sensors ◽  
2021 ◽  
Vol 21 (19) ◽  
pp. 6438
Author(s):  
Chiara Filippini ◽  
David Perpetuini ◽  
Daniela Cardone ◽  
Arcangelo Merla
Keyword(s):  
Facial Expression ◽  
Facial Expressions ◽  
Facial Expression Recognition ◽  
Affective Computing ◽  
Human Robot Interaction ◽  
Human Interaction ◽  
Expression Recognition ◽  
Robot Interaction ◽  
Interaction Field ◽  
Nao Robot

An intriguing challenge in the human–robot interaction field is the prospect of endowing robots with emotional intelligence to make the interaction more genuine, intuitive, and natural. A crucial aspect in achieving this goal is the robot’s capability to infer and interpret human emotions. Thanks to its design and open programming platform, the NAO humanoid robot is one of the most widely used agents for human interaction. As with person-to-person communication, facial expressions are the privileged channel for recognizing the interlocutor’s emotional expressions. Although NAO is equipped with a facial expression recognition module, specific use cases may require additional features and affective computing capabilities that are not currently available. This study proposes a highly accurate convolutional-neural-network-based facial expression recognition model that is able to further enhance the NAO robot’ awareness of human facial expressions and provide the robot with an interlocutor’s arousal level detection capability. Indeed, the model tested during human–robot interactions was 91% and 90% accurate in recognizing happy and sad facial expressions, respectively; 75% accurate in recognizing surprised and scared expressions; and less accurate in recognizing neutral and angry expressions. Finally, the model was successfully integrated into the NAO SDK, thus allowing for high-performing facial expression classification with an inference time of 0.34 ± 0.04 s.

Emotional Facial Expression Recognition Task

2010 ◽  
Author(s):  
M. Fischer-Shofty ◽  
S. G. Shamay-Tsoorya ◽  
H. Harari ◽  
Y. Levkovitz
Keyword(s):  
Facial Expression ◽  
Facial Expression Recognition ◽  
Recognition Task ◽  
Expression Recognition ◽  
Emotional Facial Expression
Sign in / Sign up

Export Citation Format

Share Document