Comparison of facial expression recognition performance according to the use of depth information of structured-light type RGB-D camera

2019 ◽  
Author(s):  
Kunyoung Lee ◽  
Eui Chul Lee
Keyword(s):  
Facial Expression ◽  
Facial Expression Recognition ◽  
Recognition Performance ◽  
Structured Light ◽  
Depth Information ◽  
Expression Recognition

scholarly journals Are affective factors related to individual differences in facial expression recognition?

2020 ◽  
Vol 7 (9) ◽  
pp. 190699
Author(s):  
Sarah A. H. Alharbi ◽  
Katherine Button ◽  
Lingshan Zhang ◽  
Kieran J. O'Shea ◽  
Vanessa Fasolt ◽  
...  
Keyword(s):  
Individual Differences ◽  
Facial Expression ◽  
Emotion Recognition ◽  
Facial Expression Recognition ◽  
Recognition Performance ◽  
Expression Recognition ◽  
General Anxiety ◽  
Affective Factors ◽  
Multimodal Emotion Recognition ◽  
Anxiety Subscale

Evidence that affective factors (e.g. anxiety, depression, affect) are significantly related to individual differences in emotion recognition is mixed. Palermo et al . (Palermo et al . 2018 J. Exp. Psychol. Hum. Percept. Perform. 44 , 503–517) reported that individuals who scored lower in anxiety performed significantly better on two measures of facial-expression recognition (emotion-matching and emotion-labelling tasks), but not a third measure (the multimodal emotion recognition test). By contrast, facial-expression recognition was not significantly correlated with measures of depression, positive or negative affect, empathy, or autistic-like traits. Because the range of affective factors considered in this study and its use of multiple expression-recognition tasks mean that it is a relatively comprehensive investigation of the role of affective factors in facial expression recognition, we carried out a direct replication. In common with Palermo et al . (Palermo et al . 2018 J. Exp. Psychol. Hum. Percept. Perform. 44 , 503–517), scores on the DASS anxiety subscale negatively predicted performance on the emotion recognition tasks across multiple analyses, although these correlations were only consistently significant for performance on the emotion-labelling task. However, and by contrast with Palermo et al . (Palermo et al . 2018 J. Exp. Psychol. Hum. Percept. Perform. 44 , 503–517), other affective factors (e.g. those related to empathy) often also significantly predicted emotion-recognition performance. Collectively, these results support the proposal that affective factors predict individual differences in emotion recognition, but that these correlations are not necessarily specific to measures of general anxiety, such as the DASS anxiety subscale.

scholarly journals CNN-Based Facial Expression Recognition from Annotated RGB-D Images for Human–Robot Interaction

2019 ◽  
Vol 16 (04) ◽  
pp. 1941002 ◽  
Author(s):  
Jing Li ◽  
Yang Mi ◽  
Gongfa Li ◽  
Zhaojie Ju
Keyword(s):  
Facial Expression ◽  
Facial Expression Recognition ◽  
Recognition Task ◽  
Recognition System ◽  
Human Robot Interaction ◽  
Microsoft Kinect ◽  
Depth Information ◽  
Expression Recognition ◽  
Stream Network ◽  
Depth Images

Facial expression recognition has been widely used in human computer interaction (HCI) systems. Over the years, researchers have proposed different feature descriptors, implemented different classification methods, and carried out a number of experiments on various datasets for automatic facial expression recognition. However, most of them used 2D static images or 2D video sequences for the recognition task. The main limitations of 2D-based analysis are problems associated with variations in pose and illumination, which reduce the recognition accuracy. Therefore, an alternative way is to incorporate depth information acquired by 3D sensor, because it is invariant in both pose and illumination. In this paper, we present a two-stream convolutional neural network (CNN)-based facial expression recognition system and test it on our own RGB-D facial expression dataset collected by Microsoft Kinect for XBOX in unspontaneous scenarios since Kinect is an inexpensive and portable device to capture both RGB and depth information. Our fully annotated dataset includes seven expressions (i.e., neutral, sadness, disgust, fear, happiness, anger, and surprise) for 15 subjects (9 males and 6 females) aged from 20 to 25. The two individual CNNs are identical in architecture but do not share parameters. To combine the detection results produced by these two CNNs, we propose the late fusion approach. The experimental results demonstrate that the proposed two-stream network using RGB-D images is superior to that of using only RGB images or depth images.

scholarly journals Non-frontal facial expression recognition based on salient facial patches

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Bin Jiang ◽  
Qiuwen Zhang ◽  
Zuhe Li ◽  
Qinggang Wu ◽  
Huanlong Zhang
Keyword(s):  
Facial Expression ◽  
Facial Expression Recognition ◽  
Local Binary Pattern ◽  
Recognition Performance ◽  
Expression Recognition ◽  
Facial Landmarks ◽  
Face Images ◽  
Local Feature Extraction ◽  
Head Rotations ◽  
Part Model

AbstractMethods using salient facial patches (SFPs) play a significant role in research on facial expression recognition. However, most SFP methods use only frontal face images or videos for recognition, and they do not consider head position variations. We contend that SFP can be an effective approach for recognizing facial expressions under different head rotations. Accordingly, we propose an algorithm, called profile salient facial patches (PSFP), to achieve this objective. First, to detect facial landmarks and estimate head poses from profile face images, a tree-structured part model is used for pose-free landmark localization. Second, to obtain the salient facial patches from profile face images, the facial patches are selected using the detected facial landmarks while avoiding their overlap or the transcending of the actual face range. To analyze the PSFP recognition performance, three classical approaches for local feature extraction, specifically the histogram of oriented gradients (HOG), local binary pattern, and Gabor, were applied to extract profile facial expression features. Experimental results on the Radboud Faces Database show that PSFP with HOG features can achieve higher accuracies under most head rotations.

scholarly journals Non-frontal Facial Expression Recognition Based on Salient Facial Patches

2020 ◽  
Author(s):  
Bin Jiang ◽  
Qiuwen Zhang ◽  
Zuhe Li ◽  
Qinggang Wu ◽  
Huanlong Zhang
Keyword(s):  
Facial Expression ◽  
Facial Expression Recognition ◽  
Recognition Performance ◽  
Good Choice ◽  
Expression Recognition ◽  
Facial Landmarks ◽  
Face Images ◽  
Local Feature Extraction ◽  
Head Rotations ◽  
Part Model

Abstract Methods using salient facial patches (SFP) play a significant role in research on facial expression recognition. However, most SFP methods use only frontal face images or videos for recognition, and do not consider variations of head position. In our view, SFP can also be a good choice to recognize facial expression under different head rotations, and thus we propose an algorithm for this purpose, called Profile Salient Facial Patches (PSFP). First, in order to detect the facial landmarks from profile face images, the tree-structured part model is used for pose-free landmark localization; this approach excels at detecting facial landmarks and estimating head poses. Second, to obtain the salient facial patches from profile face images, the facial patches are selected using the detected facial landmarks, while avoiding overlap with each other or going beyond the range of the actual face. For the purpose of analyzing the recognition performance of PSFP, three classical approaches for local feature extraction-histogram of oriented Gradients (HOG), local binary pattern (LBP), and Gabor were applied to extract profile facial expression features. Experimental results on radboud faces database show that PSFP with HOG features can achieve higher accuracies under the most head rotations.

scholarly journals An expression recognition algorithm based on convolution neural network and RGB-D Images

2018 ◽  
Vol 173 ◽  
pp. 03066 ◽  
Author(s):  
HE binghua ◽  
CHEN zengzhao ◽  
LI gaoyang ◽  
JIANG lang ◽  
ZHANG zhao ◽  
...  
Keyword(s):  
Facial Expression ◽  
Facial Expression Recognition ◽  
Recognition Rate ◽  
Reference Value ◽  
Recognition Algorithm ◽  
Future Research ◽  
Depth Information ◽  
Expression Recognition ◽  
Expression Library ◽  
Training Time

Aiming at the problem of recognition effect is not stable when 2D facial expression recognition in the complex illumination and posture changes. A facial expression recognition algorithm based on RGB-D dynamic sequence analysis is proposed. The algorithm uses LBP features which are robust to illumination, and adds depth information to study the facial expression recognition. The algorithm firstly extracts 3D texture features of preprocessed RGB-D facial expression sequence, and then uses the CNN to train the dataset. At the same time, in order to verify the performance of the algorithm, a comprehensive facial expression library including 2D image, video and 3D depth information is constructed with the help of Intel RealSense technology. The experimental results show that the proposed algorithm has some advantages over other RGB-D facial expression recognition algorithms in training time and recognition rate, and has certain reference value for future research in facial expression recognition.

Facial expression recognition based on bidirectional gated recurrent units within deep residual network

2020 ◽  
Vol 13 (4) ◽  
pp. 527-543
Author(s):  
Wenjuan Shen ◽  
Xiaoling Li
Keyword(s):  
Facial Expression ◽  
Facial Expression Recognition ◽  
Clinical Medicine ◽  
Recognition Accuracy ◽  
Recognition Performance ◽  
Recognition Rate ◽  
Expression Recognition ◽  
Safe Driving ◽  
Content Type ◽  
Proposed Model

Purposerecent years, facial expression recognition has been widely used in human machine interaction, clinical medicine and safe driving. However, there is a limitation that conventional recurrent neural networks can only learn the time-series characteristics of expressions based on one-way propagation information.Design/methodology/approachTo solve such limitation, this paper proposes a novel model based on bidirectional gated recurrent unit networks (Bi-GRUs) with two-way propagations, and the theory of identity mapping residuals is adopted to effectively prevent the problem of gradient disappearance caused by the depth of the introduced network. Since the Inception-V3 network model for spatial feature extraction has too many parameters, it is prone to overfitting during training. This paper proposes a novel facial expression recognition model to add two reduction modules to reduce parameters, so as to obtain an Inception-W network with better generalization.FindingsFinally, the proposed model is pretrained to determine the best settings and selections. Then, the pretrained model is experimented on two facial expression data sets of CK+ and Oulu- CASIA, and the recognition performance and efficiency are compared with the existing methods. The highest recognition rate is 99.6%, which shows that the method has good recognition accuracy in a certain range.Originality/valueBy using the proposed model for the applications of facial expression, the high recognition accuracy and robust recognition results with lower time consumption will help to build more sophisticated applications in real world.

FEATURE-BASED FACIAL EXPRESSION RECOGNITION: SENSITIVITY ANALYSIS AND EXPERIMENTS WITH A MULTILAYER PERCEPTRON

1999 ◽  
Vol 13 (06) ◽  
pp. 893-911 ◽  
Author(s):  
ZHENGYOU ZHANG
Keyword(s):  
Sensitivity Analysis ◽  
Facial Expression ◽  
Facial Expression Recognition ◽  
Recognition Performance ◽  
Recognition Rate ◽  
Low Frequency ◽  
Gabor Wavelet ◽  
Expression Recognition ◽  
Wavelet Coefficients ◽  
Feature Based

In this paper, we report our experiments on feature-based facial expression recognition within an architecture based on a two-layer perceptron. We investigate the use of two types of features extracted from face images: the geometric positions of a set of fiducial points on a face, and a set of multiscale and multiorientation Gabor wavelet coefficients at these points. They can be used either independently or jointly. The recognition performance with different types of features has been compared, which shows that Gabor wavelet coefficients are much more powerful than geometric positions. Furthermore, since the first layer of the perceptron actually performs a nonlinear reduction of the dimensionality of the feature space, we have also studied the desired number of hidden units, i.e. the appropriate dimension to represent a facial expression in order to achieve a good recognition rate. It turns out that five to seven hidden units are probably enough to represent the space of facial expressions. Then, we have investigated the importance of each individual fiducial point to facial expression recognition. Sensitivity analysis reveals that points on cheeks and on forehead carry little useful information. After discarding them, not only the computational efficiency increases, but also the generalization performance slightly improves. Finally, we have studied the significance of image scales. Experiments show that facial expression recognition is mainly a low frequency process, and a spatial resolution of 64 pixels × 64 pixels is probably enough.

Selective Facial Expression Recognition Using fastICA

2012 ◽  
Vol 433-440 ◽  
pp. 2755-2761 ◽  
Author(s):  
Xiao Hua Zhang ◽  
Zhi Fei Liu ◽  
Ya Jun Guo ◽  
Li Qiang Zhao
Keyword(s):  
Neural Network ◽  
Facial Expression ◽  
Bp Neural Network ◽  
Facial Expression Recognition ◽  
Recognition Performance ◽  
Recognition Rate ◽  
Facial Features ◽  
Expression Recognition ◽  
Average Recognition Rate ◽  
Neural Network Classifiers

This paper proposes a facial expression recognition approach based on the combination of fastICA method and neural network classifiers. First we get some special facial expression regions, including eyebrows, eyes and mouth, in which wavelet transform is done to reduce the dimension. Then the fastICA method is used to extract these three facial features. Finally, BP neural network classifier is adopted to recognize facial expression. Experimental on the JAFFE database results show that the method is effective for both dimension reduction and recognition performance in comparison with traditional PCA and ICA method. We have obtained recognition rates as high as 93.33% in categorizing the facial expressions neutral, anger, or sadness. The best average recognition rate achieves 90.48%.

scholarly journals Differences in Facial Expression Recognition Between Unipolar and Bipolar Depression

2021 ◽  
Vol 12 ◽  
Author(s):  
Ma Ruihua ◽  
Zhao Meng ◽  
Chen Nan ◽  
Liu Panqi ◽  
Guo Hua ◽  
...  
Keyword(s):  
Facial Expression ◽  
Normal Control ◽  
Facial Expression Recognition ◽  
Bipolar Depression ◽  
Recognition Performance ◽  
Normal Control Group ◽  
Identification Accuracy ◽  
Control Group ◽  
Expression Recognition ◽  
Lower Accuracy

PurposeTo explore the differences in facial emotion recognition among patients with unipolar depression (UD), bipolar depression (BD), and normal controls.MethodsThirty patients with UD and 30 patients with BD, respectively, were recruited in Zhumadian Second People’s Hospital from July 2018 to August 2019. Fifteen groups of facial expressions including happiness, sadness, anger, surprise, fear, and disgust were identified.ResultsA single-factor ANOVA was used to analyze the facial expression recognition results of the three groups, and the differences were found in the happy-sad (P = 0.009), happy-angry (P = 0.001), happy-surprised (P = 0.034), and disgust-surprised (P = 0.038) facial expression groups. The independent sample T-test analysis showed that compared with the normal control group, there were differences in the happy-sad (P = 0.009) and happy-angry (P = 0.009) groups in patients with BD, and the accuracy of facial expression recognition was lower than the normal control group. Compared with patients with UD, there were differences between the happy-sad (P = 0.005) and happy-angry (P = 0.002) groups, and the identification accuracy of patients with UD was higher than that of patients with BD. The time of facial expression recognition in the normal control group was shorter than that in the patient group. Using happiness-sadness to distinguish unipolar and BDs, the area under the ROC curve (AUC) is 0.933, the specificity is 0.889, and the sensitivity is 0.667. Using happiness-anger to distinguish unipolar and BD, the AUC was 0.733, the specificity was 0.778, and the sensitivity was 0.600.ConclusionPatients with UD had lower performance in recognizing negative expressions and had longer recognition times. Those with BD had lower accuracy in recognizing positive expressions and longer recognition times. Rapid facial expression recognition performance may be as a potential endophenotype for early identification of unipolar and BD.

Sign in / Sign up

Export Citation Format

Share Document