Facial expression recognition based on bidirectional gated recurrent units within deep residual network

2020 ◽  
Vol 13 (4) ◽  
pp. 527-543
Author(s):  
Wenjuan Shen ◽  
Xiaoling Li
Keyword(s):  
Facial Expression ◽  
Facial Expression Recognition ◽  
Clinical Medicine ◽  
Recognition Accuracy ◽  
Recognition Performance ◽  
Recognition Rate ◽  
Expression Recognition ◽  
Safe Driving ◽  
Content Type ◽  
Proposed Model

Purposerecent years, facial expression recognition has been widely used in human machine interaction, clinical medicine and safe driving. However, there is a limitation that conventional recurrent neural networks can only learn the time-series characteristics of expressions based on one-way propagation information.Design/methodology/approachTo solve such limitation, this paper proposes a novel model based on bidirectional gated recurrent unit networks (Bi-GRUs) with two-way propagations, and the theory of identity mapping residuals is adopted to effectively prevent the problem of gradient disappearance caused by the depth of the introduced network. Since the Inception-V3 network model for spatial feature extraction has too many parameters, it is prone to overfitting during training. This paper proposes a novel facial expression recognition model to add two reduction modules to reduce parameters, so as to obtain an Inception-W network with better generalization.FindingsFinally, the proposed model is pretrained to determine the best settings and selections. Then, the pretrained model is experimented on two facial expression data sets of CK+ and Oulu- CASIA, and the recognition performance and efficiency are compared with the existing methods. The highest recognition rate is 99.6%, which shows that the method has good recognition accuracy in a certain range.Originality/valueBy using the proposed model for the applications of facial expression, the high recognition accuracy and robust recognition results with lower time consumption will help to build more sophisticated applications in real world.

scholarly journals Three-Stream Convolutional Neural Network with Squeeze-and-Excitation Block for Near-Infrared Facial Expression Recognition

Electronics ◽  
2019 ◽  
Vol 8 (4) ◽  
pp. 385 ◽  
Author(s):  
Ying Chen ◽  
Zhihao Zhang ◽  
Lei Zhong ◽  
Tong Chen ◽  
Juxiang Chen ◽  
...  
Keyword(s):  
Neural Network ◽  
Facial Expression ◽  
Facial Expression Recognition ◽  
Near Infrared ◽  
Recognition Accuracy ◽  
State Of The Art ◽  
Recognition Rate ◽  
Three Dimensional ◽  
Expression Recognition ◽  
Facial Expression Database

Near-infrared (NIR) facial expression recognition is resistant to illumination change. In this paper, we propose a three-stream three-dimensional convolution neural network with a squeeze-and-excitation (SE) block for NIR facial expression recognition. We fed each stream with different local regions, namely the eyes, nose, and mouth. By using an SE block, the network automatically allocated weights to different local features to further improve recognition accuracy. The experimental results on the Oulu-CASIA NIR facial expression database showed that the proposed method has a higher recognition rate than some state-of-the-art algorithms.

FEATURE-BASED FACIAL EXPRESSION RECOGNITION: SENSITIVITY ANALYSIS AND EXPERIMENTS WITH A MULTILAYER PERCEPTRON

1999 ◽  
Vol 13 (06) ◽  
pp. 893-911 ◽  
Author(s):  
ZHENGYOU ZHANG
Keyword(s):  
Sensitivity Analysis ◽  
Facial Expression ◽  
Facial Expression Recognition ◽  
Recognition Performance ◽  
Recognition Rate ◽  
Low Frequency ◽  
Gabor Wavelet ◽  
Expression Recognition ◽  
Wavelet Coefficients ◽  
Feature Based

In this paper, we report our experiments on feature-based facial expression recognition within an architecture based on a two-layer perceptron. We investigate the use of two types of features extracted from face images: the geometric positions of a set of fiducial points on a face, and a set of multiscale and multiorientation Gabor wavelet coefficients at these points. They can be used either independently or jointly. The recognition performance with different types of features has been compared, which shows that Gabor wavelet coefficients are much more powerful than geometric positions. Furthermore, since the first layer of the perceptron actually performs a nonlinear reduction of the dimensionality of the feature space, we have also studied the desired number of hidden units, i.e. the appropriate dimension to represent a facial expression in order to achieve a good recognition rate. It turns out that five to seven hidden units are probably enough to represent the space of facial expressions. Then, we have investigated the importance of each individual fiducial point to facial expression recognition. Sensitivity analysis reveals that points on cheeks and on forehead carry little useful information. After discarding them, not only the computational efficiency increases, but also the generalization performance slightly improves. Finally, we have studied the significance of image scales. Experiments show that facial expression recognition is mainly a low frequency process, and a spatial resolution of 64 pixels × 64 pixels is probably enough.

Selective Facial Expression Recognition Using fastICA

2012 ◽  
Vol 433-440 ◽  
pp. 2755-2761 ◽  
Author(s):  
Xiao Hua Zhang ◽  
Zhi Fei Liu ◽  
Ya Jun Guo ◽  
Li Qiang Zhao
Keyword(s):  
Neural Network ◽  
Facial Expression ◽  
Bp Neural Network ◽  
Facial Expression Recognition ◽  
Recognition Performance ◽  
Recognition Rate ◽  
Facial Features ◽  
Expression Recognition ◽  
Average Recognition Rate ◽  
Neural Network Classifiers

This paper proposes a facial expression recognition approach based on the combination of fastICA method and neural network classifiers. First we get some special facial expression regions, including eyebrows, eyes and mouth, in which wavelet transform is done to reduce the dimension. Then the fastICA method is used to extract these three facial features. Finally, BP neural network classifier is adopted to recognize facial expression. Experimental on the JAFFE database results show that the method is effective for both dimension reduction and recognition performance in comparison with traditional PCA and ICA method. We have obtained recognition rates as high as 93.33% in categorizing the facial expressions neutral, anger, or sadness. The best average recognition rate achieves 90.48%.

Computer Intelligent Nursing Care of Critically III Patients Based on Improved Patient Facial Expression Recognition Method

2014 ◽  
Vol 543-547 ◽  
pp. 2350-2353
Author(s):  
Xiao Yan Wan
Keyword(s):  
Facial Expression ◽  
Critically Ill ◽  
Nursing Care ◽  
Facial Expression Recognition ◽  
Recognition Rate ◽  
Support Vector ◽  
Appearance Model ◽  
Expression Recognition ◽  
Recognition Method ◽  
The Face

In order to extract the expression features of critically ill patients, and realize the computer intelligent nursing, an improved facial expression recognition method is proposed based on the of active appearance model, the support vector machine (SVM) for facial expression recognition is taken in research, and the face recognition model structure active appearance model is designed, and the attribute reduction algorithm of rough set affine transformation theory is introduced, and the invalid and redundant feature points are removed. The critically ill patient expressions are classified and recognized based on the support vector machine (SVM). The face image attitudes are adjusted, and the self-adaptive performance of facial expression recognition for the critical patient attitudes is improved. New method overcomes the effect of patient attitude to the recognition rate to a certain extent. The highest average recognition rate can be increased about 7%. The intelligent monitoring and nursing care of critically ill patients are realized with the computer vision effect. The nursing quality is enhanced, and it ensures the timely treatment of rescue.

A Hybrid Approach for Facial Expression Recognition Using Extended Local Binary Patterns and Principal Component Analysis

2019 ◽  
Vol 8 (2) ◽  
pp. 1-25
Author(s):  
Gopal Krishan Prajapat ◽  
Rakesh Kumar
Keyword(s):  
Principal Component Analysis ◽  
Feature Extraction ◽  
Facial Expression ◽  
Facial Expression Recognition ◽  
Local Binary Pattern ◽  
Recognition Rate ◽  
Principal Component ◽  
Component Analysis ◽  
Expression Recognition ◽  
Distance Method

Facial feature extraction and recognition plays a prominent role in human non-verbal interaction and it is one of the crucial factors among pose, speech, facial expression, behaviour and actions which are used in conveying information about the intentions and emotions of a human being. In this article an extended local binary pattern is used for the feature extraction process and a principal component analysis (PCA) is used for dimensionality reduction. The projections of the sample and model images are calculated and compared by Euclidean distance method. The combination of extended local binary pattern and PCA (ELBP+PCA) improves the accuracy of the recognition rate and also diminishes the evaluation complexity. The evaluation of proposed facial expression recognition approach will focus on the performance of the recognition rate. A series of tests are performed for the validation of algorithms and to compare the accuracy of the methods on the JAFFE, Extended Cohn-Kanade images database.

scholarly journals Are affective factors related to individual differences in facial expression recognition?

2020 ◽  
Vol 7 (9) ◽  
pp. 190699
Author(s):  
Sarah A. H. Alharbi ◽  
Katherine Button ◽  
Lingshan Zhang ◽  
Kieran J. O'Shea ◽  
Vanessa Fasolt ◽  
...  
Keyword(s):  
Individual Differences ◽  
Facial Expression ◽  
Emotion Recognition ◽  
Facial Expression Recognition ◽  
Recognition Performance ◽  
Expression Recognition ◽  
General Anxiety ◽  
Affective Factors ◽  
Multimodal Emotion Recognition ◽  
Anxiety Subscale

Evidence that affective factors (e.g. anxiety, depression, affect) are significantly related to individual differences in emotion recognition is mixed. Palermo et al . (Palermo et al . 2018 J. Exp. Psychol. Hum. Percept. Perform. 44 , 503–517) reported that individuals who scored lower in anxiety performed significantly better on two measures of facial-expression recognition (emotion-matching and emotion-labelling tasks), but not a third measure (the multimodal emotion recognition test). By contrast, facial-expression recognition was not significantly correlated with measures of depression, positive or negative affect, empathy, or autistic-like traits. Because the range of affective factors considered in this study and its use of multiple expression-recognition tasks mean that it is a relatively comprehensive investigation of the role of affective factors in facial expression recognition, we carried out a direct replication. In common with Palermo et al . (Palermo et al . 2018 J. Exp. Psychol. Hum. Percept. Perform. 44 , 503–517), scores on the DASS anxiety subscale negatively predicted performance on the emotion recognition tasks across multiple analyses, although these correlations were only consistently significant for performance on the emotion-labelling task. However, and by contrast with Palermo et al . (Palermo et al . 2018 J. Exp. Psychol. Hum. Percept. Perform. 44 , 503–517), other affective factors (e.g. those related to empathy) often also significantly predicted emotion-recognition performance. Collectively, these results support the proposal that affective factors predict individual differences in emotion recognition, but that these correlations are not necessarily specific to measures of general anxiety, such as the DASS anxiety subscale.

scholarly journals Optical Flow Filtering-Based Micro-Expression Recognition Method

Electronics ◽  
2020 ◽  
Vol 9 (12) ◽  
pp. 2056
Author(s):  
Junjie Wu ◽  
Jianfeng Xu ◽  
Deyu Lin ◽  
Min Tu
Keyword(s):  
Optical Flow ◽  
Recognition Accuracy ◽  
Recognition Performance ◽  
Recognition Rate ◽  
Expression Recognition ◽  
Recognition Method ◽  
Video Clips ◽  
Low Degree ◽  
Micro Expression

The recognition accuracy of micro-expressions in the field of facial expressions is still understudied, as current research methods mainly focus on feature extraction and classification. Based on optical flow and decision thinking theory, we propose a novel micro-expression recognition method, which can filter low-quality micro-expression video clips. Determined by preset thresholds, we develop two optical flow filtering mechanisms: one based on two-branch decisions (OFF2BD) and the other based on three-way decisions (OFF3WD). In OFF2BD, which use the classical binary logic to classify images, and divide the images into positive or negative domain for further filtering. Differ from the OFF2BD, OFF3WD added boundary domain to delay to judge the motion quality of the images. In this way, the video clips with low degree of morphological change can be eliminated, so as to directly improve the quality of micro-expression features and recognition rate. From the experimental results, we verify the recognition accuracy of 61.57%, and 65.41% for CASMEII, and SMIC datasets, respectively. Through the comparative analysis, it shows that the scheme can effectively improve the recognition performance.

scholarly journals A Facial Expression Recognition Method Using Improved Capsule Network Model

2020 ◽  
Vol 2020 ◽  
pp. 1-12
Author(s):  
Yifeng Zhao ◽  
Deyun Chen
Keyword(s):  
Facial Expression ◽  
Network Model ◽  
Facial Expression Recognition ◽  
Recognition Rate ◽  
Regression Tree ◽  
Gaussian Process Regression ◽  
Learning Technology ◽  
Expression Recognition ◽  
Recognition Method ◽  
The Face

Aiming at the problem of facial expression recognition under unconstrained conditions, a facial expression recognition method based on an improved capsule network model is proposed. Firstly, the expression image is normalized by illumination based on the improved Weber face, and the key points of the face are detected by the Gaussian process regression tree. Then, the 3dmms model is introduced. The 3D face shape, which is consistent with the face in the image, is provided by iterative estimation so as to further improve the image quality of face pose standardization. In this paper, we consider that the convolution features used in facial expression recognition need to be trained from the beginning and add as many different samples as possible in the training process. Finally, this paper attempts to combine the traditional deep learning technology with capsule configuration, adds an attention layer after the primary capsule layer in the capsule network, and proposes an improved capsule structure model suitable for expression recognition. The experimental results on JAFFE and BU-3DFE datasets show that the recognition rate can reach 96.66% and 80.64%, respectively.

CMAC WITH CLUSTERING MEMORY AND ITS APPLICATION TO FACIAL EXPRESSION RECOGNITION

2011 ◽  
Vol 25 (07) ◽  
pp. 1055-1072 ◽  
Author(s):  
YU-YI LIAO ◽  
JZAU-SHENG LIN ◽  
SHEN-CHUAN TAI
Keyword(s):  
Facial Expression ◽  
Facial Expression Recognition ◽  
Recognition Rate ◽  
Recognition System ◽  
Expression Recognition ◽  
Test Image ◽  
Frontal View ◽  
Still Images ◽  
Dct Coefficients ◽  
Block Sizes

In this paper, a facial expression recognition system based on cerebella model articulation controller with a clustering memory (CMAC-CM) is presented. Firstly, the facial expression features were automatically preprocessed and extracted from given still images in the JAFFE database in which the frontal view of faces were contained. Next, a block of lower frequency DCT coefficients was obtained by subtracting a neutral image from a given expression image and rearranged as input vectors to be fed into the CMAC-CM that can rapidly obtain output using nonlinear mapping with a look-up table in training or recognizing phase. Finally, the experimental results have demonstrated recognition rates with various block sizes of coefficients in lower frequency and cluster sizes of weight memory. A mean recognition rate of 92.86% is achieved for the testing images. CMAC-CM takes 0.028 seconds for test image in testing phase.

Sign in / Sign up

Export Citation Format

Share Document