scholarly journals Guided Spatial Transformers for Facial Expression Recognition

2021 ◽  
Vol 11 (16) ◽  
pp. 7217
Author(s):  
Cristina Luna-Jiménez ◽  
Jorge Cristóbal-Martín ◽  
Ricardo Kleinlein ◽  
Manuel Gil-Martín ◽  
José M. Moya ◽  
...  
Keyword(s):  
Facial Expression ◽  
Facial Expression Recognition ◽  
Expert Knowledge ◽  
Visual Saliency ◽  
Expression Recognition ◽  
Spatial Transformations ◽  
Facial Landmarks ◽  
Saliency Maps ◽  
Best Fit

Spatial Transformer Networks are considered a powerful algorithm to learn the main areas of an image, but still, they could be more efficient by receiving images with embedded expert knowledge. This paper aims to improve the performance of conventional Spatial Transformers when applied to Facial Expression Recognition. Based on the Spatial Transformers’ capacity of spatial manipulation within networks, we propose different extensions to these models where effective attentional regions are captured employing facial landmarks or facial visual saliency maps. This specific attentional information is then hardcoded to guide the Spatial Transformers to learn the spatial transformations that best fit the proposed regions for better recognition results. For this study, we use two datasets: AffectNet and FER-2013. For AffectNet, we achieve a 0.35% point absolute improvement relative to the traditional Spatial Transformer, whereas for FER-2013, our solution gets an increase of 1.49% when models are fine-tuned with the Affectnet pre-trained weights.

scholarly journals Non-frontal facial expression recognition based on salient facial patches

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Bin Jiang ◽  
Qiuwen Zhang ◽  
Zuhe Li ◽  
Qinggang Wu ◽  
Huanlong Zhang
Keyword(s):  
Facial Expression ◽  
Facial Expression Recognition ◽  
Local Binary Pattern ◽  
Recognition Performance ◽  
Expression Recognition ◽  
Facial Landmarks ◽  
Face Images ◽  
Local Feature Extraction ◽  
Head Rotations ◽  
Part Model

AbstractMethods using salient facial patches (SFPs) play a significant role in research on facial expression recognition. However, most SFP methods use only frontal face images or videos for recognition, and they do not consider head position variations. We contend that SFP can be an effective approach for recognizing facial expressions under different head rotations. Accordingly, we propose an algorithm, called profile salient facial patches (PSFP), to achieve this objective. First, to detect facial landmarks and estimate head poses from profile face images, a tree-structured part model is used for pose-free landmark localization. Second, to obtain the salient facial patches from profile face images, the facial patches are selected using the detected facial landmarks while avoiding their overlap or the transcending of the actual face range. To analyze the PSFP recognition performance, three classical approaches for local feature extraction, specifically the histogram of oriented gradients (HOG), local binary pattern, and Gabor, were applied to extract profile facial expression features. Experimental results on the Radboud Faces Database show that PSFP with HOG features can achieve higher accuracies under most head rotations.

scholarly journals Non-frontal Facial Expression Recognition Based on Salient Facial Patches

2020 ◽  
Author(s):  
Bin Jiang ◽  
Qiuwen Zhang ◽  
Zuhe Li ◽  
Qinggang Wu ◽  
Huanlong Zhang
Keyword(s):  
Facial Expression ◽  
Facial Expression Recognition ◽  
Recognition Performance ◽  
Good Choice ◽  
Expression Recognition ◽  
Facial Landmarks ◽  
Face Images ◽  
Local Feature Extraction ◽  
Head Rotations ◽  
Part Model

Abstract Methods using salient facial patches (SFP) play a significant role in research on facial expression recognition. However, most SFP methods use only frontal face images or videos for recognition, and do not consider variations of head position. In our view, SFP can also be a good choice to recognize facial expression under different head rotations, and thus we propose an algorithm for this purpose, called Profile Salient Facial Patches (PSFP). First, in order to detect the facial landmarks from profile face images, the tree-structured part model is used for pose-free landmark localization; this approach excels at detecting facial landmarks and estimating head poses. Second, to obtain the salient facial patches from profile face images, the facial patches are selected using the detected facial landmarks, while avoiding overlap with each other or going beyond the range of the actual face. For the purpose of analyzing the recognition performance of PSFP, three classical approaches for local feature extraction-histogram of oriented Gradients (HOG), local binary pattern (LBP), and Gabor were applied to extract profile facial expression features. Experimental results on radboud faces database show that PSFP with HOG features can achieve higher accuracies under the most head rotations.

IMPROVING CNN FEATURES FOR FACIAL EXPRESSION RECOGNITION

2019 ◽  
Vol 3 (1) ◽  
pp. 1
Author(s):  
Ahmet Serdar Karadeniz ◽  
Mehmet Fatih Karadeniz ◽  
Gerhard Wilhelm Weber ◽  
Ismail Husein
Keyword(s):  
Neural Network ◽  
Facial Expression ◽  
Facial Expression Recognition ◽  
Test Accuracy ◽  
Expression Recognition ◽  
Facial Landmarks ◽  
Challenging Tasks ◽  
Key Word

<span class="fontstyle0">Abstract </span><span class="fontstyle2">Facial expression recognition is one of the challenging tasks in computer<br />vision. In this paper, we analyzed and improved the performances both<br />handcrafted features and deep features extracted by Convolutional Neural<br />Network (CNN). Eigenfaces, HOG, Dense-SIFT were used as handcrafted features.<br />Additionally, we developed features based on the distances between facial<br />landmarks and SIFT descriptors around the centroids of the facial landmarks,<br />leading to a better performance than Dense-SIFT. We achieved 68.34 % accuracy<br />with a CNN model trained from scratch. By combining CNN features with<br />handcrafted features, we achieved 69.54 % test accuracy.<br /></span><span class="fontstyle0">Key Word</span><span class="fontstyle3">: </span><span class="fontstyle2">Neural network, facial expression recognition, handcrafted features</span> <br /><br />

Sign in / Sign up

Export Citation Format

Share Document