Micro-expression recognition convolutional network based on dual-stream temporal-domain information interaction

2020 ◽  
Author(s):  
Weijie Zhu ◽  
Ying Chen
Keyword(s):  
Expression Recognition ◽  
Convolutional Network ◽  
Temporal Domain ◽  
Information Interaction ◽  
Micro Expression ◽  
Domain Information

Deep Convolutional Neural Network with Optical Flow for Facial Micro-Expression Recognition

2019 ◽  
Vol 29 (01) ◽  
pp. 2050006 ◽  
Author(s):  
Qiuyu Li ◽  
Jun Yu ◽  
Toru Kurihara ◽  
Haiyan Zhang ◽  
Shu Zhan
Keyword(s):  
Neural Network ◽  
Optical Flow ◽  
Video Clip ◽  
Support Vector ◽  
Expression Recognition ◽  
Convolutional Network ◽  
Facial Landmarks ◽  
Flow Features ◽  
Optical Flow Features ◽  
Micro Expression

Micro-expression is a kind of brief facial movements which could not be controlled by the nervous system. Micro-expression indicates that a person is hiding his true emotion consciously. Micro-expression recognition has various potential applications in public security and clinical medicine. Researches are focused on the automatic micro-expression recognition, because it is hard to recognize the micro-expression by people themselves. This research proposed a novel algorithm for automatic micro-expression recognition which combined a deep multi-task convolutional network for detecting the facial landmarks and a fused deep convolutional network for estimating the optical flow features of the micro-expression. First, the deep multi-task convolutional network is employed to detect facial landmarks with the manifold-related tasks for dividing the facial region. Furthermore, a fused convolutional network is applied for extracting the optical flow features from the facial regions which contain the muscle changes when the micro-expression appears. Because each video clip has many frames, the original optical flow features of the whole video clip will have high number of dimensions and redundant information. This research revises the optical flow features for reducing the redundant dimensions. Finally, a revised optical flow feature is applied for refining the information of the features and a support vector machine classifier is adopted for recognizing the micro-expression. The main contribution of work is combining the deep multi-task learning neural network and the fusion optical flow network for micro-expression recognition and revising the optical flow features for reducing the redundant dimensions. The results of experiments on two spontaneous micro-expression databases prove that our method achieved competitive performance in micro-expression recognition.

scholarly journals Micro-Expression Recognition Enhanced by Macro-Expression from Spatial-Temporal Domain

2021 ◽  
Author(s):  
Bin Xia ◽  
Shangfei Wang
Keyword(s):  
High Performance ◽  
Spatial Domain ◽  
Behavior Patterns ◽  
Expression Recognition ◽  
Expression Data ◽  
Baseline Model ◽  
Temporal Domain ◽  
Temporal Features ◽  
Facial Behavior ◽  
Micro Expression

Facial micro-expression recognition has attracted much attention due to its objectiveness to reveal the true emotion of a person. However, the limited micro-expression datasets have posed a great challenge to train a high performance micro-expression classifier. Since micro-expression and macro-expression share some similarities in both spatial and temporal facial behavior patterns, we propose a macro-to-micro transformation framework for micro-expression recognition. Specifically, we first pretrain two-stream baseline model from micro-expression data and macro-expression data respectively, named MiNet and MaNet. Then, we introduce two auxiliary tasks to align the spatial and temporal features learned from micro-expression data and macro-expression data. In spatial domain, we introduce a domain discriminator to align the features of MiNet and MaNet. In temporal domain, we introduce relation classifier to predict the correct relation for temporal features from MaNet and MiNet. Finally, we propose contrastive loss to encourage the MiNet to give closely aligned features to all entries from the same class in each instance. Experiments on three benchmark databases demonstrate the superiority of the proposed method.

scholarly journals Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network

Sensors ◽  
2021 ◽  
Vol 21 (9) ◽  
pp. 3046
Author(s):  
Shervin Minaee ◽  
Mehdi Minaei ◽  
Amirali Abdolrashidi
Keyword(s):  
Deep Learning ◽  
Facial Expression ◽  
Facial Expression Recognition ◽  
Expression Recognition ◽  
Visualization Technique ◽  
Convolutional Network ◽  
The Past ◽  
Multiple Datasets ◽  
The Face ◽  
Traditional Approaches

Facial expression recognition has been an active area of research over the past few decades, and it is still challenging due to the high intra-class variation. Traditional approaches for this problem rely on hand-crafted features such as SIFT, HOG, and LBP, followed by a classifier trained on a database of images or videos. Most of these works perform reasonably well on datasets of images captured in a controlled condition but fail to perform as well on more challenging datasets with more image variation and partial faces. In recent years, several works proposed an end-to-end framework for facial expression recognition using deep learning models. Despite the better performance of these works, there are still much room for improvement. In this work, we propose a deep learning approach based on attentional convolutional network that is able to focus on important parts of the face and achieves significant improvement over previous models on multiple datasets, including FER-2013, CK+, FERG, and JAFFE. We also use a visualization technique that is able to find important facial regions to detect different emotions based on the classifier’s output. Through experimental results, we show that different emotions are sensitive to different parts of the face.

Sign in / Sign up

Export Citation Format

Share Document