Cross-database Micro Expression Recognition Based on Apex Frame Optical Flow and Multi-head Self-attention

2021 ◽  
pp. 128-139
Author(s):  
Jiebin Wen ◽  
Wenzhong Yang ◽  
LieJun Wang ◽  
Wenyu Wei ◽  
Sixiang Tan ◽  
...  
Keyword(s):  
Optical Flow ◽  
Expression Recognition ◽  
Micro Expression

scholarly journals Optical Flow Filtering-Based Micro-Expression Recognition Method

Electronics ◽  
2020 ◽  
Vol 9 (12) ◽  
pp. 2056
Author(s):  
Junjie Wu ◽  
Jianfeng Xu ◽  
Deyu Lin ◽  
Min Tu
Keyword(s):  
Optical Flow ◽  
Recognition Accuracy ◽  
Recognition Performance ◽  
Recognition Rate ◽  
Expression Recognition ◽  
Recognition Method ◽  
Video Clips ◽  
Low Degree ◽  
Micro Expression

The recognition accuracy of micro-expressions in the field of facial expressions is still understudied, as current research methods mainly focus on feature extraction and classification. Based on optical flow and decision thinking theory, we propose a novel micro-expression recognition method, which can filter low-quality micro-expression video clips. Determined by preset thresholds, we develop two optical flow filtering mechanisms: one based on two-branch decisions (OFF2BD) and the other based on three-way decisions (OFF3WD). In OFF2BD, which use the classical binary logic to classify images, and divide the images into positive or negative domain for further filtering. Differ from the OFF2BD, OFF3WD added boundary domain to delay to judge the motion quality of the images. In this way, the video clips with low degree of morphological change can be eliminated, so as to directly improve the quality of micro-expression features and recognition rate. From the experimental results, we verify the recognition accuracy of 61.57%, and 65.41% for CASMEII, and SMIC datasets, respectively. Through the comparative analysis, it shows that the scheme can effectively improve the recognition performance.

Deep Convolutional Neural Network with Optical Flow for Facial Micro-Expression Recognition

2019 ◽  
Vol 29 (01) ◽  
pp. 2050006 ◽  
Author(s):  
Qiuyu Li ◽  
Jun Yu ◽  
Toru Kurihara ◽  
Haiyan Zhang ◽  
Shu Zhan
Keyword(s):  
Neural Network ◽  
Optical Flow ◽  
Video Clip ◽  
Support Vector ◽  
Expression Recognition ◽  
Convolutional Network ◽  
Facial Landmarks ◽  
Flow Features ◽  
Optical Flow Features ◽  
Micro Expression

Micro-expression is a kind of brief facial movements which could not be controlled by the nervous system. Micro-expression indicates that a person is hiding his true emotion consciously. Micro-expression recognition has various potential applications in public security and clinical medicine. Researches are focused on the automatic micro-expression recognition, because it is hard to recognize the micro-expression by people themselves. This research proposed a novel algorithm for automatic micro-expression recognition which combined a deep multi-task convolutional network for detecting the facial landmarks and a fused deep convolutional network for estimating the optical flow features of the micro-expression. First, the deep multi-task convolutional network is employed to detect facial landmarks with the manifold-related tasks for dividing the facial region. Furthermore, a fused convolutional network is applied for extracting the optical flow features from the facial regions which contain the muscle changes when the micro-expression appears. Because each video clip has many frames, the original optical flow features of the whole video clip will have high number of dimensions and redundant information. This research revises the optical flow features for reducing the redundant dimensions. Finally, a revised optical flow feature is applied for refining the information of the features and a support vector machine classifier is adopted for recognizing the micro-expression. The main contribution of work is combining the deep multi-task learning neural network and the fusion optical flow network for micro-expression recognition and revising the optical flow features for reducing the redundant dimensions. The results of experiments on two spontaneous micro-expression databases prove that our method achieved competitive performance in micro-expression recognition.

scholarly journals An Improved Micro-Expression Recognition Method Based on Necessary Morphological Patches

Symmetry ◽  
2019 ◽  
Vol 11 (4) ◽  
pp. 497 ◽  
Author(s):  
Yue Zhao ◽  
Jiancheng Xu
Keyword(s):  
Optical Flow ◽  
Support Vector ◽  
Svm Classifier ◽  
Expression Recognition ◽  
Optical Flow Method ◽  
Psychological Analysis ◽  
Emotional Representation ◽  
Discriminant Ability ◽  
Casme Ii ◽  
Micro Expression

Micro-expression is a spontaneous emotional representation that is not controlled by logic. A micro-expression is both transitory (short duration) and subtle (small intensity), so it is difficult to detect in people. Micro-expression detection is widely used in the fields of psychological analysis, criminal justice and human-computer interaction. Additionally, like traditional facial expressions, micro-expressions also have local muscle movement. Psychologists have shown micro-expressions have necessary morphological patches (NMPs), which are triggered by emotion. Furthermore, the objective of this paper is to sort and filter these NMPs and extract features from NMPs to train classifiers to recognize micro-expressions. Firstly, we use the optical flow method to compare the on-set frame and the apex frame of the micro-expression sequences. By doing this, we could find facial active patches. Secondly, to find the NMPs of micro-expressions, this study calculates the local binary pattern from three orthogonal planes (LBP-TOP) operators and cascades them with optical flow histograms to form the fusion features of the active patches. Finally, a random forest feature selection (RFFS) algorithm is used to identify the NMPs and to characterize them via support vector machine (SVM) classifier. We evaluated the proposed method on two popular publicly available databases: CASME II and SMIC. Results show that NMPs are statistically determined and contribute to significant discriminant ability instead of holistic utilization of all facial regions.

A Main Directional Mean Optical Flow Feature for Spontaneous Micro-Expression Recognition

2016 ◽  
Vol 7 (4) ◽  
pp. 299-310 ◽  
Author(s):  
Yong-Jin Liu ◽  
Jin-Kai Zhang ◽  
Wen-Jing Yan ◽  
Su-Jing Wang ◽  
Guoying Zhao ◽  
...  
Keyword(s):  
Optical Flow ◽  
Expression Recognition ◽  
Flow Feature ◽  
Micro Expression

scholarly journals Multi-scale joint feature network for micro-expression recognition

2021 ◽  
Author(s):  
Xinyu Li ◽  
Guangshun Wei ◽  
Jie Wang ◽  
Yuanfeng Zhou
Keyword(s):  
Optical Flow ◽  
Expression Recognition ◽  
Facial Motion ◽  
Multi Scale ◽  
Flow Image ◽  
Wide Range ◽  
Recognition Ability ◽  
Benchmark Datasets ◽  
Casme Ii ◽  
Micro Expression

AbstractMicro-expression recognition is a substantive cross-study of psychology and computer science, and it has a wide range of applications (e.g., psychological and clinical diagnosis, emotional analysis, criminal investigation, etc.). However, the subtle and diverse changes in facial muscles make it difficult for existing methods to extract effective features, which limits the improvement of micro-expression recognition accuracy. Therefore, we propose a multi-scale joint feature network based on optical flow images for micro-expression recognition. First, we generate an optical flow image that reflects subtle facial motion information. The optical flow image is then fed into the multi-scale joint network for feature extraction and classification. The proposed joint feature module (JFM) integrates features from different layers, which is beneficial for the capture of micro-expression features with different amplitudes. To improve the recognition ability of the model, we also adopt a strategy for fusing the feature prediction results of the three JFMs with the backbone network. Our experimental results show that our method is superior to state-of-the-art methods on three benchmark datasets (SMIC, CASME II, and SAMM) and a combined dataset (3DB).

Sign in / Sign up

Export Citation Format

Share Document