scholarly journals FACS-Based Graph Features for Real-Time Micro-Expression Recognition

2020 ◽  
Vol 6 (12) ◽  
pp. 130
Author(s):  
Adamu Muhammad Buhari ◽  
Chee-Pun Ooi ◽  
Vishnu Monn Baskaran ◽  
Raphaël C. W. Phan ◽  
KokSheik Wong ◽  
...  
Keyword(s):  
Real Time ◽  
Recognition Accuracy ◽  
Expression Recognition ◽  
Coding System ◽  
Facial Landmark ◽  
Single Input ◽  
Measured Distance ◽  
The Cost ◽  
Micro Expression ◽  
Real Time Application

Several studies on micro-expression recognition have contributed mainly to accuracy improvement. However, the computational complexity receives lesser attention comparatively and therefore increases the cost of micro-expression recognition for real-time application. In addition, majority of the existing approaches required at least two frames (i.e., onset and apex frames) to compute features of every sample. This paper puts forward new facial graph features based on 68-point landmarks using Facial Action Coding System (FACS). The proposed feature extraction technique (FACS-based graph features) utilizes facial landmark points to compute graph for different Action Units (AUs), where the measured distance and gradient of every segment within an AU graph is presented as feature. Moreover, the proposed technique processes ME recognition based on single input frame sample. Results indicate that the proposed FACS-baed graph features achieve up to 87.33% of recognition accuracy with F1-score of 0.87 using leave one subject out cross-validation on SAMM datasets. Besides, the proposed technique computes features at the speed of 2 ms per sample on Xeon Processor E5-2650 machine.

scholarly journals Optical Flow Filtering-Based Micro-Expression Recognition Method

Electronics ◽  
2020 ◽  
Vol 9 (12) ◽  
pp. 2056
Author(s):  
Junjie Wu ◽  
Jianfeng Xu ◽  
Deyu Lin ◽  
Min Tu
Keyword(s):  
Optical Flow ◽  
Recognition Accuracy ◽  
Recognition Performance ◽  
Recognition Rate ◽  
Expression Recognition ◽  
Recognition Method ◽  
Video Clips ◽  
Low Degree ◽  
Micro Expression

The recognition accuracy of micro-expressions in the field of facial expressions is still understudied, as current research methods mainly focus on feature extraction and classification. Based on optical flow and decision thinking theory, we propose a novel micro-expression recognition method, which can filter low-quality micro-expression video clips. Determined by preset thresholds, we develop two optical flow filtering mechanisms: one based on two-branch decisions (OFF2BD) and the other based on three-way decisions (OFF3WD). In OFF2BD, which use the classical binary logic to classify images, and divide the images into positive or negative domain for further filtering. Differ from the OFF2BD, OFF3WD added boundary domain to delay to judge the motion quality of the images. In this way, the video clips with low degree of morphological change can be eliminated, so as to directly improve the quality of micro-expression features and recognition rate. From the experimental results, we verify the recognition accuracy of 61.57%, and 65.41% for CASMEII, and SMIC datasets, respectively. Through the comparative analysis, it shows that the scheme can effectively improve the recognition performance.

scholarly journals Cost-Effective CNNs for Real-Time Micro-Expression Recognition

2020 ◽  
Vol 10 (14) ◽  
pp. 4959
Author(s):  
Reda Belaiche ◽  
Yu Liu ◽  
Cyrille Migniot ◽  
Dominique Ginhac ◽  
Fan Yang
Keyword(s):  
Machine Learning ◽  
Computer Vision ◽  
Real Time ◽  
Video Clip ◽  
Cost Effective ◽  
Compact Representation ◽  
Expression Recognition ◽  
Memory Space ◽  
Computational Resources ◽  
Micro Expression

Micro-Expression (ME) recognition is a hot topic in computer vision as it presents a gateway to capture and understand daily human emotions. It is nonetheless a challenging problem due to ME typically being transient (lasting less than 200 ms) and subtle. Recent advances in machine learning enable new and effective methods to be adopted for solving diverse computer vision tasks. In particular, the use of deep learning techniques on large datasets outperforms classical approaches based on classical machine learning which rely on hand-crafted features. Even though available datasets for spontaneous ME are scarce and much smaller, using off-the-shelf Convolutional Neural Networks (CNNs) still demonstrates satisfactory classification results. However, these networks are intense in terms of memory consumption and computational resources. This poses great challenges when deploying CNN-based solutions in many applications, such as driver monitoring and comprehension recognition in virtual classrooms, which demand fast and accurate recognition. As these networks were initially designed for tasks of different domains, they are over-parameterized and need to be optimized for ME recognition. In this paper, we propose a new network based on the well-known ResNet18 which we optimized for ME classification in two ways. Firstly, we reduced the depth of the network by removing residual layers. Secondly, we introduced a more compact representation of optical flow used as input to the network. We present extensive experiments and demonstrate that the proposed network obtains accuracies comparable to the state-of-the-art methods while significantly reducing the necessary memory space. Our best classification accuracy was 60.17% on the challenging composite dataset containing five objectives classes. Our method takes only 24.6 ms for classifying a ME video clip (less than the occurrence time of the shortest ME which lasts 40 ms). Our CNN design is suitable for real-time embedded applications with limited memory and computing resources.

scholarly journals Study of DRMF and ASM facial landmark point for micro expression recognition using KLT tracking point feature

2019 ◽  
Vol 1402 ◽  
pp. 077039
Author(s):  
R A Asmara ◽  
P Choirina ◽  
C Rahmad ◽  
A Setiawan ◽  
F Rahutomo ◽  
...  
Keyword(s):  
Expression Recognition ◽  
Landmark Point ◽  
Facial Landmark ◽  
Tracking Point ◽  
Micro Expression ◽  
Point Feature

Invisible emotion magnification algorithm (IEMA) for real-time micro-expression recognition with graph-based features

2022 ◽  
Author(s):  
Adamu Muhammad Buhari ◽  
Chee-Pun Ooi ◽  
Vishnu Monn Baskaran ◽  
Raphael CW Phan ◽  
KokSheik Wong ◽  
...  
Keyword(s):  
Real Time ◽  
Expression Recognition ◽  
Micro Expression

scholarly journals Objective Classes for Micro-Facial Expression Recognition

2018 ◽  
Vol 4 (10) ◽  
pp. 119 ◽  
Author(s):  
Adrian Davison ◽  
Walied Merghani ◽  
Moi Yap
Keyword(s):  
Facial Expressions ◽  
Local Binary Patterns ◽  
Expression Recognition ◽  
Coding System ◽  
Histogram Of Oriented Gradient ◽  
Action Units ◽  
Self Reports ◽  
Casme Ii ◽  
Academy Of Sciences ◽  
Micro Expression

Micro-expressions are brief spontaneous facial expressions that appear on a face when a person conceals an emotion, making them different to normal facial expressions in subtlety and duration. Currently, emotion classes within the CASME II dataset (Chinese Academy of Sciences Micro-expression II) are based on Action Units and self-reports, creating conflicts during machine learning training. We will show that classifying expressions using Action Units, instead of predicted emotion, removes the potential bias of human reporting. The proposed classes are tested using LBP-TOP (Local Binary Patterns from Three Orthogonal Planes), HOOF (Histograms of Oriented Optical Flow) and HOG 3D (3D Histogram of Oriented Gradient) feature descriptors. The experiments are evaluated on two benchmark FACS (Facial Action Coding System) coded datasets: CASME II and SAMM (A Spontaneous Micro-Facial Movement). The best result achieves 86.35% accuracy when classifying the proposed 5 classes on CASME II using HOG 3D, outperforming the result of the state-of-the-art 5-class emotional-based classification in CASME II. Results indicate that classification based on Action Units provides an objective method to improve micro-expression recognition.

scholarly journals Necessary Morphological Patches Extraction for Automatic Micro-Expression Recognition

2018 ◽  
Vol 8 (10) ◽  
pp. 1811 ◽  
Author(s):  
Yue Zhao ◽  
Jiancheng Xu
Keyword(s):  
Computer Vision ◽  
Expression Recognition ◽  
Emotional States ◽  
Coding System ◽  
Entropy Weight ◽  
Facial Muscles ◽  
Rational Thought ◽  
Entropy Weight Method ◽  
Casme Ii ◽  
Micro Expression

Micro expressions are usually subtle and brief facial expressions that humans use to hide their true emotional states. In recent years, micro-expression recognition has attracted wide attention in the fields of psychology, mass media, and computer vision. The shortest micro expression lasts only 1/25 s. Furthermore, different from macro-expressions, micro-expressions have considerable low intensity and inadequate contraction of the facial muscles. Based on these characteristics, automatic micro-expression detection and recognition are great challenges in the field of computer vision. In this paper, we propose a novel automatic facial expression recognition framework based on necessary morphological patches (NMPs) to better detect and identify micro expressions. Micro expression is a subconscious facial muscle response. It is not controlled by the rational thought of the brain. Therefore, it calls on a few facial muscles and has local properties. NMPs are the facial regions that must be involved when a micro expression occurs. NMPs were screened based on weighting the facial active patches instead of the holistic utilization of the entire facial area. Firstly, we manually define the active facial patches according to the facial landmark coordinates and the facial action coding system (FACS). Secondly, we use a LBP-TOP descriptor to extract features in these patches and the Entropy-Weight method to select NMP. Finally, we obtain the weighted LBP-TOP features of these NMP. We test on two recent publicly available datasets: CASME II and SMIC database that provided sufficient samples. Compared with many recent state-of-the-art approaches, our method achieves more promising recognition results.

scholarly journals Real-time Facial Expression Recognition to Track Non-verbal Behaviors as Lie Indicators During Interview

2019 ◽  
Vol 1 (1) ◽  
pp. 25-31
Author(s):  
Arif Budi Setiawan ◽  
Kaspul Anwar ◽  
Laelatul Azizah ◽  
Adhi Prahara
Keyword(s):  
Facial Expression ◽  
Real Time ◽  
Facial Expression Recognition ◽  
Eye Gaze ◽  
Processing Method ◽  
Facial Features ◽  
Expression Recognition ◽  
Nonverbal Behaviors ◽  
Facial Landmark ◽  
Short Time

During interview, a psychologist should pay attention to every gesture and response, both verbal and nonverbal language/behaviors, made by the client. Psychologist certainly has limitation in recognizing every gesture and response that indicates a lie, especially in interpreting nonverbal behaviors that usually occurs in a short time. In this research, a real time facial expression recognition is proposed to track nonverbal behaviors to help psychologist keep informed about the change of facial expression that indicate a lie. The method tracks eye gaze, wrinkles on the forehead, and false smile using combination of face detection and facial landmark recognition to find the facial features and image processing method to track the nonverbal behaviors in facial features. Every nonverbal behavior is recorded and logged according to the video timeline to assist the psychologist analyze the behavior of the client. The result of tracking nonverbal behaviors of face is accurate and expected to be useful assistant for the psychologists.

scholarly journals Motion and Geometric Feature Analysis for Real-time Automatic Micro-expression Recognition Systems

F1000Research ◽  
2021 ◽  
Vol 10 ◽  
pp. 1029
Author(s):  
Adamu Muhammad Buhari ◽  
Chee-Pun Ooi ◽  
Vishnu Monn Baskaran ◽  
Wooi-Haw Tan
Keyword(s):  
Feature Extraction ◽  
Real Time ◽  
Recognition Performance ◽  
Detection Algorithm ◽  
Geometric Feature ◽  
Expression Recognition ◽  
Image Size ◽  
Recognition Systems ◽  
Full Face ◽  
Micro Expression

The trend of real-time micro-expression recognition systems has increased with recent advancements in human-computer interaction (HCI) in security and healthcare. Several studies in this field contributed towards recognition accuracy, while few studies look into addressing the computation costs. In this paper, two approaches for micro-expression feature extraction are analyzed for real-time automatic micro-expression recognition. Firstly, motion-based approach, which calculates motion of subtle changes from an image sequence and present as features. Then, secondly, a low computational geometric-based feature extraction technique, a very popular method for facial expression recognition in real-time. These approaches were integrated in a developed system together with a facial landmark detection algorithm and a classifier for real-time analysis. Moreover, the recognition performance were evaluated using SMIC, CASME, CAS(ME)2 and SAMM datasets. The results suggest that the optimized Bi-WOOF (leveraging on motion-based features) yields the highest accuracy of 68.5%, while the full-face graph (leveraging on geometric-based features) yields 75.53% on the SAMM dataset. On the other hand, the optimized Bi-WOOF processes sample at 0.36 seconds and full-face graph processes sample at 0.10 seconds with a 640x480 image size. All experiments were performed on an Intel i5-3470 machine.

Sign in / Sign up

Export Citation Format

Share Document