Spatiotemporal Convolutional Neural Network with Convolutional Block Attention Module for Micro-Expression Recognition

A micro-expression is defined as an uncontrollable muscular movement shown on the face of humans when one is trying to conceal or repress his true emotions. Many researchers have applied the deep learning framework to micro-expression recognition in recent years. However, few have introduced the human visual attention mechanism to micro-expression recognition. In this study, we propose a three-dimensional (3D) spatiotemporal convolutional neural network with the convolutional block attention module (CBAM) for micro-expression recognition. First image sequences were input to a medium-sized convolutional neural network (CNN) to extract visual features. Afterwards, it learned to allocate the feature weights in an adaptive manner with the help of a convolutional block attention module. The method was testified in spontaneous micro-expression databases (Chinese Academy of Sciences Micro-expression II (CASME II), Spontaneous Micro-expression Database (SMIC)). The experimental results show that the 3D CNN with convolutional block attention module outperformed other algorithms in micro-expression recognition.

Download Full-text

Micro-expression recognition based on 3D flow convolutional neural network

Pattern Analysis and Applications ◽

10.1007/s10044-018-0757-5 ◽

2018 ◽

Vol 22 (4) ◽

pp. 1331-1339 ◽

Cited By ~ 18

Author(s):

Jing Li ◽

Yandan Wang ◽

John See ◽

Wenbin Liu

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Expression Recognition ◽

3D Flow ◽

Micro Expression

Download Full-text

Three-dimensional convolutional neural network (3D-CNN) for heterogeneous material homogenization

Computational Materials Science ◽

10.1016/j.commatsci.2020.109850 ◽

2020 ◽

Vol 184 ◽

pp. 109850 ◽

Cited By ~ 1

Author(s):

Chengping Rao ◽

Yang Liu

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Three Dimensional ◽

Heterogeneous Material ◽

3D Cnn

Download Full-text

Dual Temporal Scale Convolutional Neural Network for Micro-Expression Recognition

Frontiers in Psychology ◽

10.3389/fpsyg.2017.01745 ◽

2017 ◽

Vol 8 ◽

Cited By ~ 40

Author(s):

Min Peng ◽

Chongyang Wang ◽

Tong Chen ◽

Guangyuan Liu ◽

Xiaolan Fu

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Temporal Scale ◽

Expression Recognition ◽

Micro Expression

Download Full-text

Facial Expression Recognition using CNN

International Journal of Scientific Research in Computer Science Engineering and Information Technology ◽

10.32628/cseit206248 ◽

2020 ◽

pp. 179-183

Author(s):

S. Mary Hima Preethi ◽

P. Sobha ◽

P. Rajalakshmi Kamalini ◽

K. Gowri Raghavendra Narayan

Keyword(s):

Neural Network ◽

Facial Expression ◽

Convolutional Neural Network ◽

Facial Expression Recognition ◽

Confusion Matrix ◽

Neural System ◽

Expression Recognition ◽

The Face ◽

Human Faces ◽

The Ideal

People have consistently been able to perceive and recognize faces and their feelings. Presently PCs can do likewise. We propose a model which recognizes human faces and classifies the emotion on the face as happy, angry, sad, neutral, surprise, disgust or fear. It is developed utilizing a convolutional neural network(CNN) and involves various stages. All these are carried out using a dataset available on the Kaggle repository named fer2013. Precision and execution of the neural system can be assessed utilizing a confusion matrix. We applied cross-approval to decide the ideal hyper-parameters and assessed the presentation of the created models by looking at their training histories.

Download Full-text

A Novel Approach on Motion Estimation for Micro-Expression Recognition Using Phase Only Correlation with All Block Search (POC-ABS)

International Journal of Intelligent Engineering and Systems ◽

10.22266/ijies2020.1231.48 ◽

2020 ◽

Vol 13 (6) ◽

pp. 546-559

Author(s):

Ulla Rosiani ◽

◽

Priska Choirina ◽

Niyalatul Muna ◽

Eko Mulyanto ◽

...

Keyword(s):

Motion Vector ◽

Block Matching ◽

Expression Recognition ◽

Motion Feature ◽

Novel Approach ◽

The Face ◽

Casme Ii ◽

The Right ◽

Performance Results ◽

Micro Expression

Micro-expression is an expression when a person tries to held or hidden, but the leak of this emotion still occurs in one or two areas of the face or maybe a short expression that across in the whole-face. Not more than 500ms, micro-expressions can be difficult to recognize and detect where the leakage area is located. This study presents a new method to recognize and detect the subtle motion on the facial components area using Phase Only Correlation algorithm with All Block Search (POC-ABS) to estimate the motion of all block areas. This block matching method is proposed by comparing each block in the two frames to determine whether there is movement or not. If the two blocks are identical, then the motion vector value is not displayed, whereas if the blocks are non-identical, the motion vector value of the POC is displayed. The motion vector, which is as a motion feature, estimates whether or not there are movements in the same block. In order to further confirm the reliability of the proposed method, two different classifiers were used for the micro-expression recognition of the CASME II dataset. The highest performance results are for SVM at 94.3 percent and for KNN at 95.6 percent. Finally, this algorithm detects leaks of motion based on the ratio of the motion vectors. The left and right eyebrows are dominant when expressing disgust, sadness, and surprise. Meanwhile, the movements of the right eye and left eye were the most dominant when the happiness expression.

Download Full-text

Three-Dimensional Convolutional Neural Network Model for Early Detection of Pine Wilt Disease Using UAV-Based Hyperspectral Images

Remote Sensing ◽

10.3390/rs13204065 ◽

2021 ◽

Vol 13 (20) ◽

pp. 4065

Author(s):

Run Yu ◽

Youqing Luo ◽

Haonan Li ◽

Liyuan Yang ◽

Huaguo Huang ◽

...

Keyword(s):

Neural Network ◽

Early Detection ◽

Convolutional Neural Network ◽

Three Dimensional ◽

Pine Wilt Disease ◽

Wilt Disease ◽

Hyperspectral Images ◽

Pine Wilt ◽

Pine Trees ◽

3D Cnn

As one of the most devastating disasters to pine forests, pine wilt disease (PWD) has caused tremendous ecological and economic losses in China. An effective way to prevent large-scale PWD outbreaks is to detect and remove the damaged pine trees at the early stage of PWD infection. However, early infected pine trees do not show obvious changes in morphology or color in the visible wavelength range, making early detection of PWD tricky. Unmanned aerial vehicle (UAV)-based hyperspectral imagery (HI) has great potential for early detection of PWD. However, the commonly used methods, such as the two-dimensional convolutional neural network (2D-CNN), fail to simultaneously extract and fully utilize the spatial and spectral information, whereas the three-dimensional convolutional neural network (3D-CNN) is able to collect this information from raw hyperspectral data. In this paper, we applied the residual block to 3D-CNN and constructed a 3D-Res CNN model, the performance of which was then compared with that of 3D-CNN, 2D-CNN, and 2D-Res CNN in identifying PWD-infected pine trees from the hyperspectral images. The 3D-Res CNN model outperformed the other models, achieving an overall accuracy (OA) of 88.11% and an accuracy of 72.86% for detecting early infected pine trees (EIPs). Using only 20% of the training samples, the OA and EIP accuracy of 3D-Res CNN can still achieve 81.06% and 51.97%, which is superior to the state-of-the-art method in the early detection of PWD based on hyperspectral images. Collectively, 3D-Res CNN was more accurate and effective in early detection of PWD. In conclusion, 3D-Res CNN is proposed for early detection of PWD in this paper, making the prediction and control of PWD more accurate and effective. This model can also be applied to detect pine trees damaged by other diseases or insect pests in the forest.

Download Full-text