Emotion Training: Image Color Transfer with Facial Expression and Emotion Recognition

With the development of intelligent automotive human-machine systems, driver emotion detection and recognition has become an emerging research topic. Facial expression-based emotion recognition approaches have achieved outstanding results on laboratory-controlled data. However, these studies cannot represent the environment of real driving situations. In order to address this, this paper proposes a facial expression-based on-road driver emotion recognition network called FERDERnet. This method divides the on-road driver facial expression recognition task into three modules: a face detection module that detects the driver’s face, an augmentation-based resampling module that performs data augmentation and resampling, and an emotion recognition module that adopts a deep convolutional neural network pre-trained on FER and CK+ datasets and then fine-tuned as a backbone for driver emotion recognition. This method adopts five different backbone networks as well as an ensemble method. Furthermore, to evaluate the proposed method, this paper collected an on-road driver facial expression dataset, which contains various road scenarios and the corresponding driver’s facial expression during the driving task. Experiments were performed on the on-road driver facial expression dataset that this paper collected. Based on efficiency and accuracy, the proposed FERDERnet with Xception backbone was effective in identifying on-road driver facial expressions and obtained superior performance compared to the baseline networks and some state-of-the-art networks.

Download Full-text

Dual-Modal Emotion Recognition Based on Facial Expression and Body Posture in Video Sequences

Laser & Optoelectronics Progress ◽

10.3788/lop55.071004 ◽

2018 ◽

Vol 55 (7) ◽

pp. 071004 ◽

Cited By ~ 1

Author(s):

姜明星 Jiang Mingxing ◽

胡敏 Hu Min ◽

王晓华 Wang Xiaohua ◽

任福继 Ren Fuji ◽

王浩文 Wang Haowen

Keyword(s):

Facial Expression ◽

Emotion Recognition ◽

Body Posture ◽

Video Sequences

Download Full-text

Real Time Facial Expression and Emotion Recognition using Eigen faces, LBPH and Fisher Algorithms

2020 10th International Conference on Cloud Computing, Data Science & Engineering (Confluence) ◽

10.1109/confluence47617.2020.9057985 ◽

2020 ◽

Cited By ~ 1

Author(s):

Shrayan Mukhopadhyay ◽

Shilpi Sharma

Keyword(s):

Facial Expression ◽

Emotion Recognition ◽

Real Time

Download Full-text

eXnet: An Efficient Approach for Emotion Recognition in the Wild

Sensors ◽

10.3390/s20041087 ◽

2020 ◽

Vol 20 (4) ◽

pp. 1087

Author(s):

Muhammad Naveed Riaz ◽

Yao Shen ◽

Muhammad Sohail ◽

Minyi Guo

Keyword(s):

Facial Expression ◽

Emotion Recognition ◽

Facial Expression Recognition ◽

Data Augmentation ◽

Raspberry Pi ◽

Expression Recognition ◽

Extensive Evaluation ◽

Benchmark Datasets ◽

Augmentation Techniques ◽

In The Wild

Facial expression recognition has been well studied for its great importance in the areas of human–computer interaction and social sciences. With the evolution of deep learning, there have been significant advances in this area that also surpass human-level accuracy. Although these methods have achieved good accuracy, they are still suffering from two constraints (high computational power and memory), which are incredibly critical for small hardware-constrained devices. To alleviate this issue, we propose a new Convolutional Neural Network (CNN) architecture eXnet (Expression Net) based on parallel feature extraction which surpasses current methods in accuracy and contains a much smaller number of parameters (eXnet: 4.57 million, VGG19: 14.72 million), making it more efficient and lightweight for real-time systems. Several modern data augmentation techniques are applied for generalization of eXnet; these techniques improve the accuracy of the network by overcoming the problem of overfitting while containing the same size. We provide an extensive evaluation of our network against key methods on Facial Expression Recognition 2013 (FER-2013), Extended Cohn-Kanade Dataset (CK+), and Real-world Affective Faces Database (RAF-DB) benchmark datasets. We also perform ablation evaluation to show the importance of different components of our architecture. To evaluate the efficiency of eXnet on embedded systems, we deploy it on Raspberry Pi 4B. All these evaluations show the superiority of eXnet for emotion recognition in the wild in terms of accuracy, the number of parameters, and size on disk.

Download Full-text