Illumination invariant facial expression recognition using selected merged binary patterns for real world images

Optik ◽  
2018 ◽  
Vol 158 ◽  
pp. 1016-1025 ◽  
Author(s):  
Asim Munir ◽  
Ayyaz Hussain ◽  
Sajid Ali Khan ◽  
Muhammad Nadeem ◽  
Sadia Arshid
Keyword(s):  
Facial Expression ◽  
Real World ◽  
Facial Expression Recognition ◽  
Expression Recognition ◽  
Illumination Invariant

scholarly journals Facial Expression Recognition Based on Weighted-Cluster Loss and Deep Transfer Learning Using a Highly Imbalanced Dataset

Sensors ◽  
2020 ◽  
Vol 20 (9) ◽  
pp. 2639
Author(s):  
Quan T. Ngo ◽  
Seokhoon Yoon
Keyword(s):  
Facial Expression ◽  
Transfer Learning ◽  
Loss Function ◽  
Real World ◽  
Facial Expression Recognition ◽  
Training Data ◽  
Fine Tuning ◽  
Expression Recognition ◽  
Recent Success ◽  
Deep Cnn

Facial expression recognition (FER) is a challenging problem in the fields of pattern recognition and computer vision. The recent success of convolutional neural networks (CNNs) in object detection and object segmentation tasks has shown promise in building an automatic deep CNN-based FER model. However, in real-world scenarios, performance degrades dramatically owing to the great diversity of factors unrelated to facial expressions, and due to a lack of training data and an intrinsic imbalance in the existing facial emotion datasets. To tackle these problems, this paper not only applies deep transfer learning techniques, but also proposes a novel loss function called weighted-cluster loss, which is used during the fine-tuning phase. Specifically, the weighted-cluster loss function simultaneously improves the intra-class compactness and the inter-class separability by learning a class center for each emotion class. It also takes the imbalance in a facial expression dataset into account by giving each emotion class a weight based on its proportion of the total number of images. In addition, a recent, successful deep CNN architecture, pre-trained in the task of face identification with the VGGFace2 database from the Visual Geometry Group at Oxford University, is employed and fine-tuned using the proposed loss function to recognize eight basic facial emotions from the AffectNet database of facial expression, valence, and arousal computing in the wild. Experiments on an AffectNet real-world facial dataset demonstrate that our method outperforms the baseline CNN models that use either weighted-softmax loss or center loss.

Facial expression recognition on real world face images using intelligent techniques: A survey

Optik ◽  
2016 ◽  
Vol 127 (15) ◽  
pp. 6195-6203 ◽  
Author(s):  
Sajid Ali Khan ◽  
Ayyaz Hussain ◽  
Muhammad Usman
Keyword(s):  
Facial Expression ◽  
Real World ◽  
Facial Expression Recognition ◽  
Expression Recognition ◽  
Face Images

scholarly journals Illumination Invariant Facial Expression Recognition using Convolutional Neural Networks

2019 ◽  
Vol 8 (4) ◽  
pp. 6140-6144
Keyword(s):  
Facial Expression ◽  
Visual Information ◽  
Facial Expression Recognition ◽  
Vital Role ◽  
Machine Learning Algorithms ◽  
Expression Recognition ◽  
Illumination Invariant ◽  
Novel Method ◽  
Machine Interface ◽  
Interface Modules

In this work, we propose a prospective novel method to address illumination invariant system for facial expression recognition. Facial expressions are used to convey nonverbal visual information among humans. This also plays a vital role in human-machine interface modules that have invoked attention of many researchers. Earlier machine learning algorithms require complex feature extraction algorithms and are relying on the size and uniqueness of features related to the subjects. In this paper, a deep convolutional neural network is proposed for facial expression recognition and it is trained on two publicly available datasets such as JAFFE and Yale databases under different illumination conditions. Furthermore, transfer learning is used with pre-trained networks such as AlexNet and ResNet-101 trained on ImageNet database. Experimental results show that the designed network could recognize up to 30% variation in the illumination and it achieves an accuracy of 92%.

scholarly journals A Review on Automatic Facial Expression Recognition Systems Assisted by Multimodal Sensor Data

Sensors ◽  
2019 ◽  
Vol 19 (8) ◽  
pp. 1863 ◽  
Author(s):  
Samadiani ◽  
Huang ◽  
Cai ◽  
Luo ◽  
Chi ◽  
...  
Keyword(s):  
Facial Expression ◽  
Emotional Expression ◽  
Real World ◽  
Facial Expression Recognition ◽  
Recognition System ◽  
Sensor Data ◽  
Expression Recognition ◽  
Illumination Variation ◽  
Interaction Detection ◽  
Extra Information

Facial Expression Recognition (FER) can be widely applied to various research areas, such as mental diseases diagnosis and human social/physiological interaction detection. With the emerging advanced technologies in hardware and sensors, FER systems have been developed to support real-world application scenes, instead of laboratory environments. Although the laboratory-controlled FER systems achieve very high accuracy, around 97%, the technical transferring from the laboratory to real-world applications faces a great barrier of very low accuracy, approximately 50%. In this survey, we comprehensively discuss three significant challenges in the unconstrained real-world environments, such as illumination variation, head pose, and subject-dependence, which may not be resolved by only analysing images/videos in the FER system. We focus on those sensors that may provide extra information and help the FER systems to detect emotion in both static images and video sequences. We introduce three categories of sensors that may help improve the accuracy and reliability of an expression recognition system by tackling the challenges mentioned above in pure image/video processing. The first group is detailed-face sensors, which detect a small dynamic change of a face component, such as eye-trackers, which may help differentiate the background noise and the feature of faces. The second is non-visual sensors, such as audio, depth, and EEG sensors, which provide extra information in addition to visual dimension and improve the recognition reliability for example in illumination variation and position shift situation. The last is target-focused sensors, such as infrared thermal sensors, which can facilitate the FER systems to filter useless visual contents and may help resist illumination variation. Also, we discuss the methods of fusing different inputs obtained from multimodal sensors in an emotion system. We comparatively review the most prominent multimodal emotional expression recognition approaches and point out their advantages and limitations. We briefly introduce the benchmark data sets related to FER systems for each category of sensors and extend our survey to the open challenges and issues. Meanwhile, we design a framework of an expression recognition system, which uses multimodal sensor data (provided by the three categories of sensors) to provide complete information about emotions to assist the pure face image/video analysis. We theoretically analyse the feasibility and achievability of our new expression recognition system, especially for the use in the wild environment, and point out the future directions to design an efficient, emotional expression recognition system.

Multi-stage binary patterns for facial expression recognition in real world

2017 ◽  
Vol 21 (1) ◽  
pp. 323-331 ◽  
Author(s):  
Sadia Arshid ◽  
Ayyaz Hussain ◽  
Asim Munir ◽  
Anum Nawaz ◽  
Sanneya Aziz
Keyword(s):  
Facial Expression ◽  
Real World ◽  
Facial Expression Recognition ◽  
Expression Recognition ◽  
Multi Stage

scholarly journals Facial Expression Recognition of Nonlinear Facial Variations Using Deep Locality De-Expression Residue Learning in the Wild

Electronics ◽  
2019 ◽  
Vol 8 (12) ◽  
pp. 1487 ◽  
Author(s):  
Asad Ullah ◽  
Jing Wang ◽  
M. Shahid Anwar ◽  
Usman Ahmad ◽  
Uzair Saeed ◽  
...  
Keyword(s):  
Facial Expression ◽  
Real World ◽  
Facial Expression Recognition ◽  
Eye Gaze ◽  
Input Image ◽  
Training Data ◽  
Head Pose Estimation ◽  
Expression Recognition ◽  
Action Unit ◽  
In The Wild

Automatic facial expression recognition is an emerging field. Moreover, the interest has been increased with the transition from laboratory-controlled conditions to in the wild scenarios. Most of the research has been done over nonoccluded faces under the constrained environment, while automatic facial expression is less understood/implemented for partial occlusion in the real world conditions. Apart from that, our research aims to tackle the issues of overfitting (caused by the shortage of adequate training data) and to alleviate the expression-unrelated/intraclass/nonlinear facial variations, such as head pose estimation, eye gaze estimation, intensity and microexpressions. In our research, we control the magnitude of each Action Unit (AU) and combine several of the Action Unit combinations to leverage learning from the generative and discriminative representations for automatic FER. We have also addressed the problem of diversification of expressions from lab controlled to real-world scenarios from our cross-database study and proposed a model for enhancement of the discriminative power of deep features while increasing the interclass scatters, by preserving the locality closeness. Furthermore, facial expression consists of an expressive component as well as neutral component, so we proposed a generative model which is capable of generating neutral expression from an input image using cGAN. The expressive component is filtered and passed to the intermediate layers and the process is called De-expression Residue Learning. The residue in the intermediate/middle layers is very important for learning through expressive components. Finally, we validate the effectiveness of our method (DLP-DeRL) through qualitative and quantitative experimental results using four databases. Our method is more accurate and robust, and outperforms all the existing methods (hand crafted features and deep learning) while dealing the images in the wild.

Sign in / Sign up

Export Citation Format

Share Document