scholarly journals EMPATH: A Neural Network that Categorizes Facial Expressions

2002 ◽  
Vol 14 (8) ◽  
pp. 1158-1173 ◽  
Author(s):  
Matthew N. Dailey ◽  
Garrison W. Cottrell ◽  
Curtis Padgett ◽  
Ralph Adolphs
Keyword(s):  
Neural Network ◽  
Facial Expression ◽  
Facial Expressions ◽  
Categorical Perception ◽  
Dimensional Space ◽  
Parameter Tuning ◽  
Expression Recognition ◽  
Psychological Phenomena ◽  
Expression Perception ◽  
Facial Expression Perception

There are two competing theories of facial expression recognition. Some researchers have suggested that it is an example of “categorical perception.” In this view, expression categories are considered to be discrete entities with sharp boundaries, and discrimination of nearby pairs of expressive faces is enhanced near those boundaries. Other researchers, however, suggest that facial expression perception is more graded and that facial expressions are best thought of as points in a continuous, low-dimensional space, where, for instance, “surprise” expressions lie between “happiness” and “fear” expressions due to their perceptual similarity. In this article, we show that a simple yet biologically plausible neural network model, trained to classify facial expressions into six basic emotions, predicts data used to support both of these theories. Without any parameter tuning, the model matches a variety of psychological data on categorization, similarity, reaction times, discrimination, and recognition difficulty, both qualitatively and quantitatively. We thus explain many of the seemingly complex psychological phenomena related to facial expression perception as natural consequences of the tasks' implementations in the brain.

scholarly journals Facial Expression Recognition of Instructor Using Deep Features and Extreme Learning Machine

2021 ◽  
Vol 2021 ◽  
pp. 1-17
Author(s):  
Yusra Khalid Bhatti ◽  
Afshan Jamil ◽  
Nudrat Nida ◽  
Muhammad Haroon Yousaf ◽  
Serestina Viriri ◽  
...  
Keyword(s):  
Facial Expression ◽  
Extreme Learning Machine ◽  
Facial Expressions ◽  
Classroom Environment ◽  
Facial Expression Recognition ◽  
Parameter Tuning ◽  
Expression Recognition ◽  
Fast Learning ◽  
Learning Machine ◽  
The Impact

Classroom communication involves teacher’s behavior and student’s responses. Extensive research has been done on the analysis of student’s facial expressions, but the impact of instructor’s facial expressions is yet an unexplored area of research. Facial expression recognition has the potential to predict the impact of teacher’s emotions in a classroom environment. Intelligent assessment of instructor behavior during lecture delivery not only might improve the learning environment but also could save time and resources utilized in manual assessment strategies. To address the issue of manual assessment, we propose an instructor’s facial expression recognition approach within a classroom using a feedforward learning model. First, the face is detected from the acquired lecture videos and key frames are selected, discarding all the redundant frames for effective high-level feature extraction. Then, deep features are extracted using multiple convolution neural networks along with parameter tuning which are then fed to a classifier. For fast learning and good generalization of the algorithm, a regularized extreme learning machine (RELM) classifier is employed which classifies five different expressions of the instructor within the classroom. Experiments are conducted on a newly created instructor’s facial expression dataset in classroom environments plus three benchmark facial datasets, i.e., Cohn–Kanade, the Japanese Female Facial Expression (JAFFE) dataset, and the Facial Expression Recognition 2013 (FER2013) dataset. Furthermore, the proposed method is compared with state-of-the-art techniques, traditional classifiers, and convolutional neural models. Experimentation results indicate significant performance gain on parameters such as accuracy, F1-score, and recall.

scholarly journals Blurred Facial Expression Recognition System by Using Convolution Neural Network

Webology ◽  
2020 ◽  
Vol 17 (2) ◽  
pp. 804-816
Author(s):  
Elaf J. Al Taee ◽  
Qasim Mohammed Jasim
Keyword(s):  
Neural Network ◽  
Facial Expression ◽  
Facial Expressions ◽  
Facial Expression Recognition ◽  
Cognitive Activity ◽  
Recognition System ◽  
Vital Role ◽  
Convolution Neural Network ◽  
Expression Recognition ◽  
Recognition Of Facial Expressions

A facial expression is a visual impression of a person's situations, emotions, cognitive activity, personality, intention and psychopathology, it has an active and vital role in the exchange of information and communication between people. In machines and robots which dedicated to communication with humans, the facial expressions recognition play an important and vital role in communication and reading of what is the person implies, especially in the field of health. For that the research in this field leads to development in communication with the robot. This topic has been discussed extensively, and with the progress of deep learning and use Convolution Neural Network CNN in image processing which widely proved efficiency, led to use CNN in the recognition of facial expressions. Automatic system for Facial Expression Recognition FER require to perform detection and location of faces in a cluttered scene, feature extraction, and classification. In this research, the CNN used for perform the process of FER. The target is to label each image of facial into one of the seven facial emotion categories considered in the JAFFE database. JAFFE facial expression database with seven facial expression labels as sad, happy, fear, surprise, anger, disgust, and natural are used in this research. We trained CNN with different depths using gray-scale images from the JAFFE database.The accuracy of proposed system was 100%.

scholarly journals Using a Social Robot to Evaluate Facial Expressions in the Wild

Sensors ◽  
2020 ◽  
Vol 20 (23) ◽  
pp. 6716
Author(s):  
Silvia Ramis ◽  
Jose Maria Buades ◽  
Francisco J. Perales
Keyword(s):  
Neural Network ◽  
Facial Expression ◽  
Facial Expressions ◽  
Affective Computing ◽  
Social Robot ◽  
Human Robot Interaction ◽  
Expression Recognition ◽  
Designed Experiment ◽  
In The Wild ◽  
Trained Neural Network

In this work an affective computing approach is used to study the human-robot interaction using a social robot to validate facial expressions in the wild. Our global goal is to evaluate that a social robot can be used to interact in a convincing manner with human users to recognize their potential emotions through facial expressions, contextual cues and bio-signals. In particular, this work is focused on analyzing facial expression. A social robot is used to validate a pre-trained convolutional neural network (CNN) which recognizes facial expressions. Facial expression recognition plays an important role in recognizing and understanding human emotion by robots. Robots equipped with expression recognition capabilities can also be a useful tool to get feedback from the users. The designed experiment allows evaluating a trained neural network in facial expressions using a social robot in a real environment. In this paper a comparison between the CNN accuracy and human experts is performed, in addition to analyze the interaction, attention and difficulty to perform a particular expression by 29 non-expert users. In the experiment, the robot leads the users to perform different facial expressions in motivating and entertaining way. At the end of the experiment, the users are quizzed about their experience with the robot. Finally, a set of experts and the CNN classify the expressions. The obtained results allow affirming that the use of social robot is an adequate interaction paradigm for the evaluation on facial expression.

scholarly journals Facial Expression Recognition By Using Fisherface Methode With Backpropagation Neural Network

2011 ◽  
Vol 5 (1) ◽  
Author(s):  
Zaenal Abidin ◽  
Agus Harjoko
Keyword(s):  
Neural Network ◽  
Facial Expression ◽  
Facial Expressions ◽  
False Positive ◽  
Facial Expression Recognition ◽  
Type I ◽  
Expression Recognition ◽  
Backpropagation Neural Network ◽  
Neural Network Approach ◽  
Image Type

Abstract— In daily lives, especially in interpersonal communication, face often used for expression. Facial expressions give information about the emotional state of the person. A facial expression is one of the behavioral characteristics. The components of a basic facial expression analysis system are face detection, face data extraction, and facial expression recognition. Fisherface method with backpropagation artificial neural network approach can be used for facial expression recognition. This method consists of two-stage process, namely PCA and LDA. PCA is used to reduce the dimension, while the LDA is used for features extraction of facial expressions. The system was tested with 2 databases namely JAFFE database and MUG database. The system correctly classified the expression with accuracy of 86.85%, and false positive 25 for image type I of JAFFE, for image type II of JAFFE 89.20% and false positive 15,  for type III of JAFFE 87.79%, and false positive for 16. The image of MUG are 98.09%, and false positive 5.Keywords— facial expression, fisherface method, PCA, LDA, backpropagation neural network.

Finding Useful Features for Facial Expression Recognition and Intensity Estimation by Neural Network

2020 ◽  
Vol 8 (2) ◽  
pp. 68-84
Author(s):  
Naoki Imamura ◽  
Hiroki Nomiya ◽  
Teruhisa Hochin
Keyword(s):  
Neural Network ◽  
Facial Expression ◽  
Facial Expressions ◽  
Facial Expression Recognition ◽  
Facial Features ◽  
Expression Recognition ◽  
Intensity Estimation ◽  
Expression Intensity ◽  
The Neural Network

Facial expression intensity has been proposed to digitize the degree of facial expressions in order to retrieve impressive scenes from lifelog videos. The intensity is calculated based on the correlation of facial features compared to each facial expression. However, the correlation is not determined objectively. It should be determined statistically based on the contribution score of the facial features necessary for expression recognition. Therefore, the proposed method recognizes facial expressions by using a neural network and calculates the contribution score of input toward the output. First, the authors improve some facial features. After that, they verify the score correctly by comparing the accuracy transitions depending on reducing useful and useless features and process the score statistically. As a result, they extract useful facial features from the neural network.

scholarly journals Facial Expression Recognition Based on Random Forest and Convolutional Neural Network

Information ◽  
2019 ◽  
Vol 10 (12) ◽  
pp. 375 ◽  
Author(s):  
Yingying Wang ◽  
Yibin Li ◽  
Yong Song ◽  
Xuewen Rong
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Random Forest ◽  
Facial Expression ◽  
Convolutional Neural Network ◽  
Facial Expressions ◽  
Facial Expression Recognition ◽  
Recognition Rate ◽  
Great Success ◽  
Expression Recognition

As an important part of emotion research, facial expression recognition is a necessary requirement in human–machine interface. Generally, a face expression recognition system includes face detection, feature extraction, and feature classification. Although great success has been made by the traditional machine learning methods, most of them have complex computational problems and lack the ability to extract comprehensive and abstract features. Deep learning-based methods can realize a higher recognition rate for facial expressions, but a large number of training samples and tuning parameters are needed, and the hardware requirement is very high. For the above problems, this paper proposes a method combining features that extracted by the convolutional neural network (CNN) with the C4.5 classifier to recognize facial expressions, which not only can address the incompleteness of handcrafted features but also can avoid the high hardware configuration in the deep learning model. Considering some problems of overfitting and weak generalization ability of the single classifier, random forest is applied in this paper. Meanwhile, this paper makes some improvements for C4.5 classifier and the traditional random forest in the process of experiments. A large number of experiments have proved the effectiveness and feasibility of the proposed method.

scholarly journals Emerged human-like facial expression representation in a deep convolutional neural network

2021 ◽  
Author(s):  
Liqin Zhou ◽  
Ming Meng ◽  
Ke Zhou
Keyword(s):  
Facial Expression ◽  
Visual Experience ◽  
Categorical Perception ◽  
Deep Convolutional Neural Networks ◽  
Face Identity ◽  
Domain Specific ◽  
Facial Identity ◽  
Specific Experience ◽  
Expression Perception ◽  
Facial Expression Perception

Face identity and expression play critical roles in social communication. Recent research found that the deep convolutional neural networks (DCNNs) trained to recognize facial identities spontaneously learn features that support facial expression recognition, and vice versa, suggesting an integrated representation of facial identity and expression. In the present study, we found that the expression-selective units spontaneously emerged in a VGG-Face trained for facial identity recognition and tuned to distinct basic expressions. Importantly, they exhibited typical hallmarks of human expression perception, i.e., the facial expression confusion effect and categorical perception effect. We then investigated whether the emergence of expression-selective units is attributed to either face-specific experience or domain-general processing, by carrying out the same analysis on a VGG-16 trained for object classification and an untrained VGG-Face without any visual experience, both of them having the identical architecture with the pretrained VGG-Face. Although Similar expression-selective units were found in both DCNNs, they did not exhibit reliable human-like characteristics of facial expression perception. Taken together, our computational findings revealed the necessity of domain-specific visual experience of face identity for the development of facial expression perception, highlighting the contribution of nurture to form human-like facial expression perception. Beyond the weak equivalence between human and DCNNS at the input-output behavior, emerging simulated algorithms between models and humans could be established through domain-specific experience.

Finding Facial Expression Patterns on Videos based on Smile and Eyes-Open Confidence Values

2021 ◽  
Vol 12 (5) ◽  
pp. 27-36
Author(s):  
Setiawan Hadi ◽  
Asep K Supriatna ◽  
Faishal Wahiduddin ◽  
Wilis Srisayekti ◽  
Achmad Djunaidi ◽  
...  
Keyword(s):  
Facial Expression ◽  
Facial Expressions ◽  
Dimensional Space ◽  
Expression Patterns ◽  
Detection Methods ◽  
Expression Recognition ◽  
Eyes Open ◽  
Heuristic Clustering ◽  
Parameter Values ◽  
Two Parameter

Facial expression recognition is one of the types of non-verbal communication that is not only commons for human but also plays an essential role in everyday lives. The development of science and technology allows the machine to automatically detect human facial expressions based on images and videos. Numerous facial expression detection methods have been proposed in the literature. This paper presents a method to find three basic facial expressions (neutral, happy, and angry) from two parameter values: smile and eyes-open. The analysis involves a preprocessing step using a combination of pre-designed proprietary algorithm and Luxand library. Firstly, the parameters were mapped into two-dimensional space and then grouped into three clusters using K-means, a popular heuristic clustering method. Secondly, more than 50,000 frames for each video were experimented using the proprietary research data. The result shows that the proposed method successfully performed a simple video analysis of facial expressions.

scholarly journals Facial Expression Recognition Based on Deep Learning Convolution Neural Network: A Review

2021 ◽  
Vol 2 (1) ◽  
Author(s):  
Sharmeen M. Saleem Abdullah ◽  
◽  
Adnan Mohsin Abdulazeez ◽  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Facial Expression ◽  
Facial Expressions ◽  
Video Game ◽  
Facial Expression Recognition ◽  
Emotional Processing ◽  
Convolution Neural Network ◽  
Data Sets ◽  
Expression Recognition

Facial emotional processing is one of the most important activities in effective calculations, engagement with people and computers, machine vision, video game testing, and consumer research. Facial expressions are a form of nonverbal communication, as they reveal a person's inner feelings and emotions. Extensive attention to Facial Expression Recognition (FER) has recently been received as facial expressions are considered. As the fastest communication medium of any kind of information. Facial expression recognition gives a better understanding of a person's thoughts or views and analyzes them with the currently trending deep learning methods. Accuracy rate sharply compared to traditional state-of-the-art systems. This article provides a brief overview of the different FER fields of application and publicly accessible databases used in FER and studies the latest and current reviews in FER using Convolution Neural Network (CNN) algorithms. Finally, it is observed that everyone reached good results, especially in terms of accuracy, with different rates, and using different data sets, which impacts the results.

Sign in / Sign up

Export Citation Format

Share Document