Automatic Coding of Facial Expressions of Pain: Are We There Yet?

Introduction. The experience of pain is regularly accompanied by facial expressions. The gold standard for analyzing these facial expressions is the Facial Action Coding System (FACS), which provides so-called action units (AUs) as parametrical indicators of facial muscular activity. Particular combinations of AUs have appeared to be pain-indicative. The manual coding of AUs is, however, too time- and labor-intensive in clinical practice. New developments in automatic facial expression analysis have promised to enable automatic detection of AUs, which might be used for pain detection. Objective. Our aim is to compare manual with automatic AU coding of facial expressions of pain. Methods. FaceReader7 was used for automatic AU detection. We compared the performance of FaceReader7 using videos of 40 participants (20 younger with a mean age of 25.7 years and 20 older with a mean age of 52.1 years) undergoing experimentally induced heat pain to manually coded AUs as gold standard labeling. Percentages of correctly and falsely classified AUs were calculated, and we computed as indicators of congruency, “sensitivity/recall,” “precision,” and “overall agreement (F1).” Results. The automatic coding of AUs only showed poor to moderate outcomes regarding sensitivity/recall, precision, and F1. The congruency was better for younger compared to older faces and was better for pain-indicative AUs compared to other AUs. Conclusion. At the moment, automatic analyses of genuine facial expressions of pain may qualify at best as semiautomatic systems, which require further validation by human observers before they can be used to validly assess facial expressions of pain.

If You're Happy, I'm Happy

An increasing body of research has addressed what a tourism experience is and how it should best be measured and managed. One conclusion has been to recommend observational methods such as facial expression analysis. The chapter uses facial expression analysis to determine whether the emotions of employees in the tourism industry affect the emotions of their customers, following a pattern of emotional contagion. The findings show that emotional valence and arousal are both contagious. Furthermore, the findings show that arousal is less contagious at a higher likelihood to recommend, likely due to higher employee arousal during approximately the middle third of their conversation. Furthermore, findings demonstrate that emotion measurement is now possible at reasonable convenience for the tourism industry and gives a unique insight into tourists' actual experiences that is more precise and valid than self-report alone, though with certain costs and stringent methodological limitations.

Creation and validation of the Picture-Set of Young Children’s Affective Facial Expressions (PSYCAFE)

The immediate detection and correct processing of affective facial expressions are one of the most important competences in social interaction and thus a main subject in emotion and affect research. Generally, studies in these research domains, use pictures of adults who display affective facial expressions as experimental stimuli. However, for studies investigating developmental psychology and attachment behaviour it is necessary to use age-matched stimuli, where it is children that display affective expressions. PSYCAFE represents a newly developed picture-set of children’s faces. It includes reference portraits of girls and boys aged 4 to 6 years averaged digitally from different individual pictures, that were categorized to six basic affects (fear, disgust, happiness, sadness, anger and surprise) plus a neutral facial expression by cluster analysis. This procedure led to deindividualized and affect prototypical portraits. Individual affect expressive portraits of adults from an already validated picture-set (KDEF) were used in a similar way to create affect prototypical images also of adults. The stimulus set has been validated on human observers and entail emotion recognition accuracy rates and scores for intensity, authenticity and likeability ratings of the specific affect displayed. Moreover, the stimuli have also been characterized by the iMotions Facial Expression Analysis Module, providing additional data on probability values representing the likelihood that the stimuli depict the expected affect. Finally, the validation data from human observers and iMotions are compared to data on facial mimicry of healthy adults in response to these portraits, measured by facial EMG (m. zygomaticus major and m. corrugator supercilii).

Data-Driven Facial Expression Analysis from Live Video

<p>Emotion analytics is the study of human behavior by analyzing the responses when humans experience different emotions. In this thesis, we research into emotion analytics solutions using computer vision to detect emotions from facial expressions automatically using live video. Considering anxiety is an emotion that can lead to more serious conditions like anxiety disorders and depression, we propose 2 hypotheses to detect anxiety from facial expressions. One hypothesis is that the complex emotion “anxiety” is a subset of the basic emotion “fear”. The other hypothesis is that anxiety can be distinguished from fear by differences in head and eye motion. We test the first hypothesis by implementing a basic emotions detector based on facial action coding system (FACS) to detect fear from videos of anxious faces. When we discover that this is not as accurate as we would like, an alternative solution based on Gabor filters is implemented. A comparison is done between the solutions and the Gabor-based solution is found to be inferior. The second hypothesis is tested by using scatter graphs and statistical analysis of the head and eye motions of videos for fear and anxiety expressions. It is found that head pitch has significant differences between fear and anxiety. As a conclusion to the thesis, we implement a systems software using the basic emotions detector based on FACS and evaluate the software by comparing commercials using emotions detected from facial expressions of viewers.</p>

Data-Driven Facial Expression Analysis from Live Video

<p>Emotion analytics is the study of human behavior by analyzing the responses when humans experience different emotions. In this thesis, we research into emotion analytics solutions using computer vision to detect emotions from facial expressions automatically using live video. Considering anxiety is an emotion that can lead to more serious conditions like anxiety disorders and depression, we propose 2 hypotheses to detect anxiety from facial expressions. One hypothesis is that the complex emotion “anxiety” is a subset of the basic emotion “fear”. The other hypothesis is that anxiety can be distinguished from fear by differences in head and eye motion. We test the first hypothesis by implementing a basic emotions detector based on facial action coding system (FACS) to detect fear from videos of anxious faces. When we discover that this is not as accurate as we would like, an alternative solution based on Gabor filters is implemented. A comparison is done between the solutions and the Gabor-based solution is found to be inferior. The second hypothesis is tested by using scatter graphs and statistical analysis of the head and eye motions of videos for fear and anxiety expressions. It is found that head pitch has significant differences between fear and anxiety. As a conclusion to the thesis, we implement a systems software using the basic emotions detector based on FACS and evaluate the software by comparing commercials using emotions detected from facial expressions of viewers.</p>

A Thermal Blended Facial Expression Analysis and Recognition System Using Deformed Thermal Facial Areas

There are many research works in visible as well as thermal facial expression analysis and recognition. Several facial expression databases have been designed in both modalities. However, little attention has been given for analyzing blended facial expressions in the thermal infrared spectrum. In this paper, we have introduced a Visual-Thermal Blended Facial Expression Database (VTBE) that contains visual and thermal face images with both basic and blended facial expressions. The database contains 12 posed blended facial expressions and spontaneous six basic facial expressions in both modalities. In this paper, we have proposed Deformed Thermal Facial Area (DTFA) in thermal expressive face image and make an analysis to differentiate between basic and blended expressions using DTFA. Here, the fusion of DTFA and Deformed Visual Facial Area (DVFA) has been proposed combining the features of both modalities and experiments and has been conducted on this new database. However, to show the effectiveness of our proposed approach, we have compared our method with state-of-the-art methods using USTC-NVIE database. Experiment results reveal that our approach is superior to state-of-the-art methods.

Artificial Eyes with Emotion and Light Responsive Pupils for Realistic Humanoid Robots

This study employs a novel 3D engineered robotic eye system with dielectric elastomer actuator (DEA) pupils and a 3D sculpted and colourised gelatin iris membrane to replicate the appearance and materiality of the human eye. A camera system for facial expression analysis (FEA) was installed in the left eye, and a photo-resistor for measuring light frequencies in the right. Unlike previous prototypes, this configuration permits the robotic eyes to respond to both light and emotion proximal to a human eye. A series of experiments were undertaken using a pupil tracking headset to monitor test subjects when observing positive and negative video stimuli. A second test measured pupil dilation ranges to high and low light frequencies using a high-powered artificial light. This data was converted into a series of algorithms for servomotor triangulation to control the photosensitive and emotive pupil dilation sequences. The robotic eyes were evaluated against the pupillometric data and video feeds of the human eyes to determine operational accuracy. Finally, the dilating robotic eye system was installed in a realistic humanoid robot (RHR) and comparatively evaluated in a human-robot interaction (HRI) experiment. The results of this study show that the robotic eyes can emulate the average pupil reflex of the human eye under typical light conditions and to positive and negative emotive stimuli. However, the results of the HRI experiment indicate that replicating natural eye contact behaviour was more significant than emulating pupil dilation.

Awareness and Readiness of Malaysian University Students for Emotion Recognition System

Emotion Recognition System (ERS) identifies human emotion like happiness, sadness, anger, disgust and fear. These emotions can be detected via various modalities such as facial expression analysis, voice intonation, and physiological signals like the brain’s electroencephalogram (EEG) and heart’s electrocardiogram (ECG). The emotion recognition system allows machines to recognized human emotions and reacts to it. It offers broad areas of application, from smart home automation to entertainment recommendation system to driving assistance and to automated security system. It is a promising and interesting field to be explored especially as we are moving towards industrial revolution 5.0. Therefore, a survey was conducted on the awareness and readiness of the usage of emotion recognition system among Malaysian youths, specifically among university students. The findings are presented here. Overall, positive orientation towards the technology is observed among the participants and they are ready for its adoption

Gender and Counterstereotypical Facial Expressions of Emotion in Crowdfunded Microlending

Crowdfunded microlending research implies that both communal and agentic characteristics are valued. These characteristics, however, are often viewed as being at odds with one another due to their association with gender stereotypes. Drawing upon expectancy violation theory and research on gender stereotypes, we theorize that gender-counterstereotypical facial expressions of emotion provide a means for entrepreneurs to project “missing” agentic or communal characteristics. Leveraging computer-aided facial expression analysis to analyze entrepreneur photographs from 43,210 microloan appeals, we show that women benefit from stereotypically masculine facial expressions of anger and disgust, whereas men benefit from stereotypically feminine facial expressions of sadness and happiness.