Automatic Coding of Facial Expressions of Pain: Are We There Yet?

Introduction. The experience of pain is regularly accompanied by facial expressions. The gold standard for analyzing these facial expressions is the Facial Action Coding System (FACS), which provides so-called action units (AUs) as parametrical indicators of facial muscular activity. Particular combinations of AUs have appeared to be pain-indicative. The manual coding of AUs is, however, too time- and labor-intensive in clinical practice. New developments in automatic facial expression analysis have promised to enable automatic detection of AUs, which might be used for pain detection. Objective. Our aim is to compare manual with automatic AU coding of facial expressions of pain. Methods. FaceReader7 was used for automatic AU detection. We compared the performance of FaceReader7 using videos of 40 participants (20 younger with a mean age of 25.7 years and 20 older with a mean age of 52.1 years) undergoing experimentally induced heat pain to manually coded AUs as gold standard labeling. Percentages of correctly and falsely classified AUs were calculated, and we computed as indicators of congruency, “sensitivity/recall,” “precision,” and “overall agreement (F1).” Results. The automatic coding of AUs only showed poor to moderate outcomes regarding sensitivity/recall, precision, and F1. The congruency was better for younger compared to older faces and was better for pain-indicative AUs compared to other AUs. Conclusion. At the moment, automatic analyses of genuine facial expressions of pain may qualify at best as semiautomatic systems, which require further validation by human observers before they can be used to validly assess facial expressions of pain.

Prozesse der Emotionsregulierung in psychoanalytischen Langzeitpsychotherapien

ZusammenfassungDas Projekt verfolgt das Ziel, das affektive Regulierungsgeschehen in psychotherapeutischen Interaktionen anhand von Videoaufnahmen mikroanalytisch zu beschreiben und mit einem produktiven therapeutischen Prozess in Beziehung zu setzen. Analysiert werden mimische Verhaltensweisen, die mit dem Facial Action Coding System (FACS) objektiv erfasst werden. Im Fokus des Beitrags stehen die sogenannten Prototypischen Affektiven Mikrosequenzen (PAMS). PAMs sind durch Lächeln und Lachen gekennzeichnet und dienen dazu, Störungen in der Affektregulierung mit Hilfe des Gegenübers auszuregulieren. Sie spielen in der therapeutischen Beziehung eine bedeutsame Rolle bei der Aufrechterhaltung einer Balance zwischen Beziehungssicherheit und Konfliktspannung. Unsere Analysen sollen dabei helfen, die Funktionen dieser weitgehend unbewussten Prozesse besser zu verstehen und für den therapeutischen Alltag nutzbar zu machen.

The Child Emotion Facial Expression Set: A Database for Emotion Recognition in Children

Background: This study developed a photo and video database of 4-to-6-year-olds expressing the seven induced and posed universal emotions and a neutral expression. Children participated in photo and video sessions designed to elicit the emotions, and the resulting images were further assessed by independent judges in two rounds.Methods: In the first round, two independent judges (1 and 2), experts in the Facial Action Coding System, firstly analysed 3,668 emotions facial expressions stimuli from 132 children. Both judges reached 100% agreement regarding 1,985 stimuli (124 children), which were then selected for a second round of analysis between judges 3 and 4.Results: The result was 1,985 stimuli (51% of the photographs) were produced from 124 participants (55% girls). A Kappa index of 0.70 and an accuracy of 73% between experts were observed. Lower accuracy was found for emotional expression by 4-year-olds than 6-year-olds. Happiness, disgust and contempt had the highest agreement. After a sub-analysis evaluation of all four judges, 100% agreement was reached for 1,381 stimuli which compound the ChildEFES database with 124 participants (59% girls) and 51% induced photographs. The number of stimuli of each emotion were: 87 for neutrality, 363 for happiness, 170 for disgust, 104 for surprise, 152 for fear, 144 for sadness, 157 for anger 157, and 183 for contempt.Conclusions: The findings show that this photo and video database can facilitate research on the mechanisms involved in early childhood recognition of facial emotions in children, contributing to the understanding of facial emotion recognition deficits which characterise several neurodevelopmental and psychiatric disorders.

Changes in Facial Recognition and Facial Expressions with Age

The ability to express and recognize emotion via facial expressions is well known to change with age. The present study investigated the differences in the facial recognition and facial expression of the elderly (n = 57) and the young (n = 115) and measure how each group uses different facial muscles for each emotion with Facial Action Coding System (FACS). In facial recognition task, the elderly did not recognize facial expressions better than young people and reported stronger feelings of fear and sad from photographs. In making facial expression task, the elderly rated all their facial expressions as stronger than the younger, but in fact, they expressed strong expressions in fear and anger. Furthermore, the elderly used more muscles in the lower face when making facial expressions than younger people. These results help to understand better how the facial recognition and expression of the elderly change, and show that the elderly do not effectively execute the top-down processing concerning facial expression.

Facial Action Coding System Applied to Criminal Investigations: The Analysis of a Homicide Case in Southern Italy

Recent criminal psychology research has raised critical questions about applying non-verbal communication methods for lie detection purposes in forensic settings. Research has shown low correlations between non-verbal communication and deception. However, non-verbal communication methods are still widely applied and suggested by police manuals. Results obtained by experimental and field research are biased by the following factors: (i) attention is given only to quantitative aspects of non-verbal behavior; (ii) there is a lack of research of qualitative aspects related to non-verbal behavior analysis; (iii) lack of connections between non-verbal indicators and verbal content; (iv) lack of attention on timing of non-verbal behavior; (v) most research is performed on psychology students in experimental contexts. This article proposes a new methodology for applying the Facial Action Coding System as investigative support and not as a lie detection method. The Facial Action Coding System will be introduced to integrate with verbal content analysis and a new framework to interpret non-verbal signs discussed. The aid of standardized non-verbal methods will be discussed through an in-depth psychological analysis of a case of homicide perpetrated in 2010 in Southern Italy by discussing a video analysis of the suspects’ statements.

Sintesa Ekspresi Wajah Karakter Virtual 3D menggunakan Action Unit berbasis Facial Action Coding System (FACS)

Ekspresi wajah pada karakter virtual 3D memegang penran penting dalam pembuatan sebuah film animasi. Untuk mendapatkan ekspresi wajah yang diinginkan seorang animator kadang mengalami kesulitan dan membutuhkan waktu yang tidak sedikit. Penelitian ini dilakukan untuk mendapatkan ekspresi wajah dengan menggabungkan beberapa Action Unit yang ada pada FACS dan diimplementasikan pada wajah karakter virtual 3D. Action Unit pada FACS dipilih karena mengacu pada struktur otot wajah manusia. Eksperimen yang dilakukan menghasilkan komninasi Action Unit yang dapat membentuk ekspresi seperti joy expression yang dihasilkan dari kombinasi AU 12+26, dan surprise expression yang dihasilkan dari kombinasi AU -4+5+26. Sedangkan untuk sadness expression dan disgust expression karena ada AU yang tidak terwakili pada model 3D sehingga di dapatkan hasil ekspresi yang kurang maksimal.

Implications for Emotion: Using Anatomically Based Facial Coding to Compare Emoji Faces Across Platforms

Emoji faces, which are ubiquitous in our everyday communication, are thought to resemble human faces and aid emotional communication. Yet, few studies examine whether emojis are perceived as a particular emotion and whether that perception changes based on rendering differences across electronic platforms. The current paper draws upon emotion theory to evaluate whether emoji faces depict anatomical differences that are proposed to differentiate human depictions of emotion (hereafter, “facial expressions”). We modified the existing Facial Action Coding System (FACS) (Ekman and Rosenberg, 1997) to apply to emoji faces. An equivalent “emoji FACS” rubric allowed us to evaluate two important questions: First, Anatomically, does the same emoji face “look” the same across platforms and versions? Second, Do emoji faces perceived as a particular emotion category resemble the proposed human facial expression for that emotion? To answer these questions, we compared the anatomically based codes for 31 emoji faces across three platforms and two version updates. We then compared those codes to the proposed human facial expression prototype for the emotion perceived within the emoji face. Overall, emoji faces across platforms and versions were not anatomically equivalent. Moreover, the majority of emoji faces did not conform to human facial expressions for an emotion, although the basic anatomical codes were shared among human and emoji faces. Some emotion categories were better predicted by the assortment of anatomical codes than others, with some individual differences among platforms. We discuss theories of emotion that help explain how emoji faces are perceived as an emotion, even when anatomical differences are not always consistent or specific to an emotion.

Automatic recognition of macaque facial expressions for detection of affective states

Internal affective states produce external manifestations such as facial expressions. In humans, the Facial Action Coding System (FACS) is widely used to objectively quantify the elemental facial action-units (AUs) that build complex facial expressions. A similar system has been developed for macaque monkeys - the Macaque Facial Action Coding System (MaqFACS); yet unlike the human counterpart, which is already partially replaced by automatic algorithms, this system still requires labor-intensive coding. Here, we developed and implemented the first prototype for automatic MaqFACS coding. We applied the approach to the analysis of behavioral and neural data recorded from freely interacting macaque monkeys. The method achieved high performance in recognition of six dominant AUs, generalizing between conspecific individuals (Macaca mulatta) and even between species (Macaca fascicularis). The study lays the foundation for fully automated detection of facial expressions in animals, which is crucial for investigating the neural substrates of social and affective states.

Detection of confusion behavior using a facial expression based on different classification algorithms

Confusion detection systems (CDSs) that need Noninvasive, mobile, and cost-effective methods use facial expressions as a technique to detect confusion. In previous works, the technology that the system used represents a major gap between this proposed CDS and other systems. This CDS depends on the Facial Action Coding System (FACS) that is used to extract facial features. The FACS shows the motion of the facial muscles represented by Action Units (AUs); the movement is represented with one facial muscle or more. Seven AUs are used as possible markers for detecting confusion that has been implemented in the form of a single vector of facial action; the AUs that have been used in this work are AUs 4, 5, 6, 7, 10, 12, and 23. The database used to calculate the performance of the proposed CDS is gathered from 120 participants (91males, 29 females), between the ages of 18-45. Four types of classification algorithms are used as individuals; these classifiers are (VG-RAM), (SVM), Logistic Regression and Quadratic Discriminant classifiers. The best success rate was found when using Logistic Regression and Quadratic Discriminant. This work introduces different classification techniques to detect confusion by collecting an actual database that can be used to evaluate the performance for every CDS employing facial expressions and selecting appropriate facial features.