Spontaneous Facial Expressions and Micro-expressions Coding: From Brain to Face

Facial expressions are a vital way for humans to show their perceived emotions. It is convenient for detecting and recognizing expressions or micro-expressions by annotating a lot of data in deep learning. However, the study of video-based expressions or micro-expressions requires that coders have professional knowledge and be familiar with action unit (AU) coding, leading to considerable difficulties. This paper aims to alleviate this situation. We deconstruct facial muscle movements from the motor cortex and systematically sort out the relationship among facial muscles, AU, and emotion to make more people understand coding from the basic principles: We derived the relationship between AU and emotion based on a data-driven analysis of 5,000 images from the RAF-AU database, along with the experience of professional coders.We discussed the complex facial motor cortical network system that generates facial movement properties, detailing the facial nucleus and the motor system associated with facial expressions.The supporting physiological theory for AU labeling of emotions is obtained by adding facial muscle movements patterns.We present the detailed process of emotion labeling and the detection and recognition of AU.Based on the above research, the video's coding of spontaneous expressions and micro-expressions is concluded and prospected.

Measuring engagement among older adults using a multidimensional analysis of communication: Method Design (Preprint)

BACKGROUND Characterising older adult engagement is important to determine the effectiveness of interventions. Engagement is the occupying of oneself in external stimuli and is observable across multiple dimensions of behaviour. Engagement of older adults is commonly investigated on a single behavioural dimension. OBJECTIVE In this article, we present a multidisciplinary approach for measuring and characterising engagement of older adults using techniques appropriate for people with varying degrees of dementia. METHODS Contexts for engagement included a dyadic reminiscence therapy interview and a 12-week technology driven group reminiscence therapy. Participants were older adults (8 female, 1 male, mean age: 79) who attended a day respite facility. Audio-visual recordings of the sessions were processed to analyse facial movement, lexical use, and prosodic patterns of speech. Facial movement was processed using OpenFace to measure the presence and intensity of facial movement. Lexical use was processed using the Linguistic Enquiry and Word Count to measure personal pronoun use, affective word use, and emotional tone of words in speech. Prosodic patterns of speech were processed using custom scripts written in Praat and Python, to measure mean duration of utterances, mean words per utterance, articulation rate and variability of F0. Mixed-effects modelling was used to assess effects of treatment conditions on dependent variable outcomes. RESULTS Results indicate measuring engagement through a multidimensional approach can sensitively capture older adults’ engagement. CONCLUSIONS Application of this method can enhance a researcher’s ability to measure older adult engagement, provide means to compare across interventions and contextual environments, and further develop the science of psychosocial intervention research.

Does age-dynamic movement accelerate facial age impression? Perception of age from facial movement: Studies of Japanese women

We form impressions of others by observing their constant and dynamically-shifting facial expressions during conversation and other daily life activities. However, conventional aging research has mainly considered the changing characteristics of the skin, such as wrinkles and age-spots, within very limited states of static faces. In order to elucidate the range of aging impressions that we make in daily life, it is necessary to consider the effects of facial movement. This study investigated the effects of facial movement on age impressions. An age perception test using Japanese women as face models was employed to verify the effects of the models’ age-dependent facial movements on age impression in 112 participants (all women, aged 20–49 years) as observers. Further, the observers’ gaze was analyzed to identify the facial areas of interests during age perception. The results showed that cheek movement affects age impressions, and that the impressions increase depending on the model’s age. These findings will facilitate the development of new means of provoking a more youthful impression by approaching anti-aging from a different viewpoint of facial movement.

Android Printing: Towards On-Demand Android Development Employing Multi-Material 3-D Printer

In this paper, we propose the concept of Android Printing, which is printing a full android, including skin and mechanical components in a single run using a multi-material 3-D printer. Printing an android all at once both reduces assembly time and enables intricate designs with a high degrees of freedom. To prove this concept, we tested by actual printing an android. First, we printed the skin with multiple annular ridges to test skin deformation. By pulling the skin, we show that the state of deformation of the skin can be adjusted depending on the ridge structure. This result is essential in designing humanlike skin deformations. After that, we designed and fabricated a 3-D printed android head with 31 degrees of freedom. The skin and linkage mechanism were printed together before connecting them to a unit combining several electric motors. To confirm our concept’s feasibility, we created several motions with the android based on human facial movement data. In the future, android printing might enable people to use an android as their own avatar.

Android Printing: Towards On-Demand Android Development Employing Multi-Material 3-D Printer

In this paper, we propose the concept of Android Printing, which is printing a full android, including skin and mechanical components in a single run using a multi-material 3-D printer. Printing an android all at once both reduces assembly time and enables intricate designs with a high degrees of freedom. To prove this concept, we tested by actual printing an android. First, we printed the skin with multiple annular ridges to test skin deformation. By pulling the skin, we show that the state of deformation of the skin can be adjusted depending on the ridge structure. This result is essential in designing humanlike skin deformations. After that, we designed and fabricated a 3-D printed android head with 31 degrees of freedom. The skin and linkage mechanism were printed together before connecting them to a unit combining several electric motors. To confirm our concept’s feasibility, we created several motions with the android based on human facial movement data. In the future, android printing might enable people to use an android as their own avatar.

Composite Binary Pattern Assisted Micro-Expressions Spotting Through Feature Difference Analysis

Micro-Expressions (MEs) are one kind of facial movement which is very spontaneous and involuntary in nature. MEs are observed when a person attempts to hide or conceal the experiencing emotion in a high-stakes environment. The duration of ME is very short and approximately less than 500 milliseconds. Recognition of such kinds of expressions from lengthy video consequences to a limited Micro Expression Recognition Performance and also creates the computational burden. Hence, in this paper, we propose a new ME spotting (detection of ME frames) method based on a new texture descriptor called Composite Binary Pattern (CBP). As a pre-processing, we employ the viola jones algorithm for landmark regions detection followed by landmark points detection for facial alignment. Next, every aligned face is described through CBP and subjected to feature difference analysis followed by the threshold for ME spotting. For simulation, the REVIEW dataset is used and the performance is measured through Recall, Precision, and F-Score.

How emotion is experienced and expressed in multiple cultures: a large-scale experiment

Core to understanding emotion are subjective experiences and their embodiment in facial behavior. Past studies have focused on six emotions and prototypical facial poses, reflecting limitations in scale and narrow assumptions about emotion. We examine 45,231 reactions to 2,185 evocative videos, largely in North America, Europe, and Japan, collecting participants’ self-reported experiences in English or Japanese and manual/automated annotations of facial movement. We uncover 21 dimensions of emotion underlying experiences reported across languages. Facial expressions predict at least 12 dimensions of experience, despite individual variability. We also identify culture-specific display tendencies—many facial movements differ in intensity in Japan compared to the U.S./Canada and Europe, but represent similar experiences. These results reveal how people actually experience and express emotion: in high-dimensional, categorical, and complex fashion.