scholarly journals Decoding dynamic implicit and explicit representations of facial expressions of emotion from EEG

2018 ◽  
Author(s):  
Fraser W. Smith ◽  
Marie L Smith
Keyword(s):  
Facial Expression ◽  
Facial Expressions ◽  
Time Window ◽  
Processing Conditions ◽  
Social Signals ◽  
Implicit Processing ◽  
Task Constraints ◽  
Neural Information ◽  
Time Courses ◽  
Cortical Regions

AbstractFaces transmit a wealth of important social signals. While previous studies have elucidated the network of cortical regions important for perception of facial expression, and the associated temporal components such as the P100, N170 and EPN, it is still unclear how task constraints may shape the representation of facial expression (or other face categories) in these networks. In the present experiment, we investigate the neural information available across time about two important face categories (expression and identity) when those categories are either perceived under explicit (e.g. decoding emotion when task is on emotion) or implicit task contexts (e.g. decoding emotion when task is on identity). Decoding of both face categories, across both task contexts, peaked in a 100-200ms time-window post-stimulus (across posterior electrodes). Peak decoding of expression, however, was not affected by task context whereas peak decoding of identity was significantly reduced under implicit processing conditions. In addition, errors in EEG decoding correlated with errors in behavioral categorization under explicit processing for both expression and identity, but only with implicit decoding of expression. Despite these differences, decoding time-courses and the spatial pattern of informative electrodes differed consistently for both tasks across explicit Vs implicit face processing. Finally our results show that information about both face identity and facial expression is available around the N170 time-window on lateral occipito-temporal sites. Taken together, these results reveal differences and commonalities in the processing of face categories under explicit Vs implicit task contexts and suggest that facial expressions are processed to a richer degree even under implicit processing conditions, consistent with prior work indicating the relative automaticity by which emotion is processed. Our work further demonstrates the utility in applying multivariate decoding analyses to EEG for revealing the dynamics of face perception.

From Movement to Action

2017 ◽  
Author(s):  
Peggy Mason
Keyword(s):  
Cerebral Palsy ◽  
Motor Control ◽  
Facial Expression ◽  
Postural Control ◽  
Facial Expressions ◽  
Body Mass ◽  
Emotional Facial Expressions ◽  
Cortical Regions ◽  
Self Motion

Tracts descending from motor control centers in the brainstem and cortex target motor interneurons and in select cases motoneurons. The mechanisms and constraints of postural control are elaborated and the effect of body mass on posture discussed. Feed-forward reflexes that maintain posture during standing and other conditions of self-motion are described. The role of descending tracts in postural control and the pathological posturing is described. Pyramidal (corticospinal and corticobulbar) and extrapyramidal control of body and face movements is contrasted. Special emphasis is placed on cortical regions and tracts involved in deliberate control of facial expression; these pathways are contrasted with mechanisms for generating emotional facial expressions. The signs associated with lesions of either motoneurons or motor control centers are clearly detailed. The mechanisms and presentation of cerebral palsy are described. Finally, understanding how pre-motor cortical regions generate actions is used to introduce apraxia, a disorder of action.

scholarly journals Understanding the Multidimensional and Dynamic Nature of Facial Expressions Based on Indicators for Appraisal Components as Basis for Measuring Drivers' Fear

2021 ◽  
Vol 12 ◽  
Author(s):  
Meng Zhang ◽  
Klas Ihme ◽  
Uwe Drewitz ◽  
Meike Jipp
Keyword(s):  
Facial Expression ◽  
Low Power ◽  
Dynamic Analysis ◽  
Facial Expressions ◽  
Process Model ◽  
Affective Computing ◽  
Driving Simulator ◽  
Underlying Mechanism ◽  
Multidimensional Analysis ◽  
Time Courses

Facial expressions are one of the commonly used implicit measurements for the in-vehicle affective computing. However, the time courses and the underlying mechanism of facial expressions so far have been barely focused on. According to the Component Process Model of emotions, facial expressions are the result of an individual's appraisals, which are supposed to happen in sequence. Therefore, a multidimensional and dynamic analysis of drivers' fear by using facial expression data could profit from a consideration of these appraisals. A driving simulator experiment with 37 participants was conducted, in which fear and relaxation were induced. It was found that the facial expression indicators of high novelty and low power appraisals were significantly activated after a fear event (high novelty: Z = 2.80, p < 0.01, rcontrast = 0.46; low power: Z = 2.43, p < 0.05, rcontrast = 0.50). Furthermore, after the fear event, the activation of high novelty occurred earlier than low power. These results suggest that multidimensional analysis of facial expression is suitable as an approach for the in-vehicle measurement of the drivers' emotions. Furthermore, a dynamic analysis of drivers' facial expressions considering of effects of appraisal components can add valuable information for the in-vehicle assessment of emotions.

scholarly journals Catching a Liar Through Facial Expression of Fear

2021 ◽  
Vol 12 ◽  
Author(s):  
Xunbing Shen ◽  
Gaojie Fan ◽  
Caoyuan Niu ◽  
Zhencai Chen
Keyword(s):  
Facial Expression ◽  
Facial Expressions ◽  
Time Window ◽  
Total Duration ◽  
Truth Telling ◽  
High Stakes ◽  
Game Show ◽  
Facial Movements ◽  
Video Clips ◽  
The Moment

High stakes can be stressful whether one is telling the truth or lying. However, liars can feel extra fear from worrying to be discovered than truth-tellers, and according to the “leakage theory,” the fear is almost impossible to be repressed. Therefore, we assumed that analyzing the facial expression of fear could reveal deceits. Detecting and analyzing the subtle leaked fear facial expressions is a challenging task for laypeople. It is, however, a relatively easy job for computer vision and machine learning. To test the hypothesis, we analyzed video clips from a game show “The moment of truth” by using OpenFace (for outputting the Action Units (AUs) of fear and face landmarks) and WEKA (for classifying the video clips in which the players were lying or telling the truth). The results showed that some algorithms achieved an accuracy of >80% merely using AUs of fear. Besides, the total duration of AU20 of fear was found to be shorter under the lying condition than that from the truth-telling condition. Further analysis found that the reason for a shorter duration in the lying condition was that the time window from peak to offset of AU20 under the lying condition was less than that under the truth-telling condition. The results also showed that facial movements around the eyes were more asymmetrical when people are telling lies. All the results suggested that facial clues can be used to detect deception, and fear could be a cue for distinguishing liars from truth-tellers.

The face value of feedback: Facial behavior is shaped by goals and punishments during interaction with dynamic faces

2020 ◽  
Author(s):  
Jonathan Yi ◽  
Philip Pärnamets ◽  
Andreas Olsson
Keyword(s):  
Decision Making ◽  
Facial Expression ◽  
Facial Expressions ◽  
Large Body ◽  
Mechanistic Explanation ◽  
Experimental Manipulation ◽  
Corrugator Supercilii ◽  
Dynamic Faces ◽  
Facial Behavior ◽  
Reinforcement Learning Models

Responding appropriately to others’ facial expressions is key to successful social functioning. Despite the large body of work on face perception and spontaneous responses to static faces, little is known about responses to faces in dynamic, naturalistic situations, and no study has investigated how goal directed responses to faces are influenced by learning during dyadic interactions. To experimentally model such situations, we developed a novel method based on online integration of electromyography (EMG) signals from the participants’ face (corrugator supercilii and zygomaticus major) during facial expression exchange with dynamic faces displaying happy and angry facial expressions. Fifty-eight participants learned by trial-and-error to avoid receiving aversive stimulation by either reciprocate (congruently) or respond opposite (incongruently) to the expression of the target face. Our results validated our method, showing that participants learned to optimize their facial behavior, and replicated earlier findings of faster and more accurate responses in congruent vs. incongruent conditions. Moreover, participants performed better on trials when confronted with smiling, as compared to frowning, faces, suggesting it might be easier to adapt facial responses to positively associated expressions. Finally, we applied drift diffusion and reinforcement learning models to provide a mechanistic explanation for our findings which helped clarifying the underlying decision-making processes of our experimental manipulation. Our results introduce a new method to study learning and decision-making in facial expression exchange, in which there is a need to gradually adapt facial expression selection to both social and non-social reinforcements.

Capacity-Free Automatic Processing of Facial Expressions of Emotion

2020 ◽  
Author(s):  
Joshua W Maxwell ◽  
Eric Ruthruff ◽  
michael joseph
Keyword(s):  
Facial Expression ◽  
Facial Expressions ◽  
Face Processing ◽  
Emotional Expressions ◽  
Facial Expression Of Emotion ◽  
Expression Of Emotion ◽  
Facial Expressions Of Emotion ◽  
Correspondence Effect ◽  
Experimental Logic ◽  
Face Task

Are facial expressions of emotion processed automatically? Some authors have not found this to be the case (Tomasik et al., 2009). Here we revisited the question with a novel experimental logic – the backward correspondence effect (BCE). In three dual-task studies, participants first categorized a sound (Task 1) and then indicated the location of a target face (Task 2). In Experiment 1, Task 2 required participants to search for one facial expression of emotion (angry or happy). We observed positive BCEs, indicating that facial expressions of emotion bypassed the central attentional bottleneck and thus were processed in a capacity-free, automatic manner. In Experiment 2, we replicated this effect but found that morphed emotional expressions (which were used by Tomasik) were not processed automatically. In Experiment 3, we observed similar BCEs for another type of face processing previously shown to be capacity-free – identification of familiar faces (Jung et al., 2013). We conclude that facial expressions of emotion are identified automatically when sufficiently unambiguous.

The face of future: Face expressions during future thinking

2021 ◽  
pp. 174702182199299
Author(s):  
Mohamad El Haj ◽  
Emin Altintas ◽  
Ahmed A Moustafa ◽  
Abdel Halim Boudoukha
Keyword(s):  
Facial Expression ◽  
Facial Expressions ◽  
Emotional Experience ◽  
Future Scenarios ◽  
Future Thinking ◽  
Analysis Software ◽  
Emotional Facial Expressions ◽  
Facial Analysis ◽  
The Face ◽  
Face Expressions

Future thinking, which is the ability to project oneself forward in time to pre-experience an event, is intimately associated with emotions. We investigated whether emotional future thinking can activate emotional facial expressions. We invited 43 participants to imagine future scenarios, cued by the words “happy,” “sad,” and “city.” Future thinking was video recorded and analysed with a facial analysis software to classify whether facial expressions (i.e., happy, sad, angry, surprised, scared, disgusted, and neutral facial expression) of participants were neutral or emotional. Analysis demonstrated higher levels of happy facial expressions during future thinking cued by the word “happy” than “sad” or “city.” In contrast, higher levels of sad facial expressions were observed during future thinking cued by the word “sad” than “happy” or “city.” Higher levels of neutral facial expressions were observed during future thinking cued by the word “city” than “happy” or “sad.” In the three conditions, the neutral facial expressions were high compared with happy and sad facial expressions. Together, emotional future thinking, at least for future scenarios cued by “happy” and “sad,” seems to trigger the corresponding facial expression. Our study provides an original physiological window into the subjective emotional experience during future thinking.

scholarly journals Facial Expression Recognition Based on Multi-Features Cooperative Deep Convolutional Network

2021 ◽  
Vol 11 (4) ◽  
pp. 1428
Author(s):  
Haopeng Wu ◽  
Zhiying Lu ◽  
Jianfeng Zhang ◽  
Xin Li ◽  
Mingyue Zhao ◽  
...  
Keyword(s):  
Facial Expression ◽  
Facial Expressions ◽  
Facial Expression Recognition ◽  
Video Data ◽  
Expression Recognition ◽  
Convolutional Network ◽  
Facial Movements ◽  
The Face ◽  
Deep Convolutional Network ◽  
Selection Of

This paper addresses the problem of Facial Expression Recognition (FER), focusing on unobvious facial movements. Traditional methods often cause overfitting problems or incomplete information due to insufficient data and manual selection of features. Instead, our proposed network, which is called the Multi-features Cooperative Deep Convolutional Network (MC-DCN), maintains focus on the overall feature of the face and the trend of key parts. The processing of video data is the first stage. The method of ensemble of regression trees (ERT) is used to obtain the overall contour of the face. Then, the attention model is used to pick up the parts of face that are more susceptible to expressions. Under the combined effect of these two methods, the image which can be called a local feature map is obtained. After that, the video data are sent to MC-DCN, containing parallel sub-networks. While the overall spatiotemporal characteristics of facial expressions are obtained through the sequence of images, the selection of keys parts can better learn the changes in facial expressions brought about by subtle facial movements. By combining local features and global features, the proposed method can acquire more information, leading to better performance. The experimental results show that MC-DCN can achieve recognition rates of 95%, 78.6% and 78.3% on the three datasets SAVEE, MMI, and edited GEMEP, respectively.

scholarly journals Hybrid Attention Cascade Network for Facial Expression Recognition

Sensors ◽  
2021 ◽  
Vol 21 (6) ◽  
pp. 2003 ◽  
Author(s):  
Xiaoliang Zhu ◽  
Shihao Ye ◽  
Liang Zhao ◽  
Zhicheng Dai
Keyword(s):  
Facial Expression ◽  
Facial Expressions ◽  
Facial Expression Recognition ◽  
Expression Recognition ◽  
Spatial Features ◽  
Face Images ◽  
Temporal Features ◽  
The Face ◽  
In The Wild ◽  
Fusion Features

As a sub-challenge of EmotiW (the Emotion Recognition in the Wild challenge), how to improve performance on the AFEW (Acted Facial Expressions in the wild) dataset is a popular benchmark for emotion recognition tasks with various constraints, including uneven illumination, head deflection, and facial posture. In this paper, we propose a convenient facial expression recognition cascade network comprising spatial feature extraction, hybrid attention, and temporal feature extraction. First, in a video sequence, faces in each frame are detected, and the corresponding face ROI (range of interest) is extracted to obtain the face images. Then, the face images in each frame are aligned based on the position information of the facial feature points in the images. Second, the aligned face images are input to the residual neural network to extract the spatial features of facial expressions corresponding to the face images. The spatial features are input to the hybrid attention module to obtain the fusion features of facial expressions. Finally, the fusion features are input in the gate control loop unit to extract the temporal features of facial expressions. The temporal features are input to the fully connected layer to classify and recognize facial expressions. Experiments using the CK+ (the extended Cohn Kanade), Oulu-CASIA (Institute of Automation, Chinese Academy of Sciences) and AFEW datasets obtained recognition accuracy rates of 98.46%, 87.31%, and 53.44%, respectively. This demonstrated that the proposed method achieves not only competitive performance comparable to state-of-the-art methods but also greater than 2% performance improvement on the AFEW dataset, proving the significant outperformance of facial expression recognition in the natural environment.

Can Unhappy Pictures Enhance the Effect of Personas? A User Experiment

2022 ◽  
Vol 29 (2) ◽  
pp. 1-59
Author(s):  
Joni Salminen ◽  
Sercan Şengün ◽  
João M. Santos ◽  
Soon-Gyo Jung ◽  
Bernard Jansen
Keyword(s):  
Mixed Methods ◽  
Facial Expression ◽  
Personality Traits ◽  
Facial Expressions ◽  
User Perceptions ◽  
Design Ideas ◽  
User Experiment ◽  
Pain Points

There has been little research into whether a persona's picture should portray a happy or unhappy individual. We report a user experiment with 235 participants, testing the effects of happy and unhappy image styles on user perceptions, engagement, and personality traits attributed to personas using a mixed-methods analysis. Results indicate that the participant's perceptions of the persona's realism and pain point severity increase with the use of unhappy pictures. In contrast, personas with happy pictures are perceived as more extroverted, agreeable, open, conscientious, and emotionally stable. The participants’ proposed design ideas for the personas scored more lexical empathy scores for happy personas. There were also significant perception changes along with the gender and ethnic lines regarding both empathy and perceptions of pain points. Implications are the facial expression in the persona profile can affect the perceptions of those employing the personas. Therefore, persona designers should align facial expressions with the task for which the personas will be employed. Generally, unhappy images emphasize realism and pain point severity, and happy images invoke positive perceptions.

Effects of Pedagogical Agents on Students’ Mathematics Performance: A Comparison Between Two Versions

2017 ◽  
Vol 56 (5) ◽  
pp. 701-722 ◽  
Author(s):  
Rex P. Bringula ◽  
Ian Clement O. Fosgate ◽  
Neil Peter R. Garcia ◽  
Josf Luinico M. Yorobe
Keyword(s):  
Experimental Study ◽  
Facial Expression ◽  
Prior Knowledge ◽  
Facial Expressions ◽  
Mathematics Learning ◽  
Mathematics Performance ◽  
Pedagogical Agents ◽  
Pedagogical Agent ◽  
Mathematics Scores ◽  
Mathematics Problems

This experimental study investigated the effects of the use of two versions of a pedagogical agent named personal instructing agent (PIA) on the mathematics performance of students. The first version exhibits synthetic facial expressions while the second version does not exhibit facial expression (i.e., neutral facial expression). Two groups of students with the same levels of prior knowledge in mathematics utilized two different versions of PIA. The first group—the facial group—utilized a PIA that provides textual and facial expressions feedback (happy, sad, surprise, and neutral facial expressions). The second group—the nonfacial group—used the same software except that PIA only exhibited neutral facial expression. The study showed that the mathematics scores of the students in the facial group significantly improved as compared with those who are in the nonfacial group. The posttest scores of the facial group were found significantly higher than those of the nonfacial group. The study showed that PIA that exhibited synthetic facial expressions improved students’ mathematics learning. It is concluded that synthetic facial expressions and textual feedback of pedagogical agent can be utilized to help students learn to solve mathematics problems. Limitations and recommendations are also presented.

Sign in / Sign up

Export Citation Format

Share Document