The face value of feedback: Facial behavior is shaped by goals and punishments during interaction with dynamic faces

2020 ◽  
Author(s):  
Jonathan Yi ◽  
Philip Pärnamets ◽  
Andreas Olsson
Keyword(s):  
Decision Making ◽  
Facial Expression ◽  
Facial Expressions ◽  
Large Body ◽  
Mechanistic Explanation ◽  
Experimental Manipulation ◽  
Corrugator Supercilii ◽  
Dynamic Faces ◽  
Facial Behavior ◽  
Reinforcement Learning Models

Responding appropriately to others’ facial expressions is key to successful social functioning. Despite the large body of work on face perception and spontaneous responses to static faces, little is known about responses to faces in dynamic, naturalistic situations, and no study has investigated how goal directed responses to faces are influenced by learning during dyadic interactions. To experimentally model such situations, we developed a novel method based on online integration of electromyography (EMG) signals from the participants’ face (corrugator supercilii and zygomaticus major) during facial expression exchange with dynamic faces displaying happy and angry facial expressions. Fifty-eight participants learned by trial-and-error to avoid receiving aversive stimulation by either reciprocate (congruently) or respond opposite (incongruently) to the expression of the target face. Our results validated our method, showing that participants learned to optimize their facial behavior, and replicated earlier findings of faster and more accurate responses in congruent vs. incongruent conditions. Moreover, participants performed better on trials when confronted with smiling, as compared to frowning, faces, suggesting it might be easier to adapt facial responses to positively associated expressions. Finally, we applied drift diffusion and reinforcement learning models to provide a mechanistic explanation for our findings which helped clarifying the underlying decision-making processes of our experimental manipulation. Our results introduce a new method to study learning and decision-making in facial expression exchange, in which there is a need to gradually adapt facial expression selection to both social and non-social reinforcements.

scholarly journals The face value of feedback: facial behaviour is shaped by goals and punishments during interaction with dynamic faces

2021 ◽  
Vol 8 (7) ◽  
pp. 202159
Author(s):  
Jonathan Yi ◽  
Philip Pärnamets ◽  
Andreas Olsson
Keyword(s):  
Decision Making ◽  
Facial Expression ◽  
Facial Expressions ◽  
Large Body ◽  
Mechanistic Explanation ◽  
Experimental Manipulation ◽  
Corrugator Supercilii ◽  
Dynamic Faces ◽  
The Face ◽  
Reinforcement Learning Models

Responding appropriately to others' facial expressions is key to successful social functioning. Despite the large body of work on face perception and spontaneous responses to static faces, little is known about responses to faces in dynamic, naturalistic situations, and no study has investigated how goal directed responses to faces are influenced by learning during dyadic interactions. To experimentally model such situations, we developed a novel method based on online integration of electromyography signals from the participants’ face (corrugator supercilii and zygomaticus major) during facial expression exchange with dynamic faces displaying happy and angry facial expressions. Fifty-eight participants learned by trial-and-error to avoid receiving aversive stimulation by either reciprocate (congruently) or respond opposite (incongruently) to the expression of the target face. Our results validated our method, showing that participants learned to optimize their facial behaviour, and replicated earlier findings of faster and more accurate responses in congruent versus incongruent conditions. Moreover, participants performed better on trials when confronted with smiling, when compared with frowning, faces, suggesting it might be easier to adapt facial responses to positively associated expressions. Finally, we applied drift diffusion and reinforcement learning models to provide a mechanistic explanation for our findings which helped clarifying the underlying decision-making processes of our experimental manipulation. Our results introduce a new method to study learning and decision-making in facial expression exchange, in which there is a need to gradually adapt facial expression selection to both social and non-social reinforcements.

scholarly journals Between Joy and Sympathy: Smiling and Sad Recipient Faces Increase Prosocial Behavior in the Dictator Game

2021 ◽  
Vol 18 (11) ◽  
pp. 6172
Author(s):  
Martin Weiß ◽  
Grit Hein ◽  
Johannes Hewig
Keyword(s):  
Decision Making ◽  
Facial Expression ◽  
Facial Expressions ◽  
Sequential Analysis ◽  
Dictator Game ◽  
Preceding Trial ◽  
Relevant Information ◽  
Allocation Decision ◽  
Online Study ◽  
Human Interactions

In human interactions, the facial expression of a bargaining partner may contain relevant information that affects prosocial decisions. We were interested in whether facial expressions of the recipient in the dictator game influence dictators’ behavior. To test this, we conducted an online study (n = 106) based on a modified version of a dictator game. The dictators allocated money between themselves and another person (recipient), who had no possibility to respond to the dictator. Importantly, before the allocation decision, the dictator was presented with the facial expression of the recipient (angry, disgusted, sad, smiling, or neutral). The results showed that dictators sent more money to recipients with sad or smiling facial expressions and less to recipients with angry or disgusted facial expressions compared with a neutral facial expression. Moreover, based on the sequential analysis of the decision and the interaction partner in the preceding trial, we found that decision-making depends upon previous interactions.

scholarly journals Decoding of facial expressions of pain in avatars: does sex matter?

2020 ◽  
Vol 0 (0) ◽  
Author(s):  
Eleonora Meister ◽  
Claudia Horn-Hofmann ◽  
Miriam Kunz ◽  
Eva G. Krumhuber ◽  
Stefan Lautenbacher
Keyword(s):  
Decision Making ◽  
Facial Expression ◽  
Facial Expressions ◽  
Crucial Role ◽  
Clinical Decision Making ◽  
Clinical Decision ◽  
Computer Models ◽  
Suitable Method ◽  
Facial Features

AbstractObjectivesThe decoding of facial expressions of pain plays a crucial role in pain diagnostic and clinical decision making. For decoding studies, it is necessary to present facial expressions of pain in a flexible and controllable fashion. Computer models (avatars) of human facial expressions of pain allow for systematically manipulating specific facial features. The aim of the present study was to investigate whether avatars can show realistic facial expressions of pain and how the sex of the avatars influence the decoding of pain by human observers.MethodsFor that purpose, 40 female (mean age: 23.9 years) and 40 male (mean age: 24.6 years) observers watched 80 short videos showing computer-generated avatars, who presented the five clusters of facial expressions of pain (four active and one stoic cluster) identified by Kunz and Lautenbacher (2014). After each clip, observers were asked to provide ratings for the intensity of pain the avatars seem to experience and the certainty of judgement, i.e. if the shown expression truly represents pain.ResultsResults show that three of the four active facial clusters were similarly accepted as valid expressions of pain by the observers whereas only one cluster (“raised eyebrows”) was disregarded. The sex of the observed avatars influenced the decoding of pain as indicated by increased intensity and elevated certainty ratings for female avatars.ConclusionsThe assumption of different valid facial expressions of pain could be corroborated in avatars, which contradicts the idea of only one uniform pain face. The observers’ rating of the avatars’ pain was influenced by the avatars’ sex, which resembles known observer biases for humans. The use of avatars appeared to be a suitable method in research on the decoding of the facial expression of pain, mirroring closely the known forms of human facial expressions.

scholarly journals Creation and validation of the Picture-Set of Young Children’s Affective Facial Expressions (PSYCAFE)

PLoS ONE ◽  
2021 ◽  
Vol 16 (12) ◽  
pp. e0260871
Author(s):  
Matthias Franz ◽  
Tobias Müller ◽  
Sina Hahn ◽  
Daniel Lundqvist ◽  
Dirk Rampoldt ◽  
...  
Keyword(s):  
Facial Expression ◽  
Facial Expressions ◽  
Developmental Psychology ◽  
Validation Data ◽  
Facial Emg ◽  
Facial Expression Analysis ◽  
Corrugator Supercilii ◽  
Accuracy Rates ◽  
Research Domains ◽  
Attachment Behaviour

The immediate detection and correct processing of affective facial expressions are one of the most important competences in social interaction and thus a main subject in emotion and affect research. Generally, studies in these research domains, use pictures of adults who display affective facial expressions as experimental stimuli. However, for studies investigating developmental psychology and attachment behaviour it is necessary to use age-matched stimuli, where it is children that display affective expressions. PSYCAFE represents a newly developed picture-set of children’s faces. It includes reference portraits of girls and boys aged 4 to 6 years averaged digitally from different individual pictures, that were categorized to six basic affects (fear, disgust, happiness, sadness, anger and surprise) plus a neutral facial expression by cluster analysis. This procedure led to deindividualized and affect prototypical portraits. Individual affect expressive portraits of adults from an already validated picture-set (KDEF) were used in a similar way to create affect prototypical images also of adults. The stimulus set has been validated on human observers and entail emotion recognition accuracy rates and scores for intensity, authenticity and likeability ratings of the specific affect displayed. Moreover, the stimuli have also been characterized by the iMotions Facial Expression Analysis Module, providing additional data on probability values representing the likelihood that the stimuli depict the expected affect. Finally, the validation data from human observers and iMotions are compared to data on facial mimicry of healthy adults in response to these portraits, measured by facial EMG (m. zygomaticus major and m. corrugator supercilii).

Preschooler’s faces in spontaneous emotional contexts—how well do they match adult facial expression prototypes?

2012 ◽  
Vol 36 (5) ◽  
pp. 348-357 ◽  
Author(s):  
Augusta Gaspar ◽  
Francisco G. Esteves
Keyword(s):  
Facial Expression ◽  
Facial Expressions ◽  
Facial Expressions Of Emotion ◽  
Adults And Children ◽  
Facial Behavior ◽  
Facial Activity ◽  
Recognition Of Emotions

Prototypical facial expressions of emotion, also known as universal facial expressions, are the underpinnings of most research concerning recognition of emotions in both adults and children. Data on natural occurrences of these prototypes in natural emotional contexts are rare and difficult to obtain in adults. By recording naturalistic observations targeted at emotional contexts in day-to-day kindergarten activities, we investigated the spontaneous facial behavior of 3-year-old children in order to explore associations between context and facial activity and verify the degree of matching between the well-known adult prototypes and facial configurations actually produced by children. When taken individually, most facial actions matched those that comprise the respective emotion prototypical face, but full facial configurations with all characteristic facial actions were scarce but for joy.

Capacity-Free Automatic Processing of Facial Expressions of Emotion

2020 ◽  
Author(s):  
Joshua W Maxwell ◽  
Eric Ruthruff ◽  
michael joseph
Keyword(s):  
Facial Expression ◽  
Facial Expressions ◽  
Face Processing ◽  
Emotional Expressions ◽  
Facial Expression Of Emotion ◽  
Expression Of Emotion ◽  
Facial Expressions Of Emotion ◽  
Correspondence Effect ◽  
Experimental Logic ◽  
Face Task

Are facial expressions of emotion processed automatically? Some authors have not found this to be the case (Tomasik et al., 2009). Here we revisited the question with a novel experimental logic – the backward correspondence effect (BCE). In three dual-task studies, participants first categorized a sound (Task 1) and then indicated the location of a target face (Task 2). In Experiment 1, Task 2 required participants to search for one facial expression of emotion (angry or happy). We observed positive BCEs, indicating that facial expressions of emotion bypassed the central attentional bottleneck and thus were processed in a capacity-free, automatic manner. In Experiment 2, we replicated this effect but found that morphed emotional expressions (which were used by Tomasik) were not processed automatically. In Experiment 3, we observed similar BCEs for another type of face processing previously shown to be capacity-free – identification of familiar faces (Jung et al., 2013). We conclude that facial expressions of emotion are identified automatically when sufficiently unambiguous.

scholarly journals Individual differences in experienced and observational decision-making illuminate interactions between reinforcement learning and declarative memory

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Batel Yifrah ◽  
Ayelet Ramaty ◽  
Genela Morris ◽  
Avi Mendelsohn
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Declarative Memory ◽  
Contextual Information ◽  
Memory Performance ◽  
Relevant Information ◽  
Subjective Memory ◽  
Types Of Information ◽  
Reinforcement Learning Models ◽  
Implicit And Explicit

AbstractDecision making can be shaped both by trial-and-error experiences and by memory of unique contextual information. Moreover, these types of information can be acquired either by means of active experience or by observing others behave in similar situations. The interactions between reinforcement learning parameters that inform decision updating and memory formation of declarative information in experienced and observational learning settings are, however, unknown. In the current study, participants took part in a probabilistic decision-making task involving situations that either yielded similar outcomes to those of an observed player or opposed them. By fitting alternative reinforcement learning models to each subject, we discerned participants who learned similarly from experience and observation from those who assigned different weights to learning signals from these two sources. Participants who assigned different weights to their own experience versus those of others displayed enhanced memory performance as well as subjective memory strength for episodes involving significant reward prospects. Conversely, memory performance of participants who did not prioritize their own experience over others did not seem to be influenced by reinforcement learning parameters. These findings demonstrate that interactions between implicit and explicit learning systems depend on the means by which individuals weigh relevant information conveyed via experience and observation.

The face of future: Face expressions during future thinking

2021 ◽  
pp. 174702182199299
Author(s):  
Mohamad El Haj ◽  
Emin Altintas ◽  
Ahmed A Moustafa ◽  
Abdel Halim Boudoukha
Keyword(s):  
Facial Expression ◽  
Facial Expressions ◽  
Emotional Experience ◽  
Future Scenarios ◽  
Future Thinking ◽  
Analysis Software ◽  
Emotional Facial Expressions ◽  
Facial Analysis ◽  
The Face ◽  
Face Expressions

Future thinking, which is the ability to project oneself forward in time to pre-experience an event, is intimately associated with emotions. We investigated whether emotional future thinking can activate emotional facial expressions. We invited 43 participants to imagine future scenarios, cued by the words “happy,” “sad,” and “city.” Future thinking was video recorded and analysed with a facial analysis software to classify whether facial expressions (i.e., happy, sad, angry, surprised, scared, disgusted, and neutral facial expression) of participants were neutral or emotional. Analysis demonstrated higher levels of happy facial expressions during future thinking cued by the word “happy” than “sad” or “city.” In contrast, higher levels of sad facial expressions were observed during future thinking cued by the word “sad” than “happy” or “city.” Higher levels of neutral facial expressions were observed during future thinking cued by the word “city” than “happy” or “sad.” In the three conditions, the neutral facial expressions were high compared with happy and sad facial expressions. Together, emotional future thinking, at least for future scenarios cued by “happy” and “sad,” seems to trigger the corresponding facial expression. Our study provides an original physiological window into the subjective emotional experience during future thinking.

scholarly journals Facial Expression Recognition Based on Multi-Features Cooperative Deep Convolutional Network

2021 ◽  
Vol 11 (4) ◽  
pp. 1428
Author(s):  
Haopeng Wu ◽  
Zhiying Lu ◽  
Jianfeng Zhang ◽  
Xin Li ◽  
Mingyue Zhao ◽  
...  
Keyword(s):  
Facial Expression ◽  
Facial Expressions ◽  
Facial Expression Recognition ◽  
Video Data ◽  
Expression Recognition ◽  
Convolutional Network ◽  
Facial Movements ◽  
The Face ◽  
Deep Convolutional Network ◽  
Selection Of

This paper addresses the problem of Facial Expression Recognition (FER), focusing on unobvious facial movements. Traditional methods often cause overfitting problems or incomplete information due to insufficient data and manual selection of features. Instead, our proposed network, which is called the Multi-features Cooperative Deep Convolutional Network (MC-DCN), maintains focus on the overall feature of the face and the trend of key parts. The processing of video data is the first stage. The method of ensemble of regression trees (ERT) is used to obtain the overall contour of the face. Then, the attention model is used to pick up the parts of face that are more susceptible to expressions. Under the combined effect of these two methods, the image which can be called a local feature map is obtained. After that, the video data are sent to MC-DCN, containing parallel sub-networks. While the overall spatiotemporal characteristics of facial expressions are obtained through the sequence of images, the selection of keys parts can better learn the changes in facial expressions brought about by subtle facial movements. By combining local features and global features, the proposed method can acquire more information, leading to better performance. The experimental results show that MC-DCN can achieve recognition rates of 95%, 78.6% and 78.3% on the three datasets SAVEE, MMI, and edited GEMEP, respectively.

scholarly journals Hybrid Attention Cascade Network for Facial Expression Recognition

Sensors ◽  
2021 ◽  
Vol 21 (6) ◽  
pp. 2003 ◽  
Author(s):  
Xiaoliang Zhu ◽  
Shihao Ye ◽  
Liang Zhao ◽  
Zhicheng Dai
Keyword(s):  
Facial Expression ◽  
Facial Expressions ◽  
Facial Expression Recognition ◽  
Expression Recognition ◽  
Spatial Features ◽  
Face Images ◽  
Temporal Features ◽  
The Face ◽  
In The Wild ◽  
Fusion Features

As a sub-challenge of EmotiW (the Emotion Recognition in the Wild challenge), how to improve performance on the AFEW (Acted Facial Expressions in the wild) dataset is a popular benchmark for emotion recognition tasks with various constraints, including uneven illumination, head deflection, and facial posture. In this paper, we propose a convenient facial expression recognition cascade network comprising spatial feature extraction, hybrid attention, and temporal feature extraction. First, in a video sequence, faces in each frame are detected, and the corresponding face ROI (range of interest) is extracted to obtain the face images. Then, the face images in each frame are aligned based on the position information of the facial feature points in the images. Second, the aligned face images are input to the residual neural network to extract the spatial features of facial expressions corresponding to the face images. The spatial features are input to the hybrid attention module to obtain the fusion features of facial expressions. Finally, the fusion features are input in the gate control loop unit to extract the temporal features of facial expressions. The temporal features are input to the fully connected layer to classify and recognize facial expressions. Experiments using the CK+ (the extended Cohn Kanade), Oulu-CASIA (Institute of Automation, Chinese Academy of Sciences) and AFEW datasets obtained recognition accuracy rates of 98.46%, 87.31%, and 53.44%, respectively. This demonstrated that the proposed method achieves not only competitive performance comparable to state-of-the-art methods but also greater than 2% performance improvement on the AFEW dataset, proving the significant outperformance of facial expression recognition in the natural environment.

Sign in / Sign up

Export Citation Format

Share Document