Human Capacities for Emotion Recognition and their Implications for Computer Vision

AbstractCurrent models for automated emotion recognition are developed under the assumption that emotion expressions are distinct expression patterns for basic emotions. Thereby, these approaches fail to account for the emotional processes underlying emotion expressions. We review the literature on human emotion processing and suggest an alternative approach to affective computing. We postulate that the generalizability and robustness of these models can be greatly increased by three major steps: (1) modeling emotional processes as a necessary foundation of emotion recognition; (2) basing models of emotional processes on our knowledge about the human brain; (3) conceptualizing emotions based on appraisal processes and thus regarding emotion expressions as expressive behavior linked to these appraisals rather than fixed neuro-motor patterns. Since modeling emotional processes after neurobiological processes can be considered a long-term effort, we suggest that researchers should focus on early appraisals, which evaluate intrinsic stimulus properties with little higher cortical involvement. With this goal in mind, we focus on the amygdala and its neural connectivity pattern as a promising structure for early emotional processing. We derive a model for the amygdala-visual cortex circuit from the current state of neuroscientific research. This model is capable of conditioning visual stimuli with body reactions to enable rapid emotional processing of stimuli consistent with early stages of psychological appraisal theories. Additionally, amygdala activity can feed back to visual areas to modulate attention allocation according to the emotional relevance of a stimulus. The implications of the model considering other approaches to automated emotion recognition are discussed.

Download Full-text

Decoding Emotion in Drug Abusers: Evidence for Face and Body Emotion Recognition and for Disgust Emotion

10.20944/preprints202112.0521.v1 ◽

2021 ◽

Author(s):

Natale Salvatore Bonfiglio ◽

Roberta Renati ◽

Gabriella Bottini

Keyword(s):

Emotion Recognition ◽

Cognitive Processes ◽

Cognitive Functions ◽

Emotional Processing ◽

Facial Emotion ◽

Drug Abusers ◽

Emotional Impairments ◽

Treatment Services ◽

Emotional Processes ◽

Posture Recognition

Background: Different drugs damage the frontal cortices, particularly the prefrontal areas involved in both emotional and cognitive functions, with a consequence of decoding emotion deficits for people with substance abuse. The present study aims to explore the cognitive impairments in drug abusers through facial, body and disgust emotion recognition, expanding the investigation of emotions, processing, measuring accuracy and response velocity. Method: We enrolled 13 addicted to cocaine and 12 alcohol patients attending treatment services in Italy, comparing them with 33 matched controls. Facial emotion and body posture recognition tasks, a disgust rating task, and the Barrat Impulsivity Scale were included in the experimental assessment. Results: We found that emotional processes are differently influenced by cocaine and alcohol, suggesting that these substances impact diverse cerebral systems. Conclusion: The contribution made by the duration of consumption on emotional processing seems far less important than for cognitive processes. Drug abusers seem to be slower on elaboration of emotions and, in particular, of disgust emotion. Considering that the participants were not impaired in cognition, our data support the hypothesis that emotional impairments emerge independently from damage to cognitive functions.

Download Full-text

From Micro to Long: Non-Coding RNAs in Tamoxifen Resistance of Breast Cancer Cells

Cancers ◽

10.3390/cancers13153688 ◽

2021 ◽

Vol 13 (15) ◽

pp. 3688

Author(s):

Jéssica Fernanda Barazetti ◽

Tayana Shultz Jucoski ◽

Tamyres Mingorance Carvalho ◽

Rafaela Nasser Veiga ◽

Ana Flávia Kohler ◽

...

Keyword(s):

Breast Cancer ◽

Hormone Receptor ◽

Estrogen Receptor Alpha ◽

Expression Patterns ◽

Tamoxifen Resistance ◽

Hormone Receptor Positive ◽

Current State ◽

Main Driver ◽

Underlying Mechanisms ◽

Multiple Signaling

Breast cancer is the most commonly diagnosed cancer and the leading cause of cancer mortality among women. Two thirds of patients are classified as hormone receptor positive, based on expression of estrogen receptor alpha (ERα), the main driver of breast cancer cell proliferation, and/or progesterone receptor, which is regulated by ERα. Despite presenting the best prognosis, these tumors can recur when patients acquire resistance to treatment by aromatase inhibitors or antiestrogen such as tamoxifen (Tam). The mechanisms that are involved in Tam resistance are complex and involve multiple signaling pathways. Recently, roles for microRNAs and lncRNAs in controlling ER expression and/or tamoxifen action have been described, but the underlying mechanisms are still little explored. In this review, we will discuss the current state of knowledge on the roles of microRNAs and lncRNAs in the main mechanisms of tamoxifen resistance in hormone receptor positive breast cancer. In the future, this knowledge can be used to identify patients at a greater risk of relapse due to the expression patterns of ncRNAs that impact response to Tam, in order to guide their treatment more efficiently and possibly to design therapeutic strategies to bypass mechanisms of resistance.

Download Full-text

Investigating the Relationship Between Emotion Recognition Software and Usability Metrics

i-com ◽

10.1515/icom-2020-0009 ◽

2020 ◽

Vol 19 (2) ◽

pp. 139-151

Author(s):

Thomas Schmidt ◽

Miriam Schlindwein ◽

Katharina Lichtner ◽

Christian Wolff

Keyword(s):

Emotion Recognition ◽

Regression Models ◽

Affective Computing ◽

Large Scale ◽

General Purpose ◽

Thinking Aloud ◽

Continue Research ◽

Usability Metrics ◽

Recognition Software ◽

The Relationship

AbstractDue to progress in affective computing, various forms of general purpose sentiment/emotion recognition software have become available. However, the application of such tools in usability engineering (UE) for measuring the emotional state of participants is rarely employed. We investigate if the application of sentiment/emotion recognition software is beneficial for gathering objective and intuitive data that can predict usability similar to traditional usability metrics. We present the results of a UE project examining this question for the three modalities text, speech and face. We perform a large scale usability test (N = 125) with a counterbalanced within-subject design with two websites of varying usability. We have identified a weak but significant correlation between text-based sentiment analysis on the text acquired via thinking aloud and SUS scores as well as a weak positive correlation between the proportion of neutrality in users’ voice and SUS scores. However, for the majority of the output of emotion recognition software, we could not find any significant results. Emotion metrics could not be used to successfully differentiate between two websites of varying usability. Regression models, either unimodal or multimodal could not predict usability metrics. We discuss reasons for these results and how to continue research with more sophisticated methods.

Download Full-text

Affective Computing on Machine Learning-Based Emotion Recognition Using a Self-Made EEG Device

Sensors ◽

10.3390/s21155135 ◽

2021 ◽

Vol 21 (15) ◽

pp. 5135

Author(s):

Ngoc-Dau Mai ◽

Boon-Giin Lee ◽

Wan-Young Chung

Keyword(s):

Machine Learning ◽

Emotion Recognition ◽

Temporal Lobe ◽

Affective Computing ◽

Electrode Placement ◽

Entropy Measure ◽

Support Vector ◽

Emotion Classification ◽

Eeg Signals ◽

Computing Method

In this research, we develop an affective computing method based on machine learning for emotion recognition using a wireless protocol and a wearable electroencephalography (EEG) custom-designed device. The system collects EEG signals using an eight-electrode placement on the scalp; two of these electrodes were placed in the frontal lobe, and the other six electrodes were placed in the temporal lobe. We performed experiments on eight subjects while they watched emotive videos. Six entropy measures were employed for extracting suitable features from the EEG signals. Next, we evaluated our proposed models using three popular classifiers: a support vector machine (SVM), multi-layer perceptron (MLP), and one-dimensional convolutional neural network (1D-CNN) for emotion classification; both subject-dependent and subject-independent strategies were used. Our experiment results showed that the highest average accuracies achieved in the subject-dependent and subject-independent cases were 85.81% and 78.52%, respectively; these accuracies were achieved using a combination of the sample entropy measure and 1D-CNN. Moreover, our study investigates the T8 position (above the right ear) in the temporal lobe as the most critical channel among the proposed measurement positions for emotion classification through electrode selection. Our results prove the feasibility and efficiency of our proposed EEG-based affective computing method for emotion recognition in real-world applications.

Download Full-text

State-dependent alteration in face emotion recognition in depression

The British Journal of Psychiatry ◽

10.1192/bjp.bp.110.078139 ◽

2011 ◽

Vol 198 (4) ◽

pp. 302-308 ◽

Cited By ~ 65

Author(s):

Ian M. Anderson ◽

Clare Shippen ◽

Gabriella Juhasz ◽

Diana Chase ◽

Emma Thomas ◽

...

Keyword(s):

Emotion Recognition ◽

Emotional Processing ◽

Recognition Task ◽

Control Group ◽

State Dependent ◽

Depressed Group ◽

History Of ◽

Remitted Depression ◽

Face Emotion Recognition ◽

History Of Depression

BackgroundNegative biases in emotional processing are well recognised in people who are currently depressed but are less well described in those with a history of depression, where such biases may contribute to vulnerability to relapse.AimsTo compare accuracy, discrimination and bias in face emotion recognition in those with current and remitted depression.MethodThe sample comprised a control group (n = 101), a currently depressed group (n = 30) and a remitted depression group (n = 99). Participants provided valid data after receiving a computerised face emotion recognition task following standardised assessment of diagnosis and mood symptoms.ResultsIn the control group women were more accurate in recognising emotions than men owing to greater discrimination. Among participants with depression, those in remission correctly identified more emotions than controls owing to increased response bias, whereas those currently depressed recognised fewer emotions owing to decreased discrimination. These effects were most marked for anger, fear and sadness but there was no significant emotion × group interaction, and a similar pattern tended to be seen for happiness although not for surprise or disgust. These differences were confined to participants who were antidepressant-free, with those taking antidepressants having similar results to the control group.ConclusionsAbnormalities in face emotion recognition differ between people with current depression and those in remission. Reduced discrimination in depressed participants may reflect withdrawal from the emotions of others, whereas the increased bias in those with a history of depression could contribute to vulnerability to relapse. The normal face emotion recognition seen in those taking medication may relate to the known effects of antidepressants on emotional processing and could contribute to their ability to protect against depressive relapse.

Download Full-text

Acute Effects of Heroin on Negative Emotional Processing: Relation of Amygdala Activity and Stress-Related Responses

Biological Psychiatry ◽

10.1016/j.biopsych.2013.10.019 ◽

2014 ◽

Vol 76 (4) ◽

pp. 289-296 ◽

Cited By ~ 43

Author(s):

André Schmidt ◽

Stefan Borgwardt ◽

Hana Gerber ◽

Gerhard A. Wiesbeck ◽

Otto Schmid ◽

...

Keyword(s):

Emotional Processing ◽

Acute Effects ◽

Amygdala Activity

Download Full-text

Neural Dysfunction in Postpartum Depression: An fMRI Pilot Study

CNS Spectrums ◽

10.1017/s1092852900015595 ◽

2007 ◽

Vol 12 (11) ◽

pp. 853-862 ◽

Cited By ~ 48

Author(s):

Michael E. Silverman ◽

Holly Loudon ◽

Michal Safier ◽

Xenia Protopopescu ◽

Gila Leiter ◽

...

Keyword(s):

Postpartum Depression ◽

Postpartum Period ◽

Emotional Processing ◽

The United States ◽

Depressive Illness ◽

Critical Gap ◽

Amygdala Activity ◽

Clinically Significant ◽

And Behavior ◽

Striatal Function

ABSTRACTIntroduction:With ∼4 million births each year in the United States, an estimated 760,000 women annually suffer from a clinically significant postpartum depressive illness. Yet even though the relationship between psychiatric disorders and the postpartum period has been documented since the time of Hippocrates, fewer than half of all these cases are recognized.Objective:Because postpartum depression (PPD), the most common complication of childbearing, remains poorly characterized, and its etiology remains unclear, we attempted to address a critical gap in the mechanistic understanding of PPD by probing its systems-level neuropathophysiology, in the context of a specific neurobiological model of fronto-limbic-striatal function.Methods:Using emotionally valenced word probes, with linguistic semantic specificity within an integrated functional magnetic resonance imaging (fMRI) protocol, we investigated emotional processing, behavioral regulation, and their interaction (functions of clinical relevance to PPD), in the context of fronto-limbic-striatal function.Results:We observed attenuated activity in posterior orbitofrontal cortex for negative versus neutral stimuli with greater PPD symptomatology, increased amygdala activity in response to negative words in those without PPD symptomotology, and attenuated striatum activation to positive word conditions with greater PPD symptomotology.Conclusion:Identifying the functional neuroanatomical profile of brain systems involved in the regulation of emotion and behavior in the postpartum period will not only assist in determining whether the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition psychiatric diagnostic specifier of PPD has an associated, unique, functional neuroanatomical profile, but a neurobiological characterization in relation to asymptomatic (postpartum nondepressed) control subjects, will also increase our understanding of the affective disorder spectrum, shed additional light on the possible mechanism(s) responsible for PPD and provide a necessary foundation for the development of more targeted, biologically based diagnostic and therapeutic strategies for PPD.

Download Full-text

A Mathematical Description of Emotional Processes and Its Potential Applications to Affective Computing

IEEE Transactions on Affective Computing ◽

10.1109/taffc.2018.2887385 ◽

2018 ◽

pp. 1-1 ◽

Cited By ~ 1

Author(s):

Luca Puviani ◽

Sidita Rama ◽

Giorgio Matteo Vitetta

Keyword(s):

Mathematical Description ◽

Affective Computing ◽

Emotional Processes ◽

Potential Applications

Download Full-text

Multimodal Emotion Recognition using Deep Learning

Journal of Applied Science and Technology Trends ◽

10.38094/jastt20291 ◽

2021 ◽

Vol 2 (02) ◽

pp. 52-58

Author(s):

Sharmeen M.Saleem Abdullah Abdullah ◽

Siddeeq Y. Ameen Ameen ◽

Mohammed Mohammed sadeeq ◽

Subhi Zeebaree

Keyword(s):

Deep Learning ◽

Affective Computing ◽

Emotional Awareness ◽

Computing Systems ◽

Multimodal Signals ◽

Current State ◽

Multimodal Emotion Recognition ◽

New Research ◽

State Of The Science ◽

Facial Images

New research into human-computer interaction seeks to consider the consumer's emotional status to provide a seamless human-computer interface. This would make it possible for people to survive and be used in widespread fields, including education and medicine. Multiple techniques can be defined through human feelings, including expressions, facial images, physiological signs, and neuroimaging strategies. This paper presents a review of emotional recognition of multimodal signals using deep learning and comparing their applications based on current studies. Multimodal affective computing systems are studied alongside unimodal solutions as they offer higher accuracy of classification. Accuracy varies according to the number of emotions observed, features extracted, classification system and database consistency. Numerous theories on the methodology of emotional detection and recent emotional science address the following topics. This would encourage studies to understand better physiological signals of the current state of the science and its emotional awareness problems.

Download Full-text