scholarly journals Embodied Emotion Recognition Based on Life-Logging

Sensors ◽  
2019 ◽  
Vol 19 (23) ◽  
pp. 5308
Author(s):  
Ayoung Cho ◽  
Hyunwoo Lee ◽  
Youngho Jo ◽  
Mincheol Whang
Keyword(s):  
Emotion Recognition ◽  
Environmental Factor ◽  
Physiological Response ◽  
Real Life ◽  
Behavioral Pattern ◽  
Arousal Level ◽  
Behavioral Patterns ◽  
Global Positioning ◽  
Embodied Emotion ◽  
Valence Level

Embodied emotion is associated with interaction among a person’s physiological responses, behavioral patterns, and environmental factors. However, most methods for determining embodied emotion has been considered on only fragmentary independent variables and not their inter-connectivity. This study suggests a method for determining the embodied emotion considering interactions among three factors: the physiological response, behavioral patterns, and an environmental factor based on life-logging. The physiological response was analyzed as heart rate variability (HRV) variables. The behavioral pattern was calculated from features of Global Positioning System (GPS) locations that indicate spatiotemporal property. The environmental factor was analyzed as the ambient noise, which is an external stimulus. These data were mapped with the emotion of that time. The emotion was evaluated on a seven-point scale for arousal level and valence level according to Russell’s model of emotion. These data were collected from 79 participants in daily life for two weeks. Their relationships among data were analyzed by the multiple regression analysis, after pre-processing the respective data. As a result, significant differences between the arousal level and valence level of emotion were observed based on their relations. The contributions of this study can be summarized as follows: (1) The emotion was recognized in real-life for a more practical application; (2) distinguishing the interactions that determine the levels of arousal and positive emotion by analyzing relationships of individuals’ life-log data. Through this, it was verified that emotion can be changed according to the interaction among the three factors, which was overlooked in previous emotion recognition.

Fractal dimension pattern-based multiresolution analysis for rough estimator of speaker-dependent audio emotion recognition

2017 ◽  
Vol 15 (05) ◽  
pp. 1750042 ◽  
Author(s):  
Miao Cheng ◽  
Ah Chung Tsoi
Keyword(s):  
Fractal Dimension ◽  
Emotion Recognition ◽  
Multiresolution Analysis ◽  
Real Life ◽  
Emotional States ◽  
Acoustic Feature ◽  
Comparative Performance ◽  
Audio Analysis ◽  
The Arts ◽  
The Given

As a general means of expression, audio analysis and recognition have attracted much attention for its wide applications in real-life world. Audio emotion recognition (AER) attempts to understand the emotional states of human with the given utterance signals, and has been studied abroad for its further development on friendly human–machine interfaces. Though there have been several the-state-of-the-arts auditory methods devised to audio recognition, most of them focus on discriminative usage of acoustic features, while feedback efficiency of recognition demands is ignored. This makes possible application of AER, and rapid learning of emotion patterns is desired. In order to make predication of audio emotion possible, the speaker-dependent patterns of audio emotions are learned with multiresolution analysis, and fractal dimension (FD) features are calculated for acoustic feature extraction. Furthermore, it is able to efficiently learn the intrinsic characteristics of auditory emotions, while the utterance features are learned from FDs of each sub-band. Experimental results show the proposed method is able to provide comparative performance for AER.

scholarly journals Real-Time Facial Emotion Recognition Framework for Employees of Organizations Using Raspberry-Pi

2021 ◽  
Vol 11 (22) ◽  
pp. 10540
Author(s):  
Navjot Rathour ◽  
Zeba Khanam ◽  
Anita Gehlot ◽  
Rajesh Singh ◽  
Mamoon Rashid ◽  
...  
Keyword(s):  
Emotion Recognition ◽  
Real Time ◽  
Real Life ◽  
Recognition System ◽  
Facial Emotion Recognition ◽  
Facial Emotion ◽  
Raspberry Pi ◽  
Input Device ◽  
Facial Emotions ◽  
Deep Network

There is a significant interest in facial emotion recognition in the fields of human–computer interaction and social sciences. With the advancements in artificial intelligence (AI), the field of human behavioral prediction and analysis, especially human emotion, has evolved significantly. The most standard methods of emotion recognition are currently being used in models deployed in remote servers. We believe the reduction in the distance between the input device and the server model can lead us to better efficiency and effectiveness in real life applications. For the same purpose, computational methodologies such as edge computing can be beneficial. It can also encourage time-critical applications that can be implemented in sensitive fields. In this study, we propose a Raspberry-Pi based standalone edge device that can detect real-time facial emotions. Although this edge device can be used in variety of applications where human facial emotions play an important role, this article is mainly crafted using a dataset of employees working in organizations. A Raspberry-Pi-based standalone edge device has been implemented using the Mini-Xception Deep Network because of its computational efficiency in a shorter time compared to other networks. This device has achieved 100% accuracy for detecting faces in real time with 68% accuracy, i.e., higher than the accuracy mentioned in the state-of-the-art with the FER 2013 dataset. Future work will implement a deep network on Raspberry-Pi with an Intel Movidious neural compute stick to reduce the processing time and achieve quick real time implementation of the facial emotion recognition system.

Multimodal emotion perception in young and elderly patients with multiple sclerosis

2021 ◽  
Author(s):  
Maxime Montembeault ◽  
Estefania Brando ◽  
Kim Charest ◽  
Alexandra Tremblay ◽  
Élaine Roger ◽  
...  
Keyword(s):  
Emotion Recognition ◽  
Real Life ◽  
Emotion Perception ◽  
Recognition Task ◽  
Self Report ◽  
Self Monitoring ◽  
Facial Emotions ◽  
Multimodal Emotion Recognition ◽  
The Impact ◽  
Better Than

Background. Studies suggest that emotion recognition and empathy are impaired in patients with MS (pwMS). Nonetheless, most studies of emotion recognition have used facial stimuli, are restricted to young samples, and rely self-report assessments of empathy. The aims of this study are to determine the impact of MS and age on multimodal emotion recognition (facial emotions and vocal emotional bursts) and on socioemotional sensitivity (as reported by the participants and their informants). We also aim to investigate the associations between emotion recognition, socioemotional sensitivity, and cognitive measures. Methods. We recruited 13 young healthy controls (HC), 14 young pwMS, 14 elderly HC and 15 elderly pwMS. They underwent a short neuropsychological battery, an experimental emotion recognition task including facial emotions and vocal emotional bursts. Both participants and their study informants completed the Revised-Self Monitoring Scale (RSMS) to assess the participant’s socioemotional sensitivity. Results. There was a significant effect of age and group on recognition of both facial emotions and emotional vocal bursts, HC performing significantly better than pwMS, and young participants performing better than elderly participants (no interaction effect). The same effects were observed on self-reported socioemotional sensitivity. However, lower socioemotional sensitivity in pwMS was not reported by the informants. Finally, multimodal emotion recognition did not correlate with socioemotional sensitivity, but it correlated with global cognitive severity. Conclusion. PwMS present with multimodal emotion perception deficits. Our results extend previous findings of decreased emotion perception and empathy to a group of elderly pwMS, in which advancing age does not accentuate these deficits. However, the decreased socioemotional sensitivity reported by pwMS does not appear to be observed by their relatives, nor to correlate with their emotion perception impairments. Future studies should investigate the real-life impacts of emotion perception deficits in pwMS.

Embodied Emotion Recognition System

2017 ◽  
pp. 1371-1376
Author(s):  
Ayoung Cho ◽  
Hyunwoo Lee ◽  
Hyeonsang Hwang ◽  
Youseop Jo ◽  
Mincheol Whang
Keyword(s):  
Emotion Recognition ◽  
Recognition System ◽  
Embodied Emotion

scholarly journals Toward Emotion Recognition From Physiological Signals in the Wild: Approaching the Methodological Issues in Real-Life Data Collection

2020 ◽  
Vol 11 ◽  
Author(s):  
Fanny Larradet ◽  
Radoslaw Niewiadomski ◽  
Giacinto Barresi ◽  
Darwin G. Caldwell ◽  
Leonardo S. Mattos
Keyword(s):  
Data Collection ◽  
Emotion Recognition ◽  
Real Life ◽  
Physiological Signals ◽  
Methodological Issues ◽  
Life Data ◽  
Real Life Data ◽  
In The Wild

scholarly journals An Investigation of Deep Learning Models for EEG-Based Emotion Recognition

2020 ◽  
Vol 14 ◽  
Author(s):  
Yaqing Zhang ◽  
Jinling Chen ◽  
Jen Hong Tan ◽  
Yuxuan Chen ◽  
Yunyi Chen ◽  
...  
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Emotion Recognition ◽  
Real Life ◽  
Learning Rate ◽  
Learning Models ◽  
Learning Methods ◽  
Time Frequency ◽  
Comparison Results

Emotion is the human brain reacting to objective things. In real life, human emotions are complex and changeable, so research into emotion recognition is of great significance in real life applications. Recently, many deep learning and machine learning methods have been widely applied in emotion recognition based on EEG signals. However, the traditional machine learning method has a major disadvantage in that the feature extraction process is usually cumbersome, which relies heavily on human experts. Then, end-to-end deep learning methods emerged as an effective method to address this disadvantage with the help of raw signal features and time-frequency spectrums. Here, we investigated the application of several deep learning models to the research field of EEG-based emotion recognition, including deep neural networks (DNN), convolutional neural networks (CNN), long short-term memory (LSTM), and a hybrid model of CNN and LSTM (CNN-LSTM). The experiments were carried on the well-known DEAP dataset. Experimental results show that the CNN and CNN-LSTM models had high classification performance in EEG-based emotion recognition, and their accurate extraction rate of RAW data reached 90.12 and 94.17%, respectively. The performance of the DNN model was not as accurate as other models, but the training speed was fast. The LSTM model was not as stable as the CNN and CNN-LSTM models. Moreover, with the same number of parameters, the training speed of the LSTM was much slower and it was difficult to achieve convergence. Additional parameter comparison experiments with other models, including epoch, learning rate, and dropout probability, were also conducted in the paper. Comparison results prove that the DNN model converged to optimal with fewer epochs and a higher learning rate. In contrast, the CNN model needed more epochs to learn. As for dropout probability, reducing the parameters by ~50% each time was appropriate.

Application of EDM to Understand the Online Students' Behavioral Pattern

2019 ◽  
Vol 12 (3) ◽  
pp. 154-168 ◽  
Author(s):  
Luis Naito Mendes Bezerra ◽  
Márcia Terra da Silva
Keyword(s):  
Data Mining ◽  
Educational Data Mining ◽  
Learning Management Systems ◽  
Behavioral Pattern ◽  
Management Systems ◽  
Online Course ◽  
Educational Context ◽  
Behavioral Patterns ◽  
Limited Capacity ◽  
Online Access

In distance learning, the professor cannot see that the students are having trouble with a subject, and can fail to perceive the problem in time to intervene. However, in learning management systems (LMS's) a large volume of data regarding online access, participation and progress can be registered and collected allowing analysis based on students' behavioral patterns. As traditional methods have a limited capacity to extract knowledge from big volumes of data, educational data mining (EDM) arises as a tool to help teachers interpreting the behavior of students. The objective of the present article is to describe the application of educational data mining technics aiming to obtain relevant knowledge of students' behavioral patterns in an LMS for an online course, with 1,113 students enrolled. This paper applies two algorithms on educational context, decision tree and clustering, unveiling unknown relevant aspects to professors and managers, such as the most important examinations that contribute to students' approval as well as the most significant attributes to their success.

scholarly journals Insensitive Players? A Relationship Between Violent Video Game Exposure and Recognition of Negative Emotions

2021 ◽  
Vol 12 ◽  
Author(s):  
Ewa Miedzobrodzka ◽  
Jacek Buczny ◽  
Elly A. Konijn ◽  
Lydia C. Krabbendam
Keyword(s):  
Emotion Recognition ◽  
Video Game ◽  
Negative Emotion ◽  
Negative Emotions ◽  
Real Life ◽  
Lower Sensitivity ◽  
Violent Video Game ◽  
Causal Direction ◽  
Matching Test ◽  
The One

An ability to accurately recognize negative emotions in others can initiate pro-social behavior and prevent anti-social actions. Thus, it remains of an interest of scholars studying effects of violent video games. While exposure to such games was linked to slower emotion recognition, the evidence regarding accuracy of emotion recognition among players of violent games is weak and inconsistent. The present research investigated the relationship between violent video game exposure (VVGE) and accuracy of negative emotion recognition. We assessed the level of self-reported VVGE in hours per day and the accuracy of the recognition using the Facial Expressions Matching Test. The results, with adolescents (Study 1; N = 67) and with adults (Study 2; N = 151), showed that VVGE was negatively related to accurate recognition of negative emotion expressions, even if controlled for age, gender, and trait empathy, but no causal direction could be assessed. In line with the violent media desensitization model, our findings suggest that higher self-reported VVGE relates to lower recognition of negative emotional expressions of other people. On the one hand, such lower recognition of negative emotions may underlie inaccurate reactions in real-life social situations. On the other hand, lower sensitivity to social cues may help players to better focus on their performance in a violent game.

scholarly journals Multimodal Affect Models: An Investigation of Relative Salience of Audio and Visual Cues for Emotion Prediction

2021 ◽  
Vol 3 ◽  
Author(s):  
Jingyao Wu ◽  
Ting Dang ◽  
Vidhyasaharan Sethu ◽  
Eliathamby Ambikairajah
Keyword(s):  
Emotion Recognition ◽  
Visual Cues ◽  
Emotion Perception ◽  
Arousal Level ◽  
Multiple Cues ◽  
Model Experiments ◽  
Multimodal Systems ◽  
Recognition Systems ◽  
Change Prediction ◽  
Relative Salience

People perceive emotions via multiple cues, predominantly speech and visual cues, and a number of emotion recognition systems utilize both audio and visual cues. Moreover, the perception of static aspects of emotion (speaker's arousal level is high/low) and the dynamic aspects of emotion (speaker is becoming more aroused) might be perceived via different expressive cues and these two aspects are integrated to provide a unified sense of emotion state. However, existing multimodal systems only focus on single aspect of emotion perception and the contributions of different modalities toward modeling static and dynamic emotion aspects are not well explored. In this paper, we investigate the relative salience of audio and video modalities to emotion state prediction and emotion change prediction using a Multimodal Markovian affect model. Experiments conducted in the RECOLA database showed that audio modality is better at modeling the emotion state of arousal and video for emotion state of valence, whereas audio shows superior advantages over video in modeling emotion changes for both arousal and valence.

scholarly journals Emotion Recognition from Speech Using the Bag-of-Visual Words on Audio Segment Spectrograms

Technologies ◽  
2019 ◽  
Vol 7 (1) ◽  
pp. 20 ◽  
Author(s):  
Evaggelos Spyrou ◽  
Rozalia Nikopoulou ◽  
Ioannis Vernikos ◽  
Phivos Mylonas
Keyword(s):  
Computer Vision ◽  
Emotion Recognition ◽  
Affective State ◽  
Real Life ◽  
Support Vector ◽  
Bag Of Visual Words ◽  
Educational Training ◽  
Visual Words ◽  
Speeded Up Robust Features ◽  
Digital World

It is noteworthy nowadays that monitoring and understanding a human’s emotional state plays a key role in the current and forthcoming computational technologies. On the other hand, this monitoring and analysis should be as unobtrusive as possible, since in our era the digital world has been smoothly adopted in everyday life activities. In this framework and within the domain of assessing humans’ affective state during their educational training, the most popular way to go is to use sensory equipment that would allow their observing without involving any kind of direct contact. Thus, in this work, we focus on human emotion recognition from audio stimuli (i.e., human speech) using a novel approach based on a computer vision inspired methodology, namely the bag-of-visual words method, applied on several audio segment spectrograms. The latter are considered to be the visual representation of the considered audio segment and may be analyzed by exploiting well-known traditional computer vision techniques, such as construction of a visual vocabulary, extraction of speeded-up robust features (SURF) features, quantization into a set of visual words, and image histogram construction. As a last step, support vector machines (SVM) classifiers are trained based on the aforementioned information. Finally, to further generalize the herein proposed approach, we utilize publicly available datasets from several human languages to perform cross-language experiments, both in terms of actor-created and real-life ones.

Sign in / Sign up

Export Citation Format

Share Document