scholarly journals Machine Learning and EEG for Emotional State Estimation

2021 ◽  
Author(s):  
Krzysztof Kotowski ◽  
Katarzyna Stapor
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
State Estimation ◽  
Emotional State ◽  
Emotional States ◽  
Eeg Signal ◽  
Self Assessment ◽  
Emotional Perception ◽  
Automatic Systems ◽  
Emotion Models

Defining “emotion” and its accurate measuring is a notorious problem in the psychology domain. It is usually addressed with subjective self-assessment forms filled manually by participants. Machine learning methods and EEG correlates of emotions enable to construction of automatic systems for objective emotion recognition. Such systems could help to assess emotional states and could be used to improve emotional perception. In this chapter, we present a computer system that can automatically recognize an emotional state of a human, based on EEG signals induced by a standardized affective picture database. Based on the EEG signal, trained deep neural networks are then used together with mappings between emotion models to predict the emotions perceived by the participant. This, in turn, can be used for example in validation of affective picture databases standardization.

EEG-based emotion recognition with deep convolutional neural networks

2020 ◽  
Vol 0 (0) ◽  
Author(s):  
Mehmet Akif Ozdemir ◽  
Murside Degirmenci ◽  
Elif Izci ◽  
Aydin Akan
Keyword(s):  
Neural Networks ◽  
Emotion Recognition ◽  
Convolutional Neural Networks ◽  
Spatial Information ◽  
Emotional State ◽  
Human Interaction ◽  
Emotional States ◽  
Eeg Signals ◽  
Deep Convolutional Neural Networks ◽  
Spectral Topology

AbstractThe emotional state of people plays a key role in physiological and behavioral human interaction. Emotional state analysis entails many fields such as neuroscience, cognitive sciences, and biomedical engineering because the parameters of interest contain the complex neuronal activities of the brain. Electroencephalogram (EEG) signals are processed to communicate brain signals with external systems and make predictions over emotional states. This paper proposes a novel method for emotion recognition based on deep convolutional neural networks (CNNs) that are used to classify Valence, Arousal, Dominance, and Liking emotional states. Hence, a novel approach is proposed for emotion recognition with time series of multi-channel EEG signals from a Database for Emotion Analysis and Using Physiological Signals (DEAP). We propose a new approach to emotional state estimation utilizing CNN-based classification of multi-spectral topology images obtained from EEG signals. In contrast to most of the EEG-based approaches that eliminate spatial information of EEG signals, converting EEG signals into a sequence of multi-spectral topology images, temporal, spectral, and spatial information of EEG signals are preserved. The deep recurrent convolutional network is trained to learn important representations from a sequence of three-channel topographical images. We have achieved test accuracy of 90.62% for negative and positive Valence, 86.13% for high and low Arousal, 88.48% for high and low Dominance, and finally 86.23% for like–unlike. The evaluations of this method on emotion recognition problem revealed significant improvements in the classification accuracy when compared with other studies using deep neural networks (DNNs) and one-dimensional CNNs.

scholarly journals Emotion Recognition and Regulation Based on Stacked Sparse Auto-Encoder Network and Personalized Reconfigurable Music

Mathematics ◽  
2021 ◽  
Vol 9 (6) ◽  
pp. 593
Author(s):  
Yinsheng Li ◽  
Wei Zheng
Keyword(s):  
Emotional Regulation ◽  
Emotional State ◽  
Emotional Adjustment ◽  
Recognition Rate ◽  
Traditional Music ◽  
Music Preference ◽  
Emotional States ◽  
Eeg Signal ◽  
Long Duration ◽  
The Brain

Music can regulate and improve the emotions of the brain. Traditional emotional regulation approaches often adopt complete music. As is well-known, complete music may vary in pitch, volume, and other ups and downs. An individual’s emotions may also adopt multiple states, and music preference varies from person to person. Therefore, traditional music regulation methods have problems, such as long duration, variable emotional states, and poor adaptability. In view of these problems, we use different music processing methods and stacked sparse auto-encoder neural networks to identify and regulate the emotional state of the brain in this paper. We construct a multi-channel EEG sensor network, divide brainwave signals and the corresponding music separately, and build a personalized reconfigurable music-EEG library. The 17 features in the EEG signal are extracted as joint features, and the stacked sparse auto-encoder neural network is used to classify the emotions, in order to establish a music emotion evaluation index. According to the goal of emotional regulation, music fragments are selected from the personalized reconfigurable music-EEG library, then reconstructed and combined for emotional adjustment. The results show that, compared with complete music, the reconfigurable combined music was less time-consuming for emotional regulation (76.29% less), and the number of irrelevant emotional states was reduced by 69.92%. In terms of adaptability to different participants, the reconfigurable music improved the recognition rate of emotional states by 31.32%.

scholarly journals Using Machine Learning and Smartphone and Smartwatch Data to Detect Emotional States and Transitions: Exploratory Study (Preprint)

2020 ◽  
Author(s):  
Madeena Sultana ◽  
Majed Al-Jefri ◽  
Joon Lee
Keyword(s):  
Machine Learning ◽  
Emotional State ◽  
Contextual Information ◽  
Classification Problem ◽  
Emotional States ◽  
Discrete Emotions ◽  
Location Services ◽  
State Detection ◽  
Everyday Context ◽  
The Relationship

BACKGROUND Emotional state in everyday life is an essential indicator of health and well-being. However, daily assessment of emotional states largely depends on active self-reports, which are often inconvenient and prone to incomplete information. Automated detection of emotional states and transitions on a daily basis could be an effective solution to this problem. However, the relationship between emotional transitions and everyday context remains to be unexplored. OBJECTIVE This study aims to explore the relationship between contextual information and emotional transitions and states to evaluate the feasibility of detecting emotional transitions and states from daily contextual information using machine learning (ML) techniques. METHODS This study was conducted on the data of 18 individuals from a publicly available data set called ExtraSensory. Contextual and sensor data were collected using smartphone and smartwatch sensors in a free-living condition, where the number of days for each person varied from 3 to 9. Sensors included an accelerometer, a gyroscope, a compass, location services, a microphone, a phone state indicator, light, temperature, and a barometer. The users self-reported approximately 49 discrete emotions at different intervals via a smartphone app throughout the data collection period. We mapped the 49 reported discrete emotions to the 3 dimensions of the pleasure, arousal, and dominance model and considered 6 emotional states: discordant, pleased, dissuaded, aroused, submissive, and dominant. We built general and personalized models for detecting emotional transitions and states every 5 min. The transition detection problem is a binary classification problem that detects whether a person’s emotional state has changed over time, whereas state detection is a multiclass classification problem. In both cases, a wide range of supervised ML algorithms were leveraged, in addition to data preprocessing, feature selection, and data imbalance handling techniques. Finally, an assessment was conducted to shed light on the association between everyday context and emotional states. RESULTS This study obtained promising results for emotional state and transition detection. The best area under the receiver operating characteristic (AUROC) curve for emotional state detection reached 60.55% in the general models and an average of 96.33% across personalized models. Despite the highly imbalanced data, the best AUROC curve for emotional transition detection reached 90.5% in the general models and an average of 88.73% across personalized models. In general, feature analyses show that spatiotemporal context, phone state, and motion-related information are the most informative factors for emotional state and transition detection. Our assessment showed that lifestyle has an impact on the predictability of emotion. CONCLUSIONS Our results demonstrate a strong association of daily context with emotional states and transitions as well as the feasibility of detecting emotional states and transitions using data from smartphone and smartwatch sensors.

scholarly journals Predicting Emotional States Using Behavioral Markers Derived From Passively Sensed Data: Data-Driven Machine Learning Approach

10.2196/24465 ◽  
2021 ◽  
Vol 9 (3) ◽  
pp. e24465
Author(s):  
Emese Sükei ◽  
Agnes Norbury ◽  
M Mercedes Perez-Rodriguez ◽  
Pablo M Olmos ◽  
Antonio Artés
Keyword(s):  
Mental Health ◽  
Machine Learning ◽  
Time Series ◽  
Individual Differences ◽  
Emotional State ◽  
Series Data ◽  
Emotional States ◽  
Missing Observations ◽  
State Prediction ◽  
Behavioral Markers

Background Mental health disorders affect multiple aspects of patients’ lives, including mood, cognition, and behavior. eHealth and mobile health (mHealth) technologies enable rich sets of information to be collected noninvasively, representing a promising opportunity to construct behavioral markers of mental health. Combining such data with self-reported information about psychological symptoms may provide a more comprehensive and contextualized view of a patient’s mental state than questionnaire data alone. However, mobile sensed data are usually noisy and incomplete, with significant amounts of missing observations. Therefore, recognizing the clinical potential of mHealth tools depends critically on developing methods to cope with such data issues. Objective This study aims to present a machine learning–based approach for emotional state prediction that uses passively collected data from mobile phones and wearable devices and self-reported emotions. The proposed methods must cope with high-dimensional and heterogeneous time-series data with a large percentage of missing observations. Methods Passively sensed behavior and self-reported emotional state data from a cohort of 943 individuals (outpatients recruited from community clinics) were available for analysis. All patients had at least 30 days’ worth of naturally occurring behavior observations, including information about physical activity, geolocation, sleep, and smartphone app use. These regularly sampled but frequently missing and heterogeneous time series were analyzed with the following probabilistic latent variable models for data averaging and feature extraction: mixture model (MM) and hidden Markov model (HMM). The extracted features were then combined with a classifier to predict emotional state. A variety of classical machine learning methods and recurrent neural networks were compared. Finally, a personalized Bayesian model was proposed to improve performance by considering the individual differences in the data and applying a different classifier bias term for each patient. Results Probabilistic generative models proved to be good preprocessing and feature extractor tools for data with large percentages of missing observations. Models that took into account the posterior probabilities of the MM and HMM latent states outperformed those that did not by more than 20%, suggesting that the underlying behavioral patterns identified were meaningful for individuals’ overall emotional state. The best performing generalized models achieved a 0.81 area under the curve of the receiver operating characteristic and 0.71 area under the precision-recall curve when predicting self-reported emotional valence from behavior in held-out test data. Moreover, the proposed personalized models demonstrated that accounting for individual differences through a simple hierarchical model can substantially improve emotional state prediction performance without relying on previous days’ data. Conclusions These findings demonstrate the feasibility of designing machine learning models for predicting emotional states from mobile sensing data capable of dealing with heterogeneous data with large numbers of missing observations. Such models may represent valuable tools for clinicians to monitor patients’ mood states.

scholarly journals A Review on the Computational Methods for Emotional State Estimation from the Human EEG

2013 ◽  
Vol 2013 ◽  
pp. 1-13 ◽  
Author(s):  
Min-Ki Kim ◽  
Miyoung Kim ◽  
Eunmi Oh ◽  
Sung-Phil Kim
Keyword(s):  
State Estimation ◽  
Computational Methods ◽  
Affective Computing ◽  
Emotional State ◽  
Recognition System ◽  
Self Report ◽  
Emotional States ◽  
Human Computer Interactions ◽  
Future Directions ◽  
Eeg Data

A growing number of affective computing researches recently developed a computer system that can recognize an emotional state of the human user to establish affective human-computer interactions. Various measures have been used to estimate emotional states, including self-report, startle response, behavioral response, autonomic measurement, and neurophysiologic measurement. Among them, inferring emotional states from electroencephalography (EEG) has received considerable attention as EEG could directly reflect emotional states with relatively low costs and simplicity. Yet, EEG-based emotional state estimation requires well-designed computational methods to extract information from complex and noisy multichannel EEG data. In this paper, we review the computational methods that have been developed to deduct EEG indices of emotion, to extract emotion-related features, or to classify EEG signals into one of many emotional states. We also propose using sequential Bayesian inference to estimate the continuous emotional state in real time. We present current challenges for building an EEG-based emotion recognition system and suggest some future directions.  

scholarly journals Using Machine Learning and Smartphone and Smartwatch Data to Detect Emotional States and Transitions: Exploratory Study

10.2196/17818 ◽  
2020 ◽  
Vol 8 (9) ◽  
pp. e17818 ◽  
Author(s):  
Madeena Sultana ◽  
Majed Al-Jefri ◽  
Joon Lee
Keyword(s):  
Machine Learning ◽  
Emotional State ◽  
Contextual Information ◽  
Classification Problem ◽  
Emotional States ◽  
Discrete Emotions ◽  
Location Services ◽  
State Detection ◽  
Everyday Context ◽  
The Relationship

Background Emotional state in everyday life is an essential indicator of health and well-being. However, daily assessment of emotional states largely depends on active self-reports, which are often inconvenient and prone to incomplete information. Automated detection of emotional states and transitions on a daily basis could be an effective solution to this problem. However, the relationship between emotional transitions and everyday context remains to be unexplored. Objective This study aims to explore the relationship between contextual information and emotional transitions and states to evaluate the feasibility of detecting emotional transitions and states from daily contextual information using machine learning (ML) techniques. Methods This study was conducted on the data of 18 individuals from a publicly available data set called ExtraSensory. Contextual and sensor data were collected using smartphone and smartwatch sensors in a free-living condition, where the number of days for each person varied from 3 to 9. Sensors included an accelerometer, a gyroscope, a compass, location services, a microphone, a phone state indicator, light, temperature, and a barometer. The users self-reported approximately 49 discrete emotions at different intervals via a smartphone app throughout the data collection period. We mapped the 49 reported discrete emotions to the 3 dimensions of the pleasure, arousal, and dominance model and considered 6 emotional states: discordant, pleased, dissuaded, aroused, submissive, and dominant. We built general and personalized models for detecting emotional transitions and states every 5 min. The transition detection problem is a binary classification problem that detects whether a person’s emotional state has changed over time, whereas state detection is a multiclass classification problem. In both cases, a wide range of supervised ML algorithms were leveraged, in addition to data preprocessing, feature selection, and data imbalance handling techniques. Finally, an assessment was conducted to shed light on the association between everyday context and emotional states. Results This study obtained promising results for emotional state and transition detection. The best area under the receiver operating characteristic (AUROC) curve for emotional state detection reached 60.55% in the general models and an average of 96.33% across personalized models. Despite the highly imbalanced data, the best AUROC curve for emotional transition detection reached 90.5% in the general models and an average of 88.73% across personalized models. In general, feature analyses show that spatiotemporal context, phone state, and motion-related information are the most informative factors for emotional state and transition detection. Our assessment showed that lifestyle has an impact on the predictability of emotion. Conclusions Our results demonstrate a strong association of daily context with emotional states and transitions as well as the feasibility of detecting emotional states and transitions using data from smartphone and smartwatch sensors.

scholarly journals Modeling Reader's Emotional State Response on Document's Typographic Elements

2011 ◽  
Vol 2011 ◽  
pp. 1-18 ◽  
Author(s):  
Dimitrios Tsonos ◽  
Georgios Kouroupetroglou
Keyword(s):  
Speech Synthesis ◽  
Emotional State ◽  
Emotional States ◽  
Background Color ◽  
Font Size ◽  
Self Assessment ◽  
Type Size ◽  
Discrete Values ◽  
Color Combinations ◽  
Font Type

We present the results of an experimental study towards modeling the reader's emotional state variations induced by the typographic elements in electronic documents. Based on the dimensional theory of emotions we investigate how typographic elements, like font style (bold, italics, bold-italics) and font (type, size, color and background color), affect the reader's emotional states, namely, Pleasure, Arousal, and Dominance (PAD). An experimental procedure was implemented conforming to International Affective Picture System guidelines and incorporating the Self-Assessment Manikin test. Thirty students participated in the experiment. The stimulus was a short paragraph of text for which any content, emotion, and/or domain dependent information was excluded. The Analysis of Variance revealed the dependency of (a) all the three emotional dimensions on font size and font/background color combinations and (b) the Pleasure dimension on font type and font style. We introduce a set of mapping rules showing how PAD vary on the discrete values of font style and font type elements. Moreover, we introduce a set of equations describing the PAD dimensions' dependency on font size. This novel model can contribute to the automated reader's emotional state extraction in order, for example, to enhance the acoustic rendition of the documents, utilizing text-to-speech synthesis.

scholarly journals Study of pregnant women’s emotional state

2020 ◽  
Author(s):  
Liydmila V. Tokarskaya ◽  
◽  
Anastasia S. Kolchurina ◽  
Maria A. Lavrova ◽  
Valeria V. Lapteva ◽  
...  
Keyword(s):  
Pregnant Women ◽  
Emotional State ◽  
Previous Experience ◽  
Emotional States ◽  
Emotional Instability ◽  
Self Assessment ◽  
Meaningful Life ◽  
History Of

The article discusses how the emotional state of pregnant women is influenced by their previous experience of pregnancy. The study relies on the following methods: ‘Test of Pregnant Woman’s Relations’ by I.V Dobryakova; ‘Self. Assessment of Emotional States’ by A. Wessman and D. Ricks; “Self. Estimate” by T. Dembo and S. Ya. Rubinshtein (modified by P. V. Yanshin); “Test of Meaningful Life Orientations” by D. Krambo and L. Makholikh (adapted by D. A. Leontyev). The study has shown that in the presence of complications and pathologies — in the form of a history of miscarriage — the emotional sphere of a woman will be characterized by emotional instability, increased anxiety and low self.esteem. Emotional instability is typical of pregnancy in general and it often is accompanied by dependence on others, distrustfulness, fatigue, vulnerability, impressionability combined with excitement, anxiety, and some fear.

Effect of Emotions in a Lexical Decision Task

2017 ◽  
Vol 76 (2) ◽  
pp. 71-79 ◽  
Author(s):  
Hélène Maire ◽  
Renaud Brochard ◽  
Jean-Luc Kop ◽  
Vivien Dioux ◽  
Daniel Zagar
Keyword(s):  
Lexical Decision ◽  
Lexical Decision Task ◽  
Emotional State ◽  
Control Group ◽  
Decision Task ◽  
Attentional Orienting ◽  
Emotional States ◽  
Sequence Effect ◽  
Emotion Induction ◽  
Before And After

Abstract. This study measured the effect of emotional states on lexical decision task performance and investigated which underlying components (physiological, attentional orienting, executive, lexical, and/or strategic) are affected. We did this by assessing participants’ performance on a lexical decision task, which they completed before and after an emotional state induction task. The sequence effect, usually produced when participants repeat a task, was significantly smaller in participants who had received one of the three emotion inductions (happiness, sadness, embarrassment) than in control group participants (neutral induction). Using the diffusion model ( Ratcliff, 1978 ) to resolve the data into meaningful parameters that correspond to specific psychological components, we found that emotion induction only modulated the parameter reflecting the physiological and/or attentional orienting components, whereas the executive, lexical, and strategic components were not altered. These results suggest that emotional states have an impact on the low-level mechanisms underlying mental chronometric tasks.

MACHINE LEARNING FOR SOFTWARE

2019 ◽  
pp. 251-255
Author(s):  
Shafagat Mahmudova
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Soft Computing ◽  
Computing Technology

The study machine learning for software based on Soft Computing technology. It analyzes Soft Computing components. Their use in software, their advantages and challenges are studied. Machine learning and its features are highlighted. The functions and features of neural networks are clarified, and recommendations were given.

Sign in / Sign up

Export Citation Format

Share Document