scholarly journals Using Machine Learning and Smartphone and Smartwatch Data to Detect Emotional States and Transitions: Exploratory Study (Preprint)

2020 ◽  
Author(s):  
Madeena Sultana ◽  
Majed Al-Jefri ◽  
Joon Lee
Keyword(s):  
Machine Learning ◽  
Emotional State ◽  
Contextual Information ◽  
Classification Problem ◽  
Emotional States ◽  
Discrete Emotions ◽  
Location Services ◽  
State Detection ◽  
Everyday Context ◽  
The Relationship

BACKGROUND Emotional state in everyday life is an essential indicator of health and well-being. However, daily assessment of emotional states largely depends on active self-reports, which are often inconvenient and prone to incomplete information. Automated detection of emotional states and transitions on a daily basis could be an effective solution to this problem. However, the relationship between emotional transitions and everyday context remains to be unexplored. OBJECTIVE This study aims to explore the relationship between contextual information and emotional transitions and states to evaluate the feasibility of detecting emotional transitions and states from daily contextual information using machine learning (ML) techniques. METHODS This study was conducted on the data of 18 individuals from a publicly available data set called ExtraSensory. Contextual and sensor data were collected using smartphone and smartwatch sensors in a free-living condition, where the number of days for each person varied from 3 to 9. Sensors included an accelerometer, a gyroscope, a compass, location services, a microphone, a phone state indicator, light, temperature, and a barometer. The users self-reported approximately 49 discrete emotions at different intervals via a smartphone app throughout the data collection period. We mapped the 49 reported discrete emotions to the 3 dimensions of the pleasure, arousal, and dominance model and considered 6 emotional states: discordant, pleased, dissuaded, aroused, submissive, and dominant. We built general and personalized models for detecting emotional transitions and states every 5 min. The transition detection problem is a binary classification problem that detects whether a person’s emotional state has changed over time, whereas state detection is a multiclass classification problem. In both cases, a wide range of supervised ML algorithms were leveraged, in addition to data preprocessing, feature selection, and data imbalance handling techniques. Finally, an assessment was conducted to shed light on the association between everyday context and emotional states. RESULTS This study obtained promising results for emotional state and transition detection. The best area under the receiver operating characteristic (AUROC) curve for emotional state detection reached 60.55% in the general models and an average of 96.33% across personalized models. Despite the highly imbalanced data, the best AUROC curve for emotional transition detection reached 90.5% in the general models and an average of 88.73% across personalized models. In general, feature analyses show that spatiotemporal context, phone state, and motion-related information are the most informative factors for emotional state and transition detection. Our assessment showed that lifestyle has an impact on the predictability of emotion. CONCLUSIONS Our results demonstrate a strong association of daily context with emotional states and transitions as well as the feasibility of detecting emotional states and transitions using data from smartphone and smartwatch sensors.

scholarly journals Using Machine Learning and Smartphone and Smartwatch Data to Detect Emotional States and Transitions: Exploratory Study

10.2196/17818 ◽  
2020 ◽  
Vol 8 (9) ◽  
pp. e17818 ◽  
Author(s):  
Madeena Sultana ◽  
Majed Al-Jefri ◽  
Joon Lee
Keyword(s):  
Machine Learning ◽  
Emotional State ◽  
Contextual Information ◽  
Classification Problem ◽  
Emotional States ◽  
Discrete Emotions ◽  
Location Services ◽  
State Detection ◽  
Everyday Context ◽  
The Relationship

Background Emotional state in everyday life is an essential indicator of health and well-being. However, daily assessment of emotional states largely depends on active self-reports, which are often inconvenient and prone to incomplete information. Automated detection of emotional states and transitions on a daily basis could be an effective solution to this problem. However, the relationship between emotional transitions and everyday context remains to be unexplored. Objective This study aims to explore the relationship between contextual information and emotional transitions and states to evaluate the feasibility of detecting emotional transitions and states from daily contextual information using machine learning (ML) techniques. Methods This study was conducted on the data of 18 individuals from a publicly available data set called ExtraSensory. Contextual and sensor data were collected using smartphone and smartwatch sensors in a free-living condition, where the number of days for each person varied from 3 to 9. Sensors included an accelerometer, a gyroscope, a compass, location services, a microphone, a phone state indicator, light, temperature, and a barometer. The users self-reported approximately 49 discrete emotions at different intervals via a smartphone app throughout the data collection period. We mapped the 49 reported discrete emotions to the 3 dimensions of the pleasure, arousal, and dominance model and considered 6 emotional states: discordant, pleased, dissuaded, aroused, submissive, and dominant. We built general and personalized models for detecting emotional transitions and states every 5 min. The transition detection problem is a binary classification problem that detects whether a person’s emotional state has changed over time, whereas state detection is a multiclass classification problem. In both cases, a wide range of supervised ML algorithms were leveraged, in addition to data preprocessing, feature selection, and data imbalance handling techniques. Finally, an assessment was conducted to shed light on the association between everyday context and emotional states. Results This study obtained promising results for emotional state and transition detection. The best area under the receiver operating characteristic (AUROC) curve for emotional state detection reached 60.55% in the general models and an average of 96.33% across personalized models. Despite the highly imbalanced data, the best AUROC curve for emotional transition detection reached 90.5% in the general models and an average of 88.73% across personalized models. In general, feature analyses show that spatiotemporal context, phone state, and motion-related information are the most informative factors for emotional state and transition detection. Our assessment showed that lifestyle has an impact on the predictability of emotion. Conclusions Our results demonstrate a strong association of daily context with emotional states and transitions as well as the feasibility of detecting emotional states and transitions using data from smartphone and smartwatch sensors.

scholarly journals Emotional Status and Productivity: Evidence from the Special Economic Zone in Laos

2020 ◽  
Vol 12 (4) ◽  
pp. 1544 ◽  
Author(s):  
Yoshihiko Kadoya ◽  
Mostafa Saidur Rahim Khan ◽  
Somtip Watanapongvanich ◽  
Punjapol Binnagan
Keyword(s):  
Regression Models ◽  
Emotional State ◽  
Employee Productivity ◽  
Panel Regression ◽  
Emotional States ◽  
Cosmetic Products ◽  
Emotional Status ◽  
Daily Output ◽  
Plastic Toys ◽  
The Relationship

Employee productivity is a well-studied area, which has been explained in various dimensions. However, there is insufficient research on how workers’ on-job emotional status relates to productivity. This study examined the relationship between workers’ emotional states and productivity by assessing on-job emotionality recorded using a specially designed wearable biometric device. The experiment was conducted at KP Beau Lao Co. Ltd., a Japanese plastic toys and cosmetic products company in Savannakhet province in Southwestern Laos. Participants were 15 plastic toy painters. Mental status, daily output, and other issues were recorded for three consecutive working days. Using random effects panel regression models, we examined how productivity, operationalized as the log of daily output, was related to workers’ emotional states, including the amount of time workers reported being happy, angry, relaxed, and sad. We controlled for conversation time, heart rate, and other demographic features. The results revealed that happiness, and no other emotional state, was significantly and positively related to productivity. Such findings suggested that workers’ emotional states must be addressed as part of an organization’s operational strategy to ensure higher productivity.

THE EFFECT OF EMOTIONAL STATE ON WAITING IN DECISION MAKING

2008 ◽  
Vol 36 (5) ◽  
pp. 591-602 ◽  
Author(s):  
Ya-Chung Sun ◽  
Shih-Chia Wu
Keyword(s):  
Decision Making ◽  
Experimental Study ◽  
Emotional State ◽  
Emotional States ◽  
Extra Time ◽  
New Information ◽  
Negative Emotional State ◽  
Current Mood ◽  
The Relationship ◽  
Do So

Previous research has indicated that many people often take extra time to consider existing information. They do so possibly in order to acquire more information, or even to “wait” in the hope that new information may be forthcoming before they make a decision. However, recent studies have provided scant information about how waiting affects a person's choice given different emotional states. In this paper, an experimental study was carried out to demonstrate and explain the relationship between waiting and a person's choice. Results show that when conditions are certain, more people choose to wait – when they are in a positive emotional state – in order to maintain their current mood. However, under either certain or uncertain conditions, when people are in a negative emotional state they prefer to take immediate action rather than wait. The causes and implications of this phenomenon are discussed in relation to the existing literature on emotions and choice.

Relationship between Emotional State and Performance during International Field Hockey Matches

2000 ◽  
Vol 90 (2) ◽  
pp. 691-701 ◽  
Author(s):  
Marc V. Jones ◽  
Roger D. Mace ◽  
Simon Williams
Keyword(s):  
Logistic Regression ◽  
Emotional State ◽  
Emotional States ◽  
Field Hockey ◽  
Long Duration ◽  
Hockey Players ◽  
Competition Period ◽  
And Performance ◽  
The Relationship ◽  
Better Than

The present study examined the relationship between the emotions experienced by 15 international hockey players, both immediately before and during competition, and their performance levels. Data were collected on the players' emotional states using a revised version of the Feelings Scale of Butler, which was completed retrospectively after the match was played. Players reported more annoyance and less tension during the match than before. A logistic regression correctly classified 70.2% of players from the emotional ratings immediately before the match and 85.1% of the players from the ratings during the match as either a good or poor performer. Those individuals who performed well retrospectively reported feeling Nervous and ‘Quick/Alert/Active’ before the game and Confident and Relaxed during the game. The results indicate that emotions fluctuate over the competition period, and in long duration sports assessment of emotion during competition predicts variation in performance better than assessment prior to competition.

Self-Control Skills and Negative Emotional State: A Focus on Hostility

2002 ◽  
Vol 90 (2) ◽  
pp. 627-633 ◽  
Author(s):  
Elaine M. Heiby ◽  
Adela Mearig
Keyword(s):  
College Students ◽  
Emotional State ◽  
Unipolar Depression ◽  
Brief Symptom Inventory ◽  
The Self ◽  
Self Control ◽  
Emotional States ◽  
Negative Emotional State ◽  
The Relationship

The self-control theory of psychopathology has contributed to the understanding and treatment of unipolar depression. This paper explores the relationship between self-control skills as measured by the Frequency of Self-reinforcement Questionnaire and other negative emotional states, with a focus on hostility. In Study 1, scores on the Brief Symptom Inventory were inversely related to self-control skills among a sample of 53 college students, suggesting potential generalizability of the theory. In Study 2, self-control skills were inversely related to hostility, anger, and aggression among a sample of 68 college students. The role of self-control skills in the regulation of hostility deserves further investigation.

scholarly journals Predicting Emotional States Using Behavioral Markers Derived From Passively Sensed Data: Data-Driven Machine Learning Approach

10.2196/24465 ◽  
2021 ◽  
Vol 9 (3) ◽  
pp. e24465
Author(s):  
Emese Sükei ◽  
Agnes Norbury ◽  
M Mercedes Perez-Rodriguez ◽  
Pablo M Olmos ◽  
Antonio Artés
Keyword(s):  
Mental Health ◽  
Machine Learning ◽  
Time Series ◽  
Individual Differences ◽  
Emotional State ◽  
Series Data ◽  
Emotional States ◽  
Missing Observations ◽  
State Prediction ◽  
Behavioral Markers

Background Mental health disorders affect multiple aspects of patients’ lives, including mood, cognition, and behavior. eHealth and mobile health (mHealth) technologies enable rich sets of information to be collected noninvasively, representing a promising opportunity to construct behavioral markers of mental health. Combining such data with self-reported information about psychological symptoms may provide a more comprehensive and contextualized view of a patient’s mental state than questionnaire data alone. However, mobile sensed data are usually noisy and incomplete, with significant amounts of missing observations. Therefore, recognizing the clinical potential of mHealth tools depends critically on developing methods to cope with such data issues. Objective This study aims to present a machine learning–based approach for emotional state prediction that uses passively collected data from mobile phones and wearable devices and self-reported emotions. The proposed methods must cope with high-dimensional and heterogeneous time-series data with a large percentage of missing observations. Methods Passively sensed behavior and self-reported emotional state data from a cohort of 943 individuals (outpatients recruited from community clinics) were available for analysis. All patients had at least 30 days’ worth of naturally occurring behavior observations, including information about physical activity, geolocation, sleep, and smartphone app use. These regularly sampled but frequently missing and heterogeneous time series were analyzed with the following probabilistic latent variable models for data averaging and feature extraction: mixture model (MM) and hidden Markov model (HMM). The extracted features were then combined with a classifier to predict emotional state. A variety of classical machine learning methods and recurrent neural networks were compared. Finally, a personalized Bayesian model was proposed to improve performance by considering the individual differences in the data and applying a different classifier bias term for each patient. Results Probabilistic generative models proved to be good preprocessing and feature extractor tools for data with large percentages of missing observations. Models that took into account the posterior probabilities of the MM and HMM latent states outperformed those that did not by more than 20%, suggesting that the underlying behavioral patterns identified were meaningful for individuals’ overall emotional state. The best performing generalized models achieved a 0.81 area under the curve of the receiver operating characteristic and 0.71 area under the precision-recall curve when predicting self-reported emotional valence from behavior in held-out test data. Moreover, the proposed personalized models demonstrated that accounting for individual differences through a simple hierarchical model can substantially improve emotional state prediction performance without relying on previous days’ data. Conclusions These findings demonstrate the feasibility of designing machine learning models for predicting emotional states from mobile sensing data capable of dealing with heterogeneous data with large numbers of missing observations. Such models may represent valuable tools for clinicians to monitor patients’ mood states.

scholarly journals The Evolution of Language Models Applied to Emotion Analysis of Arabic Tweets

Information ◽  
2021 ◽  
Vol 12 (2) ◽  
pp. 84
Author(s):  
Nora Al-Twairesh
Keyword(s):  
Machine Learning ◽  
Language Processing ◽  
Language Model ◽  
Classification Problem ◽  
Language Models ◽  
Machine Learning Techniques ◽  
Emotional States ◽  
Evolution Of Language ◽  
Emotion Analysis ◽  
Language Representation

The field of natural language processing (NLP) has witnessed a boom in language representation models with the introduction of pretrained language models that are trained on massive textual data then used to fine-tune downstream NLP tasks. In this paper, we aim to study the evolution of language representation models by analyzing their effect on an under-researched NLP task: emotion analysis; for a low-resource language: Arabic. Most of the studies in the field of affect analysis focused on sentiment analysis, i.e., classifying text into valence (positive, negative, neutral) while few studies go further to analyze the finer grained emotional states (happiness, sadness, anger, etc.). Emotion analysis is a text classification problem that is tackled using machine learning techniques. Different language representation models have been used as features for these machine learning models to learn from. In this paper, we perform an empirical study on the evolution of language models, from the traditional term frequency–inverse document frequency (TF–IDF) to the more sophisticated word embedding word2vec, and finally the recent state-of-the-art pretrained language model, bidirectional encoder representations from transformers (BERT). We observe and analyze how the performance increases as we change the language model. We also investigate different BERT models for Arabic. We find that the best performance is achieved with the ArabicBERT large model, which is a BERT model trained on a large dataset of Arabic text. The increase in F1-score was significant +7–21%.

Effects of a Fear Appeal on Arousal, Self-Reported Anxiety, and Attitude towards Smoking

1983 ◽  
Vol 52 (1) ◽  
pp. 139-146 ◽  
Author(s):  
M. Watson ◽  
K. W. Pettingale ◽  
D. Goldstein
Keyword(s):  
Individual Differences ◽  
Emotional State ◽  
The Self ◽  
Fear Appeals ◽  
Fear Appeal ◽  
Emotional States ◽  
Close Correspondence ◽  
The Relationship

This paper describes the effects of an anti-smoking film on level of arousal and anxiety in a group of smokers and nonsmoking control. The aims of the study were threefold: to determine whether a fear appeal of this kind would increase arousal; to examine the relationship between self-reported, behavioral, and somatic responses to this type of fear appeal; and to assess the extent to which responses were influenced by consistent individual differences in the reporting of emotional states. The results indicated that a close correspondence existed between the self-reported and somatic measures of anxiety and that smokers showed a greater increase in anxiety than nonsmokers. Individual differences in reporting of emotional state were not related to somatic responses. Over-all, fear appeals of the type used here may be useful in manipulating level of anxiety and attitudes towards smoking.

scholarly journals Machine Learning and EEG for Emotional State Estimation

2021 ◽  
Author(s):  
Krzysztof Kotowski ◽  
Katarzyna Stapor
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
State Estimation ◽  
Emotional State ◽  
Emotional States ◽  
Eeg Signal ◽  
Self Assessment ◽  
Emotional Perception ◽  
Automatic Systems ◽  
Emotion Models

Defining “emotion” and its accurate measuring is a notorious problem in the psychology domain. It is usually addressed with subjective self-assessment forms filled manually by participants. Machine learning methods and EEG correlates of emotions enable to construction of automatic systems for objective emotion recognition. Such systems could help to assess emotional states and could be used to improve emotional perception. In this chapter, we present a computer system that can automatically recognize an emotional state of a human, based on EEG signals induced by a standardized affective picture database. Based on the EEG signal, trained deep neural networks are then used together with mappings between emotion models to predict the emotions perceived by the participant. This, in turn, can be used for example in validation of affective picture databases standardization.

Sign in / Sign up

Export Citation Format

Share Document