scholarly journals Longitudinal Classification of Mental Effort Using Electrodermal Activity, Heart Rate, and Skin Temperature Data from a Wearable Sensor

2021 ◽  
Author(s):  
William Romine ◽  
Noah Schroeder ◽  
Anjali Edwards ◽  
Tanvi Banerjee
Keyword(s):  
Heart Rate ◽  
Skin Temperature ◽  
Electrodermal Activity ◽  
Wearable Sensors ◽  
Mental Effort ◽  
Physiological Data ◽  
Cognitive State ◽  
Switching Regression ◽  
Longitudinal Case Study ◽  
Time Duration

Recent studies show that physiological data can detect changes in mental effort, making way for the development of wearable sensors to monitor mental effort in school, work, and at home. We have yet to explore how such a device would work with a single participant over an extended time duration. We used a longitudinal case study design with ~38 hours of data to explore the efficacy of electrodermal activity, skin temperature, and heart rate for classifying mental effort. We utilized a 2-state Markov switching regression model to understand the efficacy of these physiological measures for predicting self-reported mental effort during logged activities. On average, a model with state-dependent relationships predicted within one unit of reported mental effort (training RMSE = 0.4, testing RMSE = 0.7). This automated sensing of mental effort can have applications in various domains including student engagement detection and cognitive state assessment in drivers, pilots, and caregivers.

scholarly journals Longitudinal Classification of Mental Effort Using Electrodermal Activity, Heart Rate, and Skin Temperature Data from a Wearable Sensor

2021 ◽  
Author(s):  
William Romine ◽  
Noah Schroeder ◽  
Anjali Edwards ◽  
Tanvi Banerjee
Keyword(s):  
Heart Rate ◽  
Skin Temperature ◽  
Electrodermal Activity ◽  
Wearable Sensors ◽  
Mental Effort ◽  
Physiological Data ◽  
Cognitive State ◽  
Switching Regression ◽  
Longitudinal Case Study ◽  
Time Duration

Recent studies show that physiological data can detect changes in mental effort, making way for the development of wearable sensors to monitor mental effort in school, work, and at home. We have yet to explore how such a device would work with a single participant over an extended time duration. We used a longitudinal case study design with ~38 hours of data to explore the efficacy of electrodermal activity, skin temperature, and heart rate for classifying mental effort. We utilized a 2-state Markov switching regression model to understand the efficacy of these physiological measures for predicting self-reported mental effort during logged activities. On average, a model with state-dependent relationships predicted within one unit of reported mental effort (training RMSE = 0.4, testing RMSE = 0.7). This automated sensing of mental effort can have applications in various domains including student engagement detection and cognitive state assessment in drivers, pilots, and caregivers.

scholarly journals Sensor-Based Prediction of Mental Effort during Learning from Physiological Data: A Longitudinal Case Study

Signals ◽  
2021 ◽  
Vol 2 (4) ◽  
pp. 886-901
Author(s):  
Ankita Agarwal ◽  
Josephine Graft ◽  
Noah Schroeder ◽  
William Romine
Keyword(s):  
Short Term Memory ◽  
Electrodermal Activity ◽  
Tracking System ◽  
Learning Algorithms ◽  
Mental Effort ◽  
Machine Learning Algorithms ◽  
Self Report ◽  
Physiological Data ◽  
Longitudinal Case Study

Trackers for activity and physical fitness have become ubiquitous. Although recent work has demonstrated significant relationships between mental effort and physiological data such as skin temperature, heart rate, and electrodermal activity, we have yet to demonstrate their efficacy for the forecasting of mental effort such that a useful mental effort tracker can be developed. Given prior difficulty in extracting relationships between mental effort and physiological responses that are repeatable across individuals, we make the case that fusing self-report measures with physiological data within an internet or smartphone application may provide an effective method for training a useful mental effort tracking system. In this case study, we utilized over 90 h of data from a single participant over the course of a college semester. By fusing the participant’s self-reported mental effort in different activities over the course of the semester with concurrent physiological data collected with the Empatica E4 wearable sensor, we explored questions around how much data were needed to train such a device, and which types of machine-learning algorithms worked best. We concluded that although baseline models such as logistic regression and Markov models provided useful explanatory information on how the student’s physiology changed with mental effort, deep-learning algorithms were able to generate accurate predictions using the first 28 h of data for training. A system that combines long short-term memory and convolutional neural networks is recommended in order to generate smooth predictions while also being able to capture transitions in mental effort when they occur in the individual using the device.

Quantifying Occupational Stress in Intensive Care Unit Nurses: An Applied Naturalistic Study of Correlations Among Stress, Heart Rate, Electrodermal Activity, and Skin Temperature

2021 ◽  
pp. 001872082110408
Author(s):  
Nima Ahmadi ◽  
Farzan Sasangohar ◽  
Tariq Nisar ◽  
Valerie Danesh ◽  
Ethan Larsen ◽  
...  
Keyword(s):  
Intensive Care Unit ◽  
Heart Rate ◽  
Intensive Care ◽  
Skin Temperature ◽  
Electrodermal Activity ◽  
Laboratory Research ◽  
Prior Literature ◽  
High Level ◽  
Intensive Care Unit Nurses ◽  
Skin Temperatures

Objective To identify physiological correlates to stress in intensive care unit nurses. Background Most research on stress correlates are done in laboratory environments; naturalistic investigation of stress remains a general gap. Method Electrodermal activity, heart rate, and skin temperatures were recorded continuously for 12-hr nursing shifts (23 participants) using a wrist-worn wearable technology (Empatica E4). Results Positive correlations included stress and heart rate (ρ = .35, p < .001), stress and skin temperature (ρ = .49, p < .05), and heart rate and skin temperatures (ρ = .54, p = .0008). Discussion The presence and direction of some correlations found in this study differ from those anticipated from prior literature, illustrating the importance of complementing laboratory research with naturalistic studies. Further work is warranted to recognize nursing activities associated with a high level of stress and the underlying reasons associated with changes in physiological responses. Application Heart rate and skin temperature may be used for real-time detection of stress, but more work is needed to validate such surrogate measures.

scholarly journals Quantitative assessment of the relationship between behavioral and autonomic dynamics during propofol-induced unconsciousness

2020 ◽  
Author(s):  
Sandya Subramanian ◽  
Patrick L. Purdon ◽  
Riccardo Barbieri ◽  
Emery N. Brown
Keyword(s):  
Heart Rate ◽  
Heart Rate Variability ◽  
Prediction Models ◽  
Electrodermal Activity ◽  
Strong Relationship ◽  
Pulse Amplitude ◽  
Physiological Data ◽  
Statistical Tool ◽  
Autonomic Changes ◽  
Before And After

ABSTRACTDuring general anesthesia, both behavioral and autonomic changes are caused by the administration of anesthetics such as propofol. Propofol produces unconsciousness by creating highly structured oscillations in brain circuits. The anesthetic also has autonomic effects due to its actions as a vasodilator and myocardial depressant. Understanding how autonomic dynamics change in relation to propofol-induced unconsciousness is an important scientific and clinical question since anesthesiologists often infer changes in level of unconsciousness from changes in autonomic dynamics. Therefore, we present a framework combining physiology-based statistical models that have been developed specifically for heart rate variability and electrodermal activity with a robust statistical tool to compare behavioral and multimodal autonomic changes before, during, and after propofol-induced unconsciousness. We tested this framework on physiological data recorded from nine healthy volunteers during computer-controlled administration of propofol. We studied how autonomic dynamics related to behavioral markers of unconsciousness: 1) overall, 2) during the transitions of loss and recovery of consciousness, and 3) before and after anesthesia as a whole. Our results show a strong relationship between behavioral state of consciousness and autonomic dynamics. All of our prediction models showed areas under the curve greater than 0.75 despite the presence of non-monotonic relationships among the variables during the transition periods. Our analysis highlighted the specific roles played by fast versus slow changes, parasympathetic vs sympathetic activity, heart rate variability vs electrodermal activity, and even pulse rate vs pulse amplitude information within electrodermal activity. Further advancement upon this work can quantify the complex and subject-specific relationship between behavioral changes and autonomic dynamics before, during, and after anesthesia. However, this work demonstrates the potential of a multimodal, physiologically-informed, statistical approach to characterize autonomic dynamics.

scholarly journals Evaluating physiological signal salience for estimating metabolic energy cost from wearable sensors

2019 ◽  
Vol 126 (3) ◽  
pp. 717-729 ◽  
Author(s):  
Kimberly A. Ingraham ◽  
Daniel P. Ferris ◽  
C. David Remy
Keyword(s):  
Heart Rate ◽  
Steady State ◽  
Indirect Calorimetry ◽  
Energy Cost ◽  
Electrodermal Activity ◽  
Wearable Sensors ◽  
Metabolic Energy ◽  
Physiological Signals ◽  
Physiological Signal ◽  
Signal Salience

Body-in-the-loop optimization algorithms have the capability to automatically tune the parameters of robotic prostheses and exoskeletons to minimize the metabolic energy expenditure of the user. However, current body-in-the-loop algorithms rely on indirect calorimetry to obtain measurements of energy cost, which are noisy, sparsely sampled, time-delayed, and require wearing a respiratory mask. To improve these algorithms, the goal of this work is to predict a user’s steady-state energy cost quickly and accurately using physiological signals obtained from portable, wearable sensors. In this paper, we quantified physiological signal salience to discover which signals, or groups of signals, have the best predictive capability when estimating metabolic energy cost. We collected data from 10 healthy individuals performing 6 activities (walking, incline walking, backward walking, running, cycling, and stair climbing) at various speeds or intensities. Subjects wore a suite of physiological sensors that measured breath frequency and volume, limb accelerations, lower limb EMG, heart rate, electrodermal activity, skin temperature, and oxygen saturation; indirect calorimetry was used to establish the ‘ground truth’ energy cost for each activity. Evaluating Pearson’s correlation coefficients and single and multiple linear regression models with cross validation (leave-one- subject-out and leave-one- task-out), we found that 1) filtering the accelerations and EMG signals improved their predictive power, 2) global signals (e.g., heart rate, electrodermal activity) were more sensitive to unknown subjects than tasks, while local signals (e.g., accelerations) were more sensitive to unknown tasks than subjects, and 3) good predictive performance was obtained combining a small number of signals (4–5) from multiple sensor modalities. NEW & NOTEWORTHY In this paper, we systematically compare a large set of physiological signals collected from portable sensors and determine which sensor signals contain the most salient information for predicting steady-state metabolic energy cost, robust to unknown subjects or tasks. This information, together with the comprehensive data set that is published in conjunction with this paper, will enable researchers and clinicians across many fields to develop novel algorithms to predict energy cost from wearable sensors.

scholarly journals Quantitative assessment of the relationship between behavioral and autonomic dynamics during propofol-induced unconsciousness

PLoS ONE ◽  
2021 ◽  
Vol 16 (8) ◽  
pp. e0254053
Author(s):  
Sandya Subramanian ◽  
Patrick L. Purdon ◽  
Riccardo Barbieri ◽  
Emery N. Brown
Keyword(s):  
Heart Rate ◽  
Heart Rate Variability ◽  
Prediction Models ◽  
Electrodermal Activity ◽  
Strong Relationship ◽  
Pulse Amplitude ◽  
Physiological Data ◽  
Statistical Tool ◽  
Autonomic Changes ◽  
Before And After

During general anesthesia, both behavioral and autonomic changes are caused by the administration of anesthetics such as propofol. Propofol produces unconsciousness by creating highly structured oscillations in brain circuits. The anesthetic also has autonomic effects due to its actions as a vasodilator and myocardial depressant. Understanding how autonomic dynamics change in relation to propofol-induced unconsciousness is an important scientific and clinical question since anesthesiologists often infer changes in level of unconsciousness from changes in autonomic dynamics. Therefore, we present a framework combining physiology-based statistical models that have been developed specifically for heart rate variability and electrodermal activity with a robust statistical tool to compare behavioral and multimodal autonomic changes before, during, and after propofol-induced unconsciousness. We tested this framework on physiological data recorded from nine healthy volunteers during computer-controlled administration of propofol. We studied how autonomic dynamics related to behavioral markers of unconsciousness: 1) overall, 2) during the transitions of loss and recovery of consciousness, and 3) before and after anesthesia as a whole. Our results show a strong relationship between behavioral state of consciousness and autonomic dynamics. All of our prediction models showed areas under the curve greater than 0.75 despite the presence of non-monotonic relationships among the variables during the transition periods. Our analysis highlighted the specific roles played by fast versus slow changes, parasympathetic vs sympathetic activity, heart rate variability vs electrodermal activity, and even pulse rate vs pulse amplitude information within electrodermal activity. Further advancement upon this work can quantify the complex and subject-specific relationship between behavioral changes and autonomic dynamics before, during, and after anesthesia. However, this work demonstrates the potential of a multimodal, physiologically-informed, statistical approach to characterize autonomic dynamics.

Detecting Subclinical Social Anxiety Using Physiological Data from a Wrist-worn Wearable: A Small-Scale Feasibility Study (Preprint)

2020 ◽  
Author(s):  
Ruksana Shaukat Jali ◽  
Nejra Van Zalk ◽  
David Boyle
Keyword(s):  
Young Adults ◽  
Social Anxiety ◽  
Skin Temperature ◽  
Wearable Sensors ◽  
Supervised Machine Learning ◽  
Physiological Data ◽  
Support Vector ◽  
Socially Anxious ◽  
Anxiety States ◽  
Reactive Anxiety

BACKGROUND Subclinical (i.e., threshold) social anxiety can greatly affect young people’s lives, but existing solutions appear inadequate considering its rising prevalence. Wearable sensors may provide a novel way to detect social anxiety and result in new opportunities for monitoring and treatment that would be greatly beneficial for sufferers, society and healthcare services. Nevertheless, indicators such as skin temperature from wrist-worn sensors have not been used in prior work on physiological social anxiety detection. OBJECTIVE This study aimed to investigate whether subclinical social anxiety in young adults can be detected using physiological data obtained from wearable sensors, including Heart Rate (HR), Skin Temperature (ST) and Electrodermal Activity (EDA). METHODS Young adults (N = 12) with self-reported subclinical social anxiety (measured by the widely used self-reported version of the Liebowitz Social Anxiety Scale, LSAS-SR) participated in an impromptu speech task. Physiological data was collected using an E4 Empatica wearable device. Using the pre-processed data and following a supervised machine learning approach, various classification algorithms such as Support Vector Machine (SVM), Decision Tree, Random Forest and K-Nearest Neighbours (KNN) were used to develop models for three different contexts. Models were trained to (1) classify between baseline and socially anxious states, (2) differentiate between baseline, anticipation anxiety and reactive anxiety states, and (3) classify between social anxiety experienced by individuals with differing social anxiety severity. The predictive capability of the singular modalities was also explored in each of the three supervised learning experiments. The generalisability of the developed models was evaluated using 10-fold cross validation as a performance index. RESULTS With modalities combined, the developed models yielded accuracies between 97.54% and 99.48% when detecting between baseline and socially anxious states. Models trained to differentiate between baseline, anticipation anxiety and reactive anxiety states yielded accuracies between 95.18% and 98.10%. Alongside this, the models developed to detect between social anxiety experienced by individuals with differing anxiety severity scores successfully classified with accuracies between 98.86% and 99.52%. Surprisingly, EDA was identified as the most effective singular modality when differentiating between baseline and social anxiety states, whereas ST was the most effective modality when differentiating between anxiety experienced by individuals with differing social anxiety severity. CONCLUSIONS The results indicate that it is possible to accurately detect social anxiety as well as distinguish between levels of severity in young adults by leveraging physiological data collected from wearable sensors.

With a Little Help from My Friend: Emotional Expressiveness in a Female Digital Human and User Gender Interact to Affect Psychological and Physiological Outcomes (Preprint)

2021 ◽  
Author(s):  
Kate Loveys ◽  
Mark Sagar ◽  
Xueyuan Zhang ◽  
Gregory Fricchione ◽  
Elizabeth Broadbent
Keyword(s):  
Social Support ◽  
Heart Rate ◽  
Skin Temperature ◽  
Emotional Expression ◽  
Electrodermal Activity ◽  
Emotional Expressiveness ◽  
Subjective Stress ◽  
Digital Humans ◽  
Physiological Outcomes ◽  
Digital Human

BACKGROUND Loneliness is a growing public health problem that has been exacerbated in vulnerable groups during the COVID-19 pandemic. Social support interventions have been shown to improve loneliness and may be delivered through technology. Digital humans are a new type of computer agent that show promise as supportive peers in healthcare. For digital humans to be effective and engaging support persons, it is important that they can develop closeness with people. Closeness can be improved by emotional expressiveness, particularly in female relationships. However, it is unknown whether emotional expressiveness improves closeness in relationships with digital humans and affects physiological responses. OBJECTIVE This study investigated whether emotional expression by a digital human can affect psychological and physiological outcomes, and whether the effects are moderated by user gender. METHODS 198 healthy adults (101 females, 95 males, 2 gender-diverse individuals) were block-randomized by gender to complete a 15-minute self-disclosure conversation with a female digital human, in one of six conditions. In these conditions, the digital human varied in modality richness and emotional expression in the face and voice (emotional/ neutral/ no face; emotional/ neutral voice). Perceived loneliness, closeness, social support, caring perceptions, and stress were measured after the interaction. Physiological measures including heart rate, skin temperature, and electrodermal activity were collected during the interaction using an Empatica E4 watch. Three-way factorial ANOVA with post hoc tests were conducted to analyse the effect of face type, voice type, and user gender on outcomes. RESULTS Overall, emotional expression in the voice was associated with greater caring perceptions and physiological arousal during the interaction, and unexpectedly, lower feelings of support. Gender was found to moderate the effect of emotional expressiveness on loneliness, social, and certain physiological outcomes. For females, an emotional voice digital human was associated with improved perceptions of closeness, social support, and caring perceptions, whereas for males, a neutral voice digital human was associated with improvements in closeness, social support, and caring perceptions. For females, a neutral face was associated with lower loneliness and subjective stress compared to no face. Whereas interacting with no face (i.e., a voice only black screen) resulted in lower loneliness and subjective stress for males compared to a neutral or emotional face digital human. No significant results were found for heart rate or skin temperature. However, average electrodermal activity was significantly higher for males while interacting with the emotional voice digital human. CONCLUSIONS Findings suggest that emotional expressiveness in a female digital human has different effects on loneliness, social, and physiological outcomes for males and females. Results may inform the design of digital human support persons, and have theoretical implications. Further research is needed to evaluate how more pronounced emotional facial expressions in a digital human might impact results. CLINICALTRIAL Australia New Zealand Clinical Trials Registry (ANZCTR) registration application Id: 381816

scholarly journals Non-invasive wearables for remote monitoring of HbA1c and glucose variability: proof of concept

2021 ◽  
Vol 9 (1) ◽  
pp. e002027
Author(s):  
Brinnae Bent ◽  
Peter J Cho ◽  
April Wittmann ◽  
Connie Thacker ◽  
Srikanth Muppidi ◽  
...  
Keyword(s):  
Heart Rate ◽  
Skin Temperature ◽  
Validation Cohort ◽  
Electrodermal Activity ◽  
External Validation ◽  
Glucose Variability ◽  
Proof Of Concept ◽  
Concept Study ◽  
Non Invasive ◽  
The Relationship

IntroductionDiabetes prevalence continues to grow and there remains a significant diagnostic gap in one-third of the US population that has pre-diabetes. Innovative, practical strategies to improve monitoring of glycemic health are desperately needed. In this proof-of-concept study, we explore the relationship between non-invasive wearables and glycemic metrics and demonstrate the feasibility of using non-invasive wearables to estimate glycemic metrics, including hemoglobin A1c (HbA1c) and glucose variability metrics.Research design and methodsWe recorded over 25 000 measurements from a continuous glucose monitor (CGM) with simultaneous wrist-worn wearable (skin temperature, electrodermal activity, heart rate, and accelerometry sensors) data over 8–10 days in 16 participants with normal glycemic state and pre-diabetes (HbA1c 5.2–6.4). We used data from the wearable to develop machine learning models to predict HbA1c recorded on day 0 and glucose variability calculated from the CGM. We tested the accuracy of the HbA1c model on a retrospective, external validation cohort of 10 additional participants and compared results against CGM-based HbA1c estimation models.ResultsA total of 250 days of data from 26 participants were collected. Out of the 27 models of glucose variability metrics that we developed using non-invasive wearables, 11 of the models achieved high accuracy (<10% mean average per cent error, MAPE). Our HbA1c estimation model using non-invasive wearables data achieved MAPE of 5.1% on an external validation cohort. The ranking of wearable sensor’s importance in estimating HbA1c was skin temperature (33%), electrodermal activity (28%), accelerometry (25%), and heart rate (14%).ConclusionsThis study demonstrates the feasibility of using non-invasive wearables to estimate glucose variability metrics and HbA1c for glycemic monitoring and investigates the relationship between non-invasive wearables and the glycemic metrics of glucose variability and HbA1c. The methods used in this study can be used to inform future studies confirming the results of this proof-of-concept study.

Sign in / Sign up

Export Citation Format

Share Document