scholarly journals Keep Me in the Loop: Real-Time Feedback with Multimodal Data

2021 ◽  
Author(s):  
Daniele Di Mitri ◽  
Jan Schneider ◽  
Hendrik Drachsler
Keyword(s):  
Field Study ◽  
Real Time ◽  
Performance Indicators ◽  
User Study ◽  
Methodological Approach ◽  
Feedback System ◽  
Multimodal Feedback ◽  
Multimodal Data ◽  
Cpr Training ◽  
The Impact

AbstractThis paper describes the CPR Tutor, a real-time multimodal feedback system for cardiopulmonary resuscitation (CPR) training. The CPR Tutor detects training mistakes using recurrent neural networks. The CPR Tutor automatically recognises and assesses the quality of the chest compressions according to five CPR performance indicators. It detects training mistakes in real-time by analysing a multimodal data stream consisting of kinematic and electromyographic data. Based on this assessment, the CPR Tutor provides audio feedback to correct the most critical mistakes and improve the CPR performance. The mistake detection models of the CPR Tutor were trained using a dataset from 10 experts. Hence, we tested the validity of the CPR Tutor and the impact of its feedback functionality in a user study involving additional 10 participants. The CPR Tutor pushes forward the current state of the art of real-time multimodal tutors by providing: (1) an architecture design, (2) a methodological approach for delivering real-time feedback using multimodal data and (3) a field study on real-time feedback for CPR training. This paper details the results of a field study by quantitatively measuring the impact of the CPR Tutor feedback on the performance indicators and qualitatively analysing the participants’ questionnaire answers.

scholarly journals Detecting Mistakes in CPR Training with Multimodal Data and Neural Networks

Sensors ◽  
2019 ◽  
Vol 19 (14) ◽  
pp. 3099 ◽  
Author(s):  
Daniele Di Mitri ◽  
Jan Schneider ◽  
Marcus Specht ◽  
Hendrik Drachsler
Keyword(s):  
Neural Networks ◽  
Body Weight ◽  
Performance Indicators ◽  
Research Team ◽  
Body Position ◽  
Microsoft Kinect ◽  
Chest Compressions ◽  
Multimodal Data ◽  
Myo Armband ◽  
Cpr Training

This study investigated to what extent multimodal data can be used to detect mistakes during Cardiopulmonary Resuscitation (CPR) training. We complemented the Laerdal QCPR ResusciAnne manikin with the Multimodal Tutor for CPR, a multi-sensor system consisting of a Microsoft Kinect for tracking body position and a Myo armband for collecting electromyogram information. We collected multimodal data from 11 medical students, each of them performing two sessions of two-minute chest compressions (CCs). We gathered in total 5254 CCs that were all labelled according to five performance indicators, corresponding to common CPR training mistakes. Three out of five indicators, CC rate, CC depth and CC release, were assessed automatically by the ReusciAnne manikin. The remaining two, related to arms and body position, were annotated manually by the research team. We trained five neural networks for classifying each of the five indicators. The results of the experiment show that multimodal data can provide accurate mistake detection as compared to the ResusciAnne manikin baseline. We also show that the Multimodal Tutor for CPR can detect additional CPR training mistakes such as the correct use of arms and body weight. Thus far, these mistakes were identified only by human instructors. Finally, to investigate user feedback in the future implementations of the Multimodal Tutor for CPR, we conducted a questionnaire to collect valuable feedback aspects of CPR training.

scholarly journals Real time system for human action evaluation

2021 ◽  
Author(s):  
Randy Tan
Keyword(s):  
Real Time ◽  
Visual Feedback ◽  
Ground Truth ◽  
Feedback System ◽  
Human Action ◽  
Complete Sequence ◽  
Real Time System ◽  
Analysis System ◽  
Value Decomposition ◽  
The Impact

This thesis presents a real-time human activity analysis system, where a user’s activity can be quantitatively evaluated with respect to a ground truth recording. Multiple Kinects are used to solve the problem of self-occlusion while performing an activity. The Kinects are placed in locations with different perspectives to extract the optimal joint positions of a user using Singular Value Decomposition (SVD) and Sequential Quadratic Programming (SQP). The extracted joint positions are then fed through our Incremental Dynamic Time Warping (IDTW) algorithm so that an incomplete sequence of an user can be optimally compared against the complete sequence from an expert (ground truth). Furthermore, the user’s performance is communicated through a novel visual feedback system, where colors on the skeleton present the user’s level of performance. Experimental results demonstrate the impact of our system, where through elaborate user testing we show that our IDTW algorithm combined with visual feedback improves the user’s performance quantitatively.

The immediate effect of deliberate practice and real-time feedback on high-quality CPR training in intern doctors, acute care providers, and lay rescuers

2021 ◽  
Keyword(s):  
Real Time ◽  
Acute Care ◽  
Deliberate Practice ◽  
Acquisition Training ◽  
Skills Acquisition ◽  
Care Providers ◽  
High Quality ◽  
Training Outcomes ◽  
Cpr Training ◽  
The Impact

The quality of cardiopulmonary resuscitation (CPR) is the main determinant of survival in cardiac arrest, so high-quality CPR (HQ-CPR) from bystanders is essential. The best instructional model for HQ-CPR performed by bystanders remains under investigation, and an instructional model’s effect on various learner types is unknown. This study examined the immediate effect of a brief, blended instructional design that combines deliberate practice (DP) with real-time feedback (RTF) on the booster training of intern doctors (IDs) and acute care providers (ACPs) as well as on the skills acquisition training of lay rescuers (LRs). This cohort crossover study was conducted in a university-affiliated hospital in January 2020. Just-in-time training on HQ-CPR that featured a popular song was provided to IDs (n = 24), ACPs (n = 29), LRs (n = 25); groups performed one-minute cardiac compressions twice, without RTF and with verbal coaching, followed by debriefing, and then with only RTF. The impact of RTF on depth, rate, compression quality (CQ), and recoil was assessed. RTF significantly improved depth, rate, CQ, and recoil (p < 0.001). Among the LRs, the depth was 0.2 millimeters below the lower cutoff. Without RTF, the previously trained IDs and ACPs tended to perform inadequately faster and deeper compressions, while the untrained LRs performed slower, shallow compressions. DP combined with RTF yielded a significant immediate effect on the HQ-CPR training outcomes of all learner types.

scholarly journals The Impact of the Coronavirus Pandemic on US healthcare workers: A Discursive Analysis of Frontline Secondary Traumatic Stress Using Twitter Data

2021 ◽  
Author(s):  
Lisa Wolf ◽  
Cydne Perhats ◽  
Altair Delao ◽  
Hannah S Noblewolf
Keyword(s):  
Discourse Analysis ◽  
Real Time ◽  
Traumatic Stress ◽  
Secondary Traumatic Stress ◽  
Healthcare Workers ◽  
Methodological Approach ◽  
High Stress ◽  
Community Setting ◽  
Post Traumatic Stress ◽  
The Impact

The demands of the COVID-19 pandemic have resulted in an increased physical, clinical, and emotional workload for healthcare workers. Both the Veterans Affairs system and the Emergency Nurses Association have recognized the specific hazards and health risks of providing frontline care in this unprecedented global emergency including an increase in multiple factors associated with post-traumatic stress disorder (PTSD). The purpose of this study was to explore real time PTSD in US healthcare workers using Twitter posts and to describe the impact on healthcare workers as the pandemic unfolds across the US. 1000 tweets were randomly selected from a larger dataset of 443,918 tweets by 281,021 unique authors posted using the hashtags #getmePPE and #getusPPE. Directed content analysis and discourse analysis were used to analyze the tweets and place them into a larger conversation about the pandemic. Healthcare workers and others, using a digital community setting delineated by the hashtags #GetUSPPE and #GetmePPE created a conversation centering around fear of illness, alarm at pandemic spread, and frustration at the label of “hero”, which is unsupported by resources at the local or Federal level. healthcare workers as a group have grave concerns, and high stress levels about inadequate material support (specifically personal protective equipment, or “PPE”) during the first 3 months of the Covid-19 pandemic. Real time analysis using social media posts as a dataset is a useful and feasible methodological approach for explicating the healthcare discourse within the social and political context of this pandemic. Keywords: emergency care; secondary traumatic stress; Covid-19; discourse analysis; emotional workload

scholarly journals Table Tennis Tutor: Forehand Strokes Classification Based on Multimodal Data and Neural Networks

Sensors ◽  
2021 ◽  
Vol 21 (9) ◽  
pp. 3121
Author(s):  
Khaleel Asyraaf Mat Sanusi ◽  
Daniele Di Mitri ◽  
Bibeg Limbu ◽  
Roland Klemke
Keyword(s):  
Real Time ◽  
Body Position ◽  
Feedback System ◽  
Microsoft Kinect ◽  
Psychomotor Skills ◽  
Table Tennis ◽  
Multimodal Data ◽  
Interview Session ◽  
Expert Interview ◽  
Smartphone Sensors

Beginner table-tennis players require constant real-time feedback while learning the fundamental techniques. However, due to various constraints such as the mentor’s inability to be around all the time, expensive sensors and equipment for sports training, beginners are unable to get the immediate real-time feedback they need during training. Sensors have been widely used to train beginners and novices for various skills development, including psychomotor skills. Sensors enable the collection of multimodal data which can be utilised with machine learning to classify training mistakes, give feedback, and further improve the learning outcomes. In this paper, we introduce the Table Tennis Tutor (T3), a multi-sensor system consisting of a smartphone device with its built-in sensors for collecting motion data and a Microsoft Kinect for tracking body position. We focused on the forehand stroke mistake detection. We collected a dataset recording an experienced table tennis player performing 260 short forehand strokes (correct) and mimicking 250 long forehand strokes (mistake). We analysed and annotated the multimodal data for training a recurrent neural network that classifies correct and incorrect strokes. To investigate the accuracy level of the aforementioned sensors, three combinations were validated in this study: smartphone sensors only, the Kinect only, and both devices combined. The results of the study show that smartphone sensors alone perform sub-par than the Kinect, but similar with better precision together with the Kinect. To further strengthen T3’s potential for training, an expert interview session was held virtually with a table tennis coach to investigate the coach’s perception of having a real-time feedback system to assist beginners during training sessions. The outcome of the interview shows positive expectations and provided more inputs that can be beneficial for the future implementations of the T3.

scholarly journals Characterization of Infant Cardiopulmonary Resuscitation Delivery with Range Sensor Feedback on Performance

2021 ◽  
Vol 11 (21) ◽  
pp. 9813
Author(s):  
Farah M. Alkhafaji ◽  
Ghaidaa A. Khalid ◽  
Ali Al-Naji ◽  
Basheer M. Hussein ◽  
Javaan Chahl
Keyword(s):  
Cardiopulmonary Resuscitation ◽  
Real Time ◽  
Survival Rates ◽  
Feedback System ◽  
High Quality ◽  
Compression Technique ◽  
The Social ◽  
Cpr Training

Cardiac arrest (CA) in infants is an issue worldwide, which causes significant morbidity and mortality rates. Cardiopulmonary resuscitation (CPR) is a technique performed in case of CA to save victims' lives. However, CPR is often not performed effectively, even when delivered by qualified rescuers. Therefore, international guidelines have proposed applying a CPR feedback device to achieve high-quality application of CPR to enhance survival rates. Currently, no feedback device is available to guide learners through infant CPR performance in contrast to a number of adult CPR feedback devices. This study presents a real-time feedback system to improve infant CPR performance by medical staff and laypersons using a commercial CPR infant manikin. The proposed system uses an IR sensor to compare CPR performance obtained with no feedback and with a real-time feedback system. Performance was validated by analysis of the CPR parameters actually delivered against the recommended target parameters. Results show that the real-time feedback system significantly improves the quality of chest compression parameters. The two-thumb compression technique is the achievable and appropriate mechanism applied to infant subjects for delivering high-quality CPR. Under the social distancing constraints imposed by the SARS-CoV-2 pandemic, the results from the training device were sent to a CPR training center and provided each participant with CPR proficiency.

Facilitating the Musician's Engagement with New Musical Interfaces: Counteractions in Music Performance

2017 ◽  
Vol 41 (2) ◽  
pp. 69-82 ◽  
Author(s):  
Koray Tahiroğlu ◽  
Juan Carlos Vasquez ◽  
Johan Kildal
Keyword(s):  
Real Time ◽  
User Study ◽  
Music Performance ◽  
Musical Instrument ◽  
Sound Design ◽  
Audio Analysis ◽  
Degree Of Satisfaction ◽  
The Impact ◽  
Design Comparison

The level of engagement of a musician performing on an instrument is related to the degree of satisfaction derived from that activity. With our work, we aim to assist musicians performing live on a new musical instrument, Network of Interactive Sonic Agents (NOISA), by helping them maintain or increase their level of engagement with the activity. The NOISA system can learn from performers through observation and estimate their engagement level in real time. The new response module, which includes new sound design, comparison of gestures, and audio-analysis features, can also decide what action to take, and when to implement it, to help the performer recover from lowering engagement levels. We report on a formative user study that evaluates the impact of this response module.

scholarly journals Real time system for human action evaluation

2021 ◽  
Author(s):  
Randy Tan
Keyword(s):  
Real Time ◽  
Visual Feedback ◽  
Ground Truth ◽  
Feedback System ◽  
Human Action ◽  
Complete Sequence ◽  
Real Time System ◽  
Analysis System ◽  
Value Decomposition ◽  
The Impact

This thesis presents a real-time human activity analysis system, where a user’s activity can be quantitatively evaluated with respect to a ground truth recording. Multiple Kinects are used to solve the problem of self-occlusion while performing an activity. The Kinects are placed in locations with different perspectives to extract the optimal joint positions of a user using Singular Value Decomposition (SVD) and Sequential Quadratic Programming (SQP). The extracted joint positions are then fed through our Incremental Dynamic Time Warping (IDTW) algorithm so that an incomplete sequence of an user can be optimally compared against the complete sequence from an expert (ground truth). Furthermore, the user’s performance is communicated through a novel visual feedback system, where colors on the skeleton present the user’s level of performance. Experimental results demonstrate the impact of our system, where through elaborate user testing we show that our IDTW algorithm combined with visual feedback improves the user’s performance quantitatively.

Studying the Impacts of Local Oceanographic Processes on Adélie Penguin Foraging Ecology

2014 ◽  
Vol 48 (5) ◽  
pp. 25-34 ◽  
Author(s):  
Josh Kohut ◽  
Kim Bernard ◽  
William Fraser ◽  
Matthew J. Oliver ◽  
Hank Statscewich ◽  
...  
Keyword(s):  
Field Study ◽  
Real Time ◽  
Antarctic Krill ◽  
Foraging Ecology ◽  
Surface Current ◽  
Adélie Penguin ◽  
Adelie Penguin ◽  
Convergence And Divergence ◽  
The Impact

AbstractWe are conducting a multi-platform field study to investigate the impact of local physical processes on Adélie penguin foraging ecology in the vicinity of Palmer Deep off Anvers Island, Western Antarctic Peninsula (WAP). Guided by real-time remotely sensed surface current measurements of convergence derived from a network of high-frequency radars (HFRs), we adaptively sample the distribution and biomass of phytoplankton and Antarctic krill, which influence Adélie penguin foraging ecology, to understand how local oceanographic processes structure the ecosystem. The recent application of ocean observing and animal telemetry technology over Palmer Deep has led to new understanding and many new questions related to polar ecosystem processes. The HFR coastal surface current mapping network is uniquely equipped to resolve local circulation patterns over Palmer Deep. The surface current measurements enable identification of regions of convergence and divergence in real time. Guided by these maps, our field study adaptively samples the measured convergence and divergence zones within the context of semi-diurnal and diurnal mixed tidal regimes. The in situ sampling includes (a) a mooring deployment, (b) multiple underwater glider deployments, (c) small boat acoustic surveys of Antarctic krill, and (d) penguin ARGOS-linked satellite telemetry and temperature-depth recorders (TDRs). The combination of real-time surface convergence maps with adaptive in situ sampling introduces HFR to the Antarctic in a way that allows us to rigorously and efficiently test the influence of local tidal processes on top predator foraging ecology.

The methodological approach for the determination of the pressure of artificial raindrops as a question of the theory of splash soil erosion

2020 ◽  
pp. 24-28
Author(s):  
Mihail Zver'kov
Keyword(s):  
Block Diagram ◽  
Methodological Approach ◽  
Control Measures ◽  
Soil Conditions ◽  
Effective Pressure ◽  
Long Distance ◽  
Splash Erosion ◽  
Rain Drops ◽  
Artificial Rain ◽  
The Impact

To the article the results of the theoretical and experimental researches are given on questions of estimates of the dynamic rate effect of raindrop impact on soil. The aim of this work was to analyze the current methods to determine the rate of artificial rain pressure on the soil for the assessment of splash erosion. There are the developed author’s method for calculation the pressure of artificial rain on the soil and the assessment of splash erosion. The study aims to the justification of evaluation methods and the obtaining of quantitative characteristics, prevention and elimination of accelerated (anthropogenic) erosion, the creation and the realization of the required erosion control measures. The paper considers the question of determining the pressure of artificial rain on the soil. At the moment of raindrops impact, there is the tension in the soil, which is called vertical effective pressure. It is noted that the impact of rain drops in the soil there are stresses called vertical effective pressure. The equation for calculation of vertical effective pressure is proposed in this study using the known spectrum of raindrops. Effective pressure was 1.4 Pa for the artificial rain by sprinkler machine «Fregat» and 5.9 Pa for long distance sprinkler DD-30. The article deals with a block diagram of the sequence for determining the effective pressure of rain drops on the soil. This diagram was created by the author’s method of calculation of the effective pressure of rain drops on the soil. The need for an integrated approach to the description of the artificial rain impact on the soil is noted. Various parameters characterizing drop erosion are considered. There are data about the mass of splashed soil in the irrigation of various irrigation machinery and installations. For example, the rate (mass) of splashed soil was 0.28…0.78 t/ha under irrigation sprinkler apparatus RACO 4260–55/701C in the conditions of the Ryazan region. The method allows examining the environmental impact of sprinkler techniques for analyzes of the pressure, caused by raindrops, on the soil. It can also be useful in determining the irrigation rate before the runoff for different types of sprinkler equipment and soil conditions.

Sign in / Sign up

Export Citation Format

Share Document