Medical Students’ Non-Technical Skills (Medi-StuNTS): preliminary work developing a behavioural marker system for the non-technical skills of medical students in acute care

2018 ◽  
Vol 5 (3) ◽  
pp. 130-139 ◽  
Author(s):  
Ailsa L Hamilton ◽  
Joanne Kerins ◽  
Marc A MacCrossan ◽  
Victoria R Tallentire
Keyword(s):  
Medical Students ◽  
Acute Care ◽  
Situation Awareness ◽  
Rating Scale ◽  
Medical Training ◽  
Poor Performance ◽  
Technical Skills ◽  
Face Validity ◽  
Self Awareness ◽  
Marker System

IntroductionGood non-technical skills (NTS) are critical to the delivery of high-quality patient care. It is increasingly recognised that training in such skills should be incorporated into primary medical training curricula. This study aimed to develop an NTS behavioural marker system (BMS), specifically applicable to medical students, for use within simulated acute care scenarios.MethodsThe methodology used to develop other BMS was adopted and modified. Following ethical approval, 16 final year medical students participated in acute care simulated scenarios. Semistructured interviews were performed to gauge the understanding of NTS. A panel meeting of subject matter experts was convened to translate key NTS into skill elements and observable behaviours. A second expert panel was consulted to refine aspects of the BMS. Further refinement and initial face validity was undertaken by a third panel of experts using the prototype BMS to observe prerecorded simulation scenarios.ResultsFive categories of NTS were identified: situation awareness, teamwork and communication, decision-making and prioritisation, self-awareness, and escalating care. Observable behaviours in each category describe good and poor performance. Escalating care was identified as a unique component that incorporated behaviours related to each of the other four skill categories. A 5-point rating scale was developed to enable both peer-to-peer and tutor-to–student feedback.ConclusionThe Medi-StuNTS (Medical Students’ Non-Technical Skills) system is the first BMS for the NTS of medical students. It reinforces the importance of escalating care effectively. It provides an exciting opportunity to provide feedback to medical students and may ultimately aid their preparedness for professional practice.

Validity of the Medi-StuNTS behavioural marker system: assessing the non-technical skills of medical students during immersive simulation

2020 ◽  
Vol 7 (1) ◽  
pp. 3-10 ◽  
Author(s):  
Emma Claire Phillips ◽  
Samantha Eve Smith ◽  
Benjamin Clarke ◽  
Ailsa Lauren Hamilton ◽  
Joanne Kerins ◽  
...  
Keyword(s):  
Factor Analysis ◽  
Medical Students ◽  
Acute Care ◽  
Situation Awareness ◽  
Test Performance ◽  
Technical Skills ◽  
Strong Positive Correlation ◽  
Marker System ◽  
Significant Difference ◽  
Better Than

BackgroundThe Medical Students’ Non-Technical Skills (Medi-StuNTS) behavioural marker system (BMS) is the first BMS to be developed specifically for medical students to facilitate training in non-technical skills (NTS) within immersive simulated acute care scenarios. In order to begin implementing the tool in practice, validity evidence must be sought. We aimed to assess the validity of the Medi-StuNTS system with reference to Messick’s contemporary validity framework.MethodsTwo raters marked video-recorded performances of acute care simulation scenarios using the Medi-StuNTS system. Three groups were marked: third-year and fourth-year medical students (novices), final-year medical students (intermediates) and core medical trainees (experts). The scores were used to make assessments of relationships to the variable of clinical experience through expert–novice comparisons, inter-rater reliability, observability, exploratory factor analysis, inter-rater disagreements and differential item functioning.ResultsA significant difference was found between the three groups (p<0.005), with experts scoring significantly better than intermediates (p<0.005) and intermediates scoring significantly better than novices (p=0.001). There was a strong positive correlation between the two raters’ scores (r=0.79), and an inter-rater disagreement of more than one point in less than one-fifth of cases. Across all scenarios, 99.7% of skill categories and 84% of skill elements were observable. Factor analysis demonstrated appropriate grouping of skill elements. Inconsistencies in test performance across learner groups were shown specifically in the skill categories of situation awareness and decision making and prioritisation.ConclusionWe have demonstrated evidence for several aspects of validity of the Medi-StuNTS system when assessing medical students’ NTS during immersive simulation. We can now begin to introduce this system into simulation-based education to maximise NTS training in this group.

scholarly journals Aprendizagem Assistida por Pares: Uma Alternativa Pedagógica no Ensino de Competências a Estudantes de Medicina

2020 ◽  
Vol 33 (11) ◽  
pp. 742
Author(s):  
Joana Fernandes Ribeiro ◽  
Manuel Rosete ◽  
Andreia Teixeira ◽  
Hugo Conceição ◽  
Lèlita Santos
Keyword(s):  
Medical Students ◽  
Rating Scale ◽  
Technical Skills ◽  
Surgical Skills ◽  
Global Rating Scale ◽  
Control Group ◽  
P Value ◽  
Global Rating ◽  
Learning Program ◽  
Peer Assisted Learning

Introduction: Technical skills training is fundamental for clinical practice although poorly emphasised in undergraduate medical curricula. In these circumstances, Peer Assisted Learning methodology has emerged as a valid alternative to overcome this insufficiency. The purpose of this study is to evaluate the impact on students of a Peer Assisted Learning program in basic surgical skills, regarding technical competences and knowledge improvement.Material and Methods: A total of 104 randomly selected third year medical students participated in a workshop delivered by fifth year students. From that total, 34 students were assessed before and after the workshop, using the Objective Structured Assessment of Technical Skills instrument, that consists of a global rating scale and a procedure-specific checklist. Sixth year students (control group) were also assessed in their performance without participating in the workshop. Before workshop versus after workshop Objective Structured Assessment of Technical Skills results were compared using Wilcoxon and McNemar tests. After workshop versus control group Objective Structured Assessment of Technical Skills results were compared using Mann-Whitney, qui-squared test and Fisher’s exact test.Results: For the global rating scale, students obtained an after the workshop score (29.5) that was significantly higher than the before the workshop score (15.5; p-value < 0.001), but no significant differences were found between after the workshop and control group scores (p-value = 0.167). For the procedure-specific checklist, 3rd year students had a substantial positive evolution in all parameters and obtained higher rates of correct achievements compared to the control group.Discussion: The final outcomes demonstrated a significant qualitative and quantitative improvement of knowledge and technical skills, which is in accordance with other literature.Conclusion: This Peer Assisted Learning program revealed promising results concerning improvement of surgical skills in medical students, with little staff faculty contribution and extension to a much broader number of students.

scholarly journals Balancing Forces: Medical Students’ Reflections on Professionalism Challenges and Professional Identity Formation

2021 ◽  
Vol 53 (3) ◽  
pp. 200-206
Author(s):  
Amrapali Maitra ◽  
Steven Lin ◽  
Tracy A. Rydel ◽  
Erika Schillinger
Keyword(s):  
Medical Students ◽  
Professional Identity ◽  
Identity Formation ◽  
Medical Training ◽  
Health Care Team ◽  
Free Text ◽  
Self Awareness ◽  
Theory Approach ◽  
Professional Identity Formation ◽  
Medicine Clerkship

Background and Objectives: Professionalism is essential in medical education, yet how it is embodied through medical students’ lived experiences remains elusive. Little research exists on how students perceive professionalism and the barriers they encounter. This study examines attitudes toward professionalism through students’ written reflections. Methods: Family medicine clerkship students at Stanford University School of Medicine answered the following prompt: “Log a patient encounter in which you experienced a professionalism challenge or improvement opportunity.” We collected and analyzed free-text responses for content and themes using a grounded theory approach. Results: One hundred responses from 106 students generated a total of 168 codes; 13 themes emerged across four domains: challenging patients, interpersonal interactions, self-awareness, and health care team dynamics. The three most frequently occurring themes were interacting with emotional patients, managing expectations in the encounter, and navigating the trainee role. Conclusions: Medical students view professionalism as a balance of forces. While many students conceived of professionalism in relation to patient encounters, they also described how professionalism manifests in inner qualities as well as in health systems. Interpersonal challenges related to communication and agenda-setting are predominant. Systems challenges include not being seen as the “real doctor” and being shaped by team behaviors through the hidden curriculum. Our findings highlight salient professionalism challenges and identity conflicts for medical students and suggest potential educational strategies such as intentional coaching and role-modeling by faculty. Overall, students’ reflections broaden our understanding of professional identity formation in medical training.

Simulation of Pediatric Life-Threatening Emergency Scenarios (SimPLES): The Development, Implementation, and Evaluation of an Undergraduate Immersive Simulation-Based Educational Intervention

2020 ◽  
Vol 19 (3) ◽  
Author(s):  
Prashant Kumar ◽  
Simon Chitnis ◽  
Louise Lawrence ◽  
Niamh Langasco ◽  
Neil McGowan
Keyword(s):  
Medical Students ◽  
Pediatric Patients ◽  
Course Design ◽  
Medical Training ◽  
Technical Skills ◽  
Free Text ◽  
Simulation Based ◽  
Clinical Workplace ◽  
Emergency Scenarios ◽  
The Impact

Background: Clinical exposure to acutely unwell pediatric patients is often limited during undergraduate medical training. Although limited within undergraduate pediatric curricula, simulation-based education (SBE) offers a tool with which such exposure can be achieved in a safe and effective learning environment. This study describes the development, implementation, and evaluation of an undergraduate multi-center immersive pediatric SBE course. Objectives: We sought to increase medical students’ exposure to acutely unwell pediatric patients and provide practical experience to improve their understanding of the importance of non-technical skills within the clinical workplace, such as teamwork, communication skills, leadership, and situational awareness. Furthermore, we sought to evaluate the impact on students’ confidence in assessing and managing acutely unwell pediatric patients while exploring their perceptions and opinions of the course. Methods: The present study included all final-year students who were on a pediatric clinical placement during the study period. Tutorials on pediatric A-E assessment and fluid management were followed by a series of immersive simulation scenarios and structured debriefs, concentrating on the importance of non-technical skills in the clinical workplace. Also, some mini-tutorials were incorporated into the course design, focussing on the technical and pathophysiological aspects of each presentation. We employed a mixed-method research methodology to evaluate the impact of the course. Also, anonymized post-course and free-text feedback was sought to explore students’ experiences and perceptions of the course. Results: All 80 students completed the pre and post-course confidence questionnaires, reporting statistically significant improvements in confidence across all 11 domains tested. Thematic analysis of the qualitative data identified six core themes: knowledge acquisition, the value of debriefing, the importance of non-technical skills, the value of faculty expertise, the value of repeated practice, and opportunities for exposure to pediatrics. The innovative post-scenario mini-tutorials were favorably received. Conclusions: Our study shows that immersive SBE can improve medical students’ confidence in managing pediatric emergency scenarios. Also, incorporating mini-tutorials within an immersive simulation course design in an undergraduate setting can be helpful and well received by students.

scholarly journals Correlation of Medical Knowledge and Non-Technical Skills Assessment in Anesthesia Residents

2020 ◽  
Vol 72 (6) ◽  
pp. 483-487
Author(s):  
Kasana Raksamani ◽  
Tachawan Jirativanont ◽  
Pavinee Sareenun
Keyword(s):  
Decision Making ◽  
Situation Awareness ◽  
Rating Scale ◽  
Technical Skills ◽  
Skills Training ◽  
Skills Assessment ◽  
Task Management ◽  
Technical Skills Assessment ◽  
The Mean ◽  
Anesthesia Residents

Objective: Non-technical skills training and assessment has been implemented in anesthesia residency training program to improve quality of patient care but have not been properly assessed. We hypothesized that trainees with good knowledge correlated with good cognitive parts of non-technical skills.Methods: Seventy anesthesia residents (24 PGY-1, 24 PGY-2 and 22 PGY-3) were assessed for their knowledge by 180-item MCQs, 5 key-feature essay questions, and 18-station OSCE’s. Subsequently, a perioperative anesthesia crisis situation was set up in the simulation lab for all residents and was video recorded. Non-technical skills were assessed by 2 independent trained raters using Anesthetists’ Non-Technical Skills (ANTS) behavioral markers. The residents’ scores were calculated to find the correlation within the ANTS rating scale.Results: The mean scores of knowledge tests were 164.3 ±18.4 out of 300 [165.5 ±18.0, 154.7 ±16.3 and 173.6 ±16.4 for PGY-1, PGY-2 and PGY-3 respectively]. The mean scores of ANTS was divided into 4 categories (rating scale 1 to 4): task management 2.9 (±0.6), teamworking 3.0 (±0.5), situation awareness 2.9 (±0.8) and decision making 2.8 (±0.7). The knowledge test results moderately correlated with ANTS score in task management, situation awareness and decision making [r=0.382 (p<0.01), r=0.433 (p<0.001) and r=0.350 (p<0.01) respectively] and weakly correlated with the teamworking category (r=0.166, p=0.16).Conclusion: Resident’s scores showed moderate correlation with non-technical skills assessment results in cognitive skills. Non-technical skills are required to be trained and assessed together with knowledge to enhance the patient’s safety and outcome.

The assessment of non-technical skills in the emergency department: Figure 1

2010 ◽  
Vol 27 (Suppl 1) ◽  
pp. A2.3-A3
Author(s):  
Lynsey Flowerdew
Keyword(s):  
Decision Making ◽  
Emergency Department ◽  
Emergency Medicine ◽  
Situation Awareness ◽  
Poor Performance ◽  
Technical Skills ◽  
Leadership Role ◽  
Critically Ill Patient ◽  
Observational Tool ◽  
Behavioural Aspects

IntroductionAnalysis of adverse events in the Emergency Department (ED) reveals that contributory factors commonly relate to non-technical skills such as decision-making. Crisis Resource Management courses, designed to improve non-technical skills, invariably centre on the critically ill patient. While resuscitation is undeniably important, little attention has been paid to the skills necessary for routine management of the ED and the vital leadership role of registrars. Furthermore, many UK trainees receive little specific assessment or feedback on the behavioural aspects of performance.Figure 1Distribution of time spent in emergency departments by ageAimTo produce an observational tool for the assessment of registrar non-technical skills in the ED, with a particular focus on leadership.MethodsFollowing a successful feasibility study, data was gathered from a variety of sources to allow subsequent triangulation of results. A literature review was undertaken to explore the evidence linking non-technical skills to safety in the ED. Various curricula were studied to ensure the observational tool reflected these competencies. Transcripts from an interview study with ED staff were analysed to determine which skills were considered to be important for effective teamwork. A series of observations were carried out to identify skills that were both observable and sufficiently common. The final skill set was reviewed by a team of behavioural psychologists and Emergency Medicine (EM) physicians.ResultsA taxonomy for non-technical skills was developed that consisted of 12 skills, each with observable behaviours indicative of good or poor performance. These skills were organised into four categories: (1) supervision and management; (2) teamwork and cooperation; (3) decision-making and (4) situation awareness.DiscussionThis observational tool is empirically based in Emergency Medicine and explicitly describes the skills required to structure observation and feedback for trainees during everyday activities. The validity of the skills taxonomy is currently being tested and future studies are planned to assess reliability, usability and acceptability.

Reliability of assessment of medical students’ non-technical skills using a behavioural marker system: does clinical experience matter?

2020 ◽  
pp. bmjstel-2020-000705
Author(s):  
Benjamin Clarke ◽  
Samantha E Smith ◽  
Emma Claire Phillips ◽  
Ailsa Hamilton ◽  
Joanne Kerins ◽  
...  
Keyword(s):  
Medical Students ◽  
Clinical Experience ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Technical Skills ◽  
Rater Reliability ◽  
Single Measure ◽  
Marker System ◽  
Intraclass Correlation Coefficients ◽  
Reliability Coefficients

IntroductionNon-technical skills are recognised to play an integral part in safe and effective patient care. Medi-StuNTS (Medical Students’ Non-Technical Skills) is a behavioural marker system developed to enable assessment of medical students’ non-technical skills. This study aimed to assess whether newly trained raters with high levels of clinical experience could achieve reliability coefficients of >0.7 and to compare differences in inter-rater reliability of raters with varying clinical experience.MethodsForty-four raters attended a workshop on Medi-StuNTS before independently rating three videos of medical students participating in immersive simulation scenarios. Data were grouped by raters’ levels of clinical experience. Inter-rater reliability was assessed by calculating intraclass correlation coefficients (ICC).ResultsEleven raters with more than 10 years of clinical experience achieved single-measure ICC of 0.37 and average-measures ICC of 0.87. Fourteen raters with more than or equal to 5 years and less than 10 years of clinical experience achieved single-measure ICC of 0.09 and average-measures ICC of 0.59. Nineteen raters with less than 5 years of clinical experience achieved single-measure ICC of 0.09 and average-measures ICC 0.65.ConclusionsUsing 11 newly trained raters with high levels of clinical experience produced highly reliable ratings that surpassed the prespecified inter-rater reliability standard; however, a single rater from this group would not achieve sufficiently reliable ratings. This is consistent with previous studies using other medical behavioural marker systems. This study demonstrated a decrease in inter-rater reliability of raters with lower levels of clinical experience, suggesting caution when using this population as raters for assessment of non-technical skills.

Development and Validation of an Assessment Tool for Technical Skills in Handheld Otoscopy

2020 ◽  
Vol 129 (7) ◽  
pp. 715-721
Author(s):  
Mads J. Guldager ◽  
Jacob Melchiors ◽  
Steven Arild Wuyts Andersen
Keyword(s):  
Medical Students ◽  
Assessment Tool ◽  
Medical Training ◽  
Technical Skills ◽  
University Teaching ◽  
Validity Evidence ◽  
Level Of Evidence ◽  
Training Interventions ◽  
Diagnostic Skills ◽  
Novices And Experts

Objective: Handheld otoscopy requires both technical and diagnostic skills, and is often reported to be insufficient after medical training. We aimed to develop and gather validity evidence for an assessment tool for handheld otoscopy using contemporary medical educational standards. Study Design: Educational study. Setting: University/teaching hospital. Subjects and Methods: A structured Delphi methodology was used to develop the assessment tool: nine key opinion leaders (otologists) in undergraduate training of otoscopy iteratively achieved consensus on the content. Next, validity evidence was gathered by the videotaped assessment of two handheld otoscopy performances of 15 medical students (novices) and 11 specialists in otorhinolaryngology using two raters. Standard setting (pass/fail criteria) was explored using the contrasting groups and Angoff methods. Results: The developed Copenhagen Assessment Tool of Handheld Otoscopy Skills (CATHOS) consists 10 items rated using a 5-point Likert scale with descriptive anchors. Validity evidence was collected and structured according to Messick’s framework: for example the CATHOS had excellent discriminative validity (mean difference in performance between novices and experts 20.4 out of 50 points, P < .001); and high internal consistency (Cronbach’s alpha = 0.94). Finally, a pass/fail score was established at 30 points for medical students and 42 points for specialists in ORL. Conclusion: We have developed and gathered validity evidence for an assessment tool of technical skills of handheld otoscopy and set standards of performance. Standardized assessment allows for individualized learning to the level of proficiency and could be implemented in under- and postgraduate handheld otoscopy training curricula, and is also useful in evaluating training interventions. Level of evidence: NA

scholarly journals Non-technical skills of medical students in the course of pediatric emergency simulations are associated with their mindfulness: an observational cohort study

2019 ◽  
Author(s):  
Jacek Chmielewski ◽  
Włodzimierz Łuczyński ◽  
Jakub Dobroch ◽  
Grzegorz Cebula ◽  
Tomasz Bielecki ◽  
...  
Keyword(s):  
Medical Students ◽  
Rating Scale ◽  
Technical Skills ◽  
Pediatric Emergency ◽  
Global Rating Scale ◽  
Attention And Memory ◽  
High Fidelity ◽  
Global Rating ◽  
Extreme Stress ◽  
Fixation Error

Abstract Background High fidelity medical simulations allow for teaching medical skills in safe and realistic conditions. Pediatric teams of emergency departments work under extreme stress, which affects high-level cognitive functions, specifically attention and memory, and increases the already high stakes for young doctors. Lapses in attention increase the risk of serious consequences such as medical errors, failure to recognize life-threatening signs and symptoms, and other essential patient safety issues. Mindfulness as a process of intentionally paying attention to each moment with curiosity, openness and acceptance of each experience without judgment can potentially contribute to improving the performance of medical teams in conditions of pediatric emergency. The aim of the study was to determine whether the actions of medical students in the course of pediatric high fidelity simulations are related to their mindfulness. Participants and methods A total of 166 standardized simulations were conducted among students of medicine in three simulation centers of medical universities, assessing: stress sensation, technical skills (checklists), non-technical skills (Ottawa Crisis Resource Management Global Rating Scale) and mindfulness using Five Facet Mindfulness Questionnaire. Results The perception of stress among students was lower and more motivating if they were more mindful. Mindfulness of students correlated positively with avoiding fixation error, but negatively with listening to and managing the team. The lowest scores among non-technical skills were noted in the area of ​​situational awareness (fixation error). In subsequent simulations of the same team, students' non-technical skills improved, although no change was noted in their technical skills. Conclusions The results of our research indicate the relationship between the mindfulness of medical students and their non-technical skills and the perception of stress in pediatric emergency simulations. Further research is needed to show whether mindfulness training leads to any changes in this field. Trial registration ClinicalTrials.gov, NCT03761355).

W16 Developing the non-technical skills of medical students: using a behavioural marker system in immersive simulation

2018 ◽  
Author(s):  
Vicky Tallentire ◽  
Ailsa Hamilton ◽  
Joanne Kerins ◽  
Katherine Leighton ◽  
Jerry Morse ◽  
...  
Keyword(s):  
Medical Students ◽  
Technical Skills ◽  
Marker System
Sign in / Sign up

Export Citation Format

Share Document