Assessment of medical students’ clinical performance using high-fidelity simulation: comparison of peer and instructor assessment

Abstract Background High-fidelity simulators are highly useful in assessing clinical competency; they enable reliable and valid evaluation. Recently, the importance of peer assessment has been highlighted in healthcare education, and studies using peer assessment in healthcare, such as medicine, nursing, dentistry, and pharmacy, have examined the value of peer assessment. This study aimed to analyze inter-rater reliability between peers and instructors and examine differences in scores between peers and instructors in the assessment of high-fidelity-simulation-based clinical performance by medical students. Methods This study analyzed the results of two clinical performance assessments of 34 groups of fifth-year students at Ajou University School of Medicine in 2020. This study utilized a modified Queen’s Simulation Assessment Tool to measure four categories: primary assessment, diagnostic actions, therapeutic actions, and communication. In order to estimate inter-rater reliability, this study calculated the intraclass correlation coefficient and used the Bland and Altman method to analyze agreement between raters. A t-test was conducted to analyze the differences in evaluation scores between colleagues and faculty members. Group differences in assessment scores between peers and instructors were analyzed using the independent t-test. Results Overall inter-rater reliability of clinical performance assessments was high. In addition, there were no significant differences in overall assessment scores between peers and instructors in the areas of primary assessment, diagnostic actions, therapeutic actions, and communication. Conclusions The results indicated that peer assessment can be used as a reliable assessment method compared to instructor assessment when evaluating clinical competency using high-fidelity simulators. Efforts should be made to enable medical students to actively participate in the evaluation process as fellow assessors in high-fidelity-simulation-based assessment of clinical performance in situations similar to real clinical settings.

Download Full-text

Effects of high-fidelity simulation education on medical students’ anxiety and confidence

PLoS ONE ◽

10.1371/journal.pone.0251078 ◽

2021 ◽

Vol 16 (5) ◽

pp. e0251078

Author(s):

Ji Hye Yu ◽

Hye Jin Chang ◽

Soon Sun Kim ◽

Ji Eun Park ◽

Wou Young Chung ◽

...

Keyword(s):

Clinical Practice ◽

Medical Students ◽

Patient Care ◽

Clinical Setting ◽

Clinical Performance ◽

High Fidelity ◽

High Fidelity Simulation ◽

Simulation Education ◽

Before And After ◽

Care Situation

Introduction Psychological factors such as anxiety and confidence that students have in the patient care situation are important in that this affects the actual clinical performance. Students who are just starting clinical practice have a lack of clinical knowledge, skill proficiency, and patient communication skills, so they experience anxiety and lack of confidence in clinical setting. Practice in a safe environment, such as simulation education, can help students perform more settled and competently in patient care. The purpose of this study was to analyze the effect of high-fidelity simulation experience on anxiety and confidence in medical students. Materials and methods This study enrolled 37 5th-year students at Ajou University School of Medicine in 2020. Two simulation trainings were implemented, and a survey was conducted to measure students’ level of anxiety and confidence before and after each simulation. Based on the research data, a paired t-test was conducted to compare these variables before and after the simulation, and whether this was their first or second simulation experience. Results Students had a significantly lower level of anxiety and a significantly higher level of confidence after the simulation than before. In addition, after one simulation experience, students had less anxiety and more confidence before the second simulation compared to those without simulation experience. Conclusions We confirmed that medical students need to be repeatedly exposed to simulation education experiences in order to have a sense of psychological stability and to competently deliver medical treatment in a clinical setting. There is a practical limitation in that medical students do not have enough opportunities to meet the patients during clinical practice in hospitals. Therefore, in order to produce excellent doctors, students should have the expanded opportunities to experience simulation education so they can experience real-world medical conditions.

Download Full-text

EFFECTIVENESS OF HIGH-FIDELITY SIMULATION BASED ADVANCED CARDIAC LIFE SUPPORT TRAINING AMONG FINAL YEAR MEDICAL STUDENTS

ICERI2017 Proceedings ◽

10.21125/iceri.2017.2247 ◽

2017 ◽

Cited By ~ 1

Author(s):

Khin Ohn ◽

May Ohn ◽

U. D'Souza ◽

Zainal Arifin ◽

Win Tin ◽

...

Keyword(s):

Medical Students ◽

Life Support ◽

Advanced Cardiac Life Support ◽

High Fidelity ◽

High Fidelity Simulation ◽

Simulation Based

Download Full-text

Team Training of Inter-Professional Students (TTIPS) for improving teamwork

BMJ Simulation and Technology Enhanced Learning ◽

10.1136/bmjstel-2017-000194 ◽

2017 ◽

Vol 3 (4) ◽

pp. 127-134 ◽

Cited By ~ 4

Author(s):

John T. Paige ◽

Deborah D. Garbee ◽

Qingzhao Yu ◽

Vadym Rusnak

Keyword(s):

Medical Students ◽

Interprofessional Education ◽

Training Session ◽

High Fidelity ◽

High Fidelity Simulation ◽

Team Structure ◽

Undergraduate Nursing ◽

Simulation Based ◽

Assessment Scales

ObjectiveIn surgery, dysfunctional teamwork is perpetuated by a ‘silo’ mentality modelled by students. Interprofessional education using high-fidelity simulation-based training (SBT) may counteract such modelling. We sought to determine whether SBT of interprofessional student teams (1) changes long-term teamwork attitudes and (2) is an effective form of team training.DesignA quasiexperimental, pre/postintervention comparison design was employed at an academic health sciences institution. High-fidelity simulation-based training of 42 interprofessional teams of third year surgery clerkship medical students and senior undergraduate nursing students was undertaken using a two-scenario format with immediate after action debriefing. Pre/postintervention TeamSTEPPS Teamwork Attitudes questionnaires (5 subscales, 30 items, Likert type) were given to the medical student and undergraduate nursing student classes. Pre/postsession Readiness for Inter-Professional Learning (RIPL; 19 items, Likert type) surveys and postscenario participant-rated and observer-rated Teamwork Assessment Scales (3 subscales, 11 items, Likert type) were given during each training session. Mean TeamSTEPPS Teamwork Attitudes Questionnaire, RIPL and Teamwork Assessment Scales scores were calculated; matched pre/postscore differences and trained versus non-trained TeamSTEPPS Teamwork Attitudes Questionnaire scores were compared using paired t-test or analysis of variance.ResultsBoth student groups had 10 significantly improved RIPL items as well as TeamSTEPPS Teamwork Attitudes Questionnaire (TTAQ) mutual support subscales. Medical students had a significantly improved TTAQ team structure subscale. Over a simulation-based training session, each observer-rated Teamwork Assessment Scales subscale and two self-rated Teamwork Assessment Scales subscales significantly improved. Trained students had significantly higher TTAQ team structure subscales than non-trained students.ConclusionsInterprofessional education using high-fidelity simulation-based training of students is effective at teaching teamwork, changing interprofessional attitudes and improving long-term teamwork attitudes.

Download Full-text

Practical tips for introducing high-fidelity simulation to undergraduates at a large scale: learning from our experience

BMJ Simulation and Technology Enhanced Learning ◽

10.1136/bmjstel-2021-000888 ◽

2021 ◽

pp. bmjstel-2021-000888

Author(s):

Sinead M Walsh ◽

Maria Costello ◽

Ella Murphy ◽

Aoife Lowery ◽

Bronwyn Reid McDermott ◽

...

Keyword(s):

Medical Students ◽

Large Scale ◽

Education Programme ◽

High Fidelity ◽

High Fidelity Simulation ◽

Undergraduate Medical Students ◽

Large Scale Learning ◽

Simulation Based

The letter provides practical tips for developing, implementing and scaling an effective simulation‑based education programme at a large scale for undergraduate medical students. Using time-lapsed scenarios and the pause-discuss method of debrief are some of the useful tips that are discussed further in the letter.

Download Full-text

The Validity of Performance Assessments Using Simulation

Anesthesiology ◽

10.1097/00000542-200107000-00011 ◽

2001 ◽

Vol 95 (1) ◽

pp. 36-42 ◽

Cited By ~ 111

Author(s):

J. Hugh Devitt ◽

Matt M. Kurrek ◽

Marsha M. Cohen ◽

Doreen Cleave-Hogg

Keyword(s):

Medical Students ◽

Construct Validity ◽

Evaluation Method ◽

Clinical Performance ◽

Evaluation Process ◽

Performance Assessments ◽

Test Scenario ◽

Community Based ◽

Simulation Based ◽

Coronary Ischemia

Background The authors wished to determine whether a simulator-based evaluation technique assessing clinical performance could demonstrate construct validity and determine the subjects' perception of realism of the evaluation process. Methods Research ethics board approval and informed consent were obtained. Subjects were 33 university-based anesthesiologists, 46 community-based anesthesiologists, 23 final-year anesthesiology residents, and 37 final-year medical students. The simulation involved patient evaluation, induction, and maintenance of anesthesia. Each problem was scored as follows: no response to the problem, score = 0; compensating intervention, score = 1; and corrective treatment, score = 2. Examples of problems included atelectasis, coronary ischemia, and hypothermia. After the simulation, participants rated the realism of their experience on a 10-point visual analog scale (VAS). Results After testing for internal consistency, a seven-item scenario remained. The mean proportion scoring correct answers (out of 7) for each group was as follows: university-based anesthesiologists = 0.53, community-based anesthesiologists = 0.38, residents = 0.54, and medical students = 0.15. The overall group differences were significant (P < 0.0001). The overall realism VAS score was 7.8. There was no relation between the simulator score and the realism VAS (R = -0.07, P = 0.41). Conclusions The simulation-based evaluation method was able to discriminate between practice categories, demonstrating construct validity. Subjects rated the realism of the test scenario highly, suggesting that familiarity or comfort with the simulation environment had little or no effect on performance.

Download Full-text

The effect of supplemental high Fidelity simulation training in medical students

BMC Medical Education ◽

10.1186/s12909-020-02322-y ◽

2020 ◽

Vol 20 (1) ◽

Author(s):

Lori Meyers ◽

Bryan Mahoney ◽

Troy Schaffernocker ◽

David Way ◽

Scott Winfield ◽

...

Keyword(s):

Medical Students ◽

First Year ◽

High Fidelity ◽

High Fidelity Simulation ◽

Clinical Environment ◽

Learning Enhancement ◽

Pulmonary Physiology ◽

Clinical Scenarios ◽

Simulation Based ◽

First Year Medical Students

Abstract Background Simulation-based education (SBE) with high-fidelity simulation (HFS) offers medical students early exposure to the clinical environment, allowing development of clinical scenarios and management. We hypothesized that supplementation of standard pulmonary physiology curriculum with HFS would improve the performance of first-year medical students on written tests of pulmonary physiology. Methods This observational pilot study included SBE with three HFS scenarios of patient care that highlighted basic pulmonary physiology. First-year medical students’ test scores of their cardio-pulmonary curriculum were compared between students who participated in SBE versus only lecture-based education (LBE). A survey was administered to the SBE group to assess their perception of the HFS. Results From a class of 188 first-year medical students, 89 (47%) participated in the SBE and the remaining 99 were considered as the LBE group. On their cardio-pulmonary curriculum test, the SBE group had a median score of 106 [IQR: 97,110] and LBE group of 99 [IQR: 89,105] (p < 0.001). For the pulmonary physiology subsection, scores were also significantly different between groups (p < 0.001). Conclusions Implementation of supplemental SBE could be an adequate technique to improve learning enhancement and overall satisfaction in preclinical medical students.

Download Full-text

The Perceptions of Nurse Educators regarding the use of High Fidelity Simulation in Nursing Education

Africa Journal of Nursing and Midwifery ◽

10.25159/2520-5293/1685 ◽

2018 ◽

Vol 20 (1) ◽

Author(s):

Viola Janse van Vuuren ◽

Eunice Seekoe ◽

Daniel Ter Goon

Keyword(s):

Nursing Education ◽

South African ◽

Clinical Training ◽

Nurse Educators ◽

Teaching Staff ◽

High Fidelity ◽

High Fidelity Simulation ◽

Simulation Based ◽

Nursing College ◽

Faculty Shortages

Although nurse educators are aware of the advantages of simulation-based training, some still feel uncomfortable to use technology or lack the motivation to learn how to use the technology. The aging population of nurse educators causes frustration and anxiety. They struggle with how to include these tools particularly in the light of faculty shortages. Nursing education programmes are increasingly adopting simulation in both undergraduate and graduate curricula. The aim of this study was to determine the perceptions of nurse educators regarding the use of high fidelity simulation (HFS) in nursing education at a South African private nursing college. A national survey of nurse educators and clinical training specialists was completed with 118 participants; however, only 79 completed the survey. The findings indicate that everyone is at the same level as far as technology readiness is concerned, however, it does not play a significant role in the use of HFS. These findings support the educators’ need for training to adequately prepare them to use simulation equipment. There is a need for further research to determine what other factors play a role in the use of HFS; and if the benefits of HFS are superior to other teaching strategies warranting the time and financial commitment. The findings of this study can be used as guidelines for other institutions to prepare their teaching staff in the use of HFS.

Download Full-text