Validity and Reliability of a Human Factors Rating Scale to Assess Performance of Obstetrical Teams using High-Fidelity Simulation.

Author(s):  
Pamela Morgan ◽  
Richard Pittini ◽  
Glenn Regehr ◽  
Carol Marrs ◽  
Michele Haley
2017 ◽  
Vol 25 (3) ◽  
pp. 431-458 ◽  
Author(s):  
Janine E. Hinton ◽  
Mary Z. Mays ◽  
Debra Hagler ◽  
Pamela Randolph ◽  
Ruth Brooks ◽  
...  

Background and Purpose: There is growing evidence that simulation testing is appropriate for assessing nursing competence. We compiled evidence on the validity and reliability of the Nursing Performance Profile (NPP) method for assessing competence. Methods: Participants (N = 67) each completed 3 high-fidelity simulation tests; raters (N = 31) scored the videotaped tests using a 41-item competency rating instrument. Results: The test identified areas of practice breakdown and distinguished among subgroups differing in age, education, and simulation experience. Supervisor assessments were positively correlated, r = .31. Self-assessments were uncorrelated, r = .07. Inter-rater agreement ranged from 93% to 100%. Test–retest reliability ranged from r = .57 to .69. Conclusions: The NPP can be used to assess competence and make decisions supporting public safety.


2012 ◽  
Vol 117 (4) ◽  
pp. 883-897 ◽  
Author(s):  
Mrinalini Balki ◽  
Mary Ellen Cooke ◽  
Susan Dunington ◽  
Aliya Salman ◽  
Eric Goldszmidt

Background The objective of this study was to develop a consensus-based algorithm for the management of the unanticipated difficult airway in obstetrics, and to use this algorithm for the assessment of anesthesia residents' performance during high-fidelity simulation. Methods An algorithm for unanticipated difficult airway in obstetrics, outlining the management of six generic clinical situations of "can and cannot ventilate" possibilities in three clinical contexts: elective cesarean section, emergency cesarean section for fetal distress, and emergency cesarean section for maternal distress, was used to create a critical skills checklist. The authors used four of these scenarios for high-fidelity simulation for residents. Their critical and crisis resource management skills were assessed independently by three raters using their checklist and the Ottawa Global rating scale. Results Sixteen residents participated. The checklist scores ranged from 64-80% and improved from scenario 1 to 4. Overall Global rating scale scores were marginal and not significantly different between scenarios. The intraclass correlation coefficient of 0.69 (95% CI: 0.58, 0.78) represents a good interrater reliability for the checklist. Multiple critical errors were identified, the most common being not calling for help or a difficult airway cart. Conclusions Aside from identifying common critical errors, the authors noted that the residents' performance was poorest in two of our scenarios: "fetal distress and cannot intubate, cannot ventilate" and "maternal distress and cannot intubate, but can ventilate." More teaching emphasis may be warranted to avoid commonly identified critical errors and to improve overall management. Our study also suggests a potential for experiential learning with successive simulations.


2018 ◽  
Vol 5 (2) ◽  
pp. 108-110
Author(s):  
Josephine Seale ◽  
Sabina Ikram ◽  
Lewis Whittingham ◽  
Colin Butchers

High-fidelity simulation (Hi-Fi SIM) is increasingly used to provide undergraduate interprofessional education (IPE). Although research has reported positive student feedback, studies have predominantly involved medical and nursing specialties. The present study sought to further explore this area by determining the perceptions of medical, physiotherapy and nursing students participating in the same simulation session. A total of 145 medical, physiotherapy and nursing undergraduate students jointly participated in a novel Hi-Fi SIM IPE programme. Immediately before and after their session, students completed the KidSIM ATTITUDES questionnaire where statements were rated regarding simulation, IPE and human factors. A high score indicated a more positive attitude. Physiotherapy students reported the lowest level of previous Hi-Fi SIM experience. Students from each specialty had more positive attitudes related to simulation, IPE and human factors following their simulation. Physiotherapy students had predominantly less positive attitudes compared with nursing and medical students. Participation in an IPE Hi-Fi SIM session positively impacted on the perceptions of medical, physiotherapy and nursing students regarding the relevance of simulation, IPE and the importance of human factors. Such findings support the use of this learning modality for the provision of IPE in a range of specialties.


Author(s):  
B Santyr ◽  
M Abbass ◽  
A Chalil ◽  
D Krivosheya ◽  
LM Denning ◽  
...  

Background: Microsurgical techniques remain a cornerstone of neurosurgical training. Despite this, neurosurgical microvascular case volumes are decreasing as endovascular and minimally invasive options expand. As such, educators are looking towards simulation to supplement operative exposure. We review a single institution’s experience with a comprehensive, longitudinal microsurgical simulation training program, and evaluate its effectiveness. Methods: Consecutive postgraduate year 2 (PGY-2) neurosurgery residents completed a one-year curriculum spanning 17 training sessions divided into 5 modules of increasing fidelity. Both perfused duck wing and live rat femoral vessel training modules were used. Trainee performance was video recorded and blindly graded using the Objective Structured Assessment of Technical Skills Global Rating Scale. Results: Eighteen participants completed 107 microvascular anastomoses during the study. There was significant improvement in six measurable skills during the curriculum. Mean overall score was significantly higher on the fifth attempt compared to the first attempt for all 3 live anastomotic modules (p<0.001). Each module had a different improvement profile across the skills assessed. The greatest improvement was observed during artery-to-artery anastomosis. Conclusions: This high-fidelity microsurgical simulation curriculum demonstrated a significant improvement in the six microneurosurgical skills assessed, supporting its use as an effective teaching model. Transferability to the operative environment is actively being investigated.


2017 ◽  
Vol 33 (09) ◽  
pp. 641-648 ◽  
Author(s):  
Charles Butler ◽  
Hiroo Suami ◽  
Jun Liu ◽  
Jesse Selber ◽  
Mark Schaverien

Background The live rat femoral artery model is the gold standard in high-fidelity simulation for microvascular anastomosis worldwide. Despite widespread use, there is a paucity of evidence for its validity. This study aims to determine the correlation between microvascular anastomosis in the live rat femoral artery model and clinical performance. Methods Twenty-nine microsurgical fellows were included in the study. At the beginning and end of the 1-year fellowship, fellows were assessed performing arterial microvascular anastomoses in the laboratory in a live rat femoral artery model. Each procedure was digitally video recorded and blinded. Fellows were also assessed in the operating room at the beginning and end of training performing microvascular anastomoses in multiple free flap cases. Validated tools were used for assessment, and duration of procedure was also recorded. The results from the evaluations were compared and interfellow comparisons were performed. Results At both the beginning and end of the fellowship, there was no significant correlation between clinical performance evaluations during operating room cases and laboratory evaluations. In the laboratory, higher performance scores were associated with faster times at both the beginning and end of the fellowship. Conclusion Contrary to conventional wisdom, microsurgical skill in the live rat femoral artery model in the laboratory does not correlate with clinical performance in the operating room. This calls into question the utility of this model for training purposes. Further research is necessary to corroborate these results and potentially establish experimental models with greater validity and reliability.


Author(s):  
Pamela J Morgan ◽  
Richard Pittini ◽  
Carol Marrs ◽  
Susan DeSousa ◽  
Michele F Haley ◽  
...  

2018 ◽  
Vol 13 (4) ◽  
pp. e10-e16
Author(s):  
Patrice Chrétien Raymer ◽  
Jean-Paul Makhzoum ◽  
Robert Gagnon ◽  
Arielle Levy ◽  
Jean-Pascal Costa

Introduction: High-fidelity simulation is an efficient and holistic teaching method. However, assessing simulation performances remains a challenge. We aimed to develop a CanMEDS competency-based global rating scale for internal medicine trainees during simulated acute care scenarios. Methods: Our scale was developed using a formal Delphi process. Validity was tested using six videotaped scenarios of two residents managing unstable atrial fibrillation, rated by 6 experts. Psychometric properties were determined using a G-study and a satisfaction questionnaire.Results: Most evaluators favorably rated the usability of our scale, and attested that the tool fully covered CanMEDS competencies. The scale showed low to intermediate generalization validity.Conclusions: This study demonstrated some validity arguments for our scale. The best assessed aspect of performance was communication; further studies are planned to gather further validity arguments for our scale and to compare assessment of teamwork and communication during scenarios with multiple versus single residents.


Sign in / Sign up

Export Citation Format

Share Document