scholarly journals Application of a Simulation-Based Interprofessional Teamwork Assessment Tool (SITAT) to Individual Student Performance in a Team-Based Simulation

2021 ◽  
Vol 8 ◽  
pp. 238212052110424
Author(s):  
Brittany J Daulton ◽  
Laura Romito ◽  
Zach Weber ◽  
Jennifer Burba ◽  
Rami A Ahmed
Keyword(s):  
Student Performance ◽  
Internal Consistency ◽  
Interprofessional Education ◽  
Interrater Reliability ◽  
Assessment Tool ◽  
Individual Performance ◽  
Individual Student ◽  
Interprofessional Teamwork ◽  
Significant Difference ◽  
Simulation Based

There are a very limited number of instruments to assess individual performance in simulation-based interprofessional education (IPE). The purpose of this study was to apply the Simulation-Based Interprofessional Teamwork Assessment Tool (SITAT) to the individualized assessment of medicine, pharmacy, and nursing students (N = 94) in a team-based IPE simulation, as well as to explore potential differences between disciplines, and calculate reliability estimates for utilization of the tool. Results of an analysis of variance provided evidence that there was no statistically significant difference among professions on overall competency ( F(2, 91)  =  0.756, P  = .472). The competency reports for nursing ( M = 3.06, SD = 0.45), medicine ( M = 3.19, SD = 0.42), and pharmacy ( M = 3.08, SD = 0.49) students were comparable across professions. Cronbach's alpha provided a reliability estimate of the tool, with evidence of high internal consistency ( α = .92). The interrater reliability of the SITAT was also investigated. There was moderate absolute agreement across the 3 faculty raters using the 2-way mixed model design and “average” unit (kappa = 0.536, P = .000, 95% CI [0.34, 0.68]). The novel SITAT demonstrates internal consistency and interrater reliability when used for evaluation of individual performance during IPE simulation. The SITAT provides value in the education and evaluation of individual students engaged in IPE curriculum.

scholarly journals Methodology for quantitative assessment of combat casualty care

SIMULATION ◽  
2018 ◽  
Vol 95 (4) ◽  
pp. 289-295
Author(s):  
Michael A. Xynidis ◽  
Brian F. Goldiez ◽  
Jack E. Norfleet ◽  
Nina Rothstein
Keyword(s):  
Student Performance ◽  
Quantitative Assessment ◽  
Medical Training ◽  
Evaluation Process ◽  
Training Effectiveness ◽  
Individual Student ◽  
Psychomotor Activity ◽  
Combat Casualty ◽  
Significant Difference ◽  
Simulation Based

Evaluating proficiency in simulation-based combat casualty training includes the assessment of hands-on training with mannequins through instructor observation. The evaluation process is error-prone due to high student–instructor ratios as well as the subjective nature of the evaluation process. Other logistical inconsistencies, such as the short amount of time to observe individual student performance, can lessen training effectiveness as well. The simulation-based methodology described in this article addresses these challenges by way of quantitative assessment of training effectiveness in combat casualty training. The methodology discusses adaptation of Lempel–Ziv (LZ) complexity indexing to quantify psychomotor activity that is otherwise only subjectively estimated by an instructor. LZ indexing has been successfully used to assess proficiency in related studies of simulation-based training conducted by Bann et al. at the Imperial College of Science Technology and Medicine in London, and more recently by Watson at the University of North Carolina at Chapel Hill. This type of analysis has been applied to using simulation as a tool to assess not only mastery of a task, but as a method to assess whether a particular simulator and training approach actually works. Data have been gathered from nearly 100 military combat medic trainees at Joint Base Lewis McChord Medical Simulation Training Center. Participant hand-acceleration data from an emergency surgical cricothyrotomy reveals a statistically significant difference in ability between expertise levels. The higher the LZ scores and self-reported expertise level, the better the participant performed. The results show that when presented with demographic and video performance-based data, it is possible to gauge experience by applying LZ scoring to motion data. The methodology provides an objective measure that complements the subjective component of simulation-based cricothyrotomy training assessments. Further study is needed to determine whether this methodology would provide similar assessment advantages in other medical training in which speed and accuracy would be significant factors in determining procedural expertise.

Validity Evidence for the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT)

2016 ◽  
Vol 13 (1) ◽  
pp. 60-68 ◽  
Author(s):  
Gerben E. Breimer ◽  
Faizal A. Haji ◽  
Giuseppe Cinalli ◽  
Eelco W. Hoving ◽  
James M. Drake
Keyword(s):  
Internal Structure ◽  
Internal Consistency ◽  
Interrater Reliability ◽  
Rating Scale ◽  
Assessment Tool ◽  
Global Rating Scale ◽  
Global Rating ◽  
Validity Evidence ◽  
Simulation Based ◽  
Endoscopic Ventriculostomy

Abstract BACKGROUND: Growing demand for transparent and standardized methods for evaluating surgical competence prompted the construction of the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT). OBJECTIVE: To provide validity evidence of the NEVAT by reporting on the tool's internal structure and its relationship with surgical expertise during simulation-based training. METHODS: The NEVAT was used to assess performance of trainees and faculty at an international neuroendoscopy workshop. All participants performed an endoscopic third ventriculostomy (ETV) on a synthetic simulator. Participants were simultaneously scored by 2 raters using the NEVAT procedural checklist and global rating scale (GRS). Evidence of internal structure was collected by calculating interrater reliability and internal consistency of raters' scores. Evidence of relationships with other variables was collected by comparing the ETV performance of experts, experienced trainees, and novices using Jonckheere's test (evidence of construct validity). RESULTS: Thirteen experts, 11 experienced trainees, and 10 novices participated. The interrater reliability by the intraclass correlation coefficient for the checklist and GRS was 0.82 and 0.94, respectively. Internal consistency (Cronbach's α) for the checklist and the GRS was 0.74 and 0.97, respectively. Median scores with interquartile range on the checklist and GRS for novices, experienced trainees, and experts were 0.69 (0.58-0.86), 0.85 (0.63-0.89), and 0.85 (0.81-0.91) and 3.1 (2.5-3.8), 3.7 (2.2-4.3) and 4.6 (4.4-4.9), respectively. Jonckheere's test showed that the median checklist and GRS score increased with performer expertise (P = .04 and .002, respectively). CONCLUSION: This study provides validity evidence for the NEVAT to support its use as a standardized method of evaluating neuroendoscopic competence during simulation-based training.

scholarly journals The Effects of Gaming on University Student Quiz Performance

2019 ◽  
Vol 2 (1) ◽  
pp. 109-119
Author(s):  
Corinne M Gist ◽  
Natalie Andzik ◽  
Elle E Smith ◽  
Menglin Xu ◽  
Nancy A Neef
Keyword(s):  
Student Performance ◽  
Undergraduate Students ◽  
Assessment Tool ◽  
Electronic Device ◽  
Classroom Engagement ◽  
Electric Device ◽  
Significant Difference ◽  
Competitive Games ◽  
Education Courses ◽  
Quiz Scores

The use of competitive games to increase classroom engagement has become common practice among many teachers. However, it is unclear if using games as an assessment tool is a viable way to increase student performance. This study examined the effects of administering quizzes through a game-based system, Kahoot!,versusprivately on an electronic device. The quiz scores of 56 undergraduate students, enrolled in one of two special education courses, were evaluated. A linear regression was used to compare student scores across the two conditions, as well as performance over the course of a 15-week semester. No significant difference in quiz scores was found between the two conditions, and quiz scores in both conditions improved similarly over time. Sixty-eight percent of the students reported preferring to take the quiz privately on an electric device as opposed to on Kahoot!. Limitations and recommendations for practitioners are discussed.

scholarly journals Peran Kepemimpinan Transformasional dalam Meningkatkan Kinerja Individu Santri pada Unit Usaha Pondok Pesantren

2019 ◽  
Vol 2 (1) ◽  
pp. 50
Author(s):  
Fuad Dhiya Ul Husaen ◽  
Reni Rosari
Keyword(s):  
Transformational Leadership ◽  
Student Performance ◽  
Sample Selection ◽  
Sampling Technique ◽  
Boarding School ◽  
Individual Performance ◽  
Individual Student ◽  
Business Unit ◽  
Innovative Behavior ◽  
The Individual

This study aims to determine how much influence transformational leadership on the individual performances of santri, mediated by the innovative behavior of santri. This research is confirmatory research that aims to examine theories that have been there before. The number of samples in this study was 102 students who worked in the business unit of the Sidogiri Islamic boarding school. The sample selection uses a purposive sampling technique. Data was collected using a questionnaire instrument with a Likert scale. Testing the hypothesis refers to Baron Kenny's mediation theory and processing data using Smart PLS 3 applications, by analyzing the results of evaluating the outer and inner models. The results of this study indicate that; (1) transformational leadership of the head of a business unit has a significant positive effect on the individual performance of students with a coefficient of 0.546. (2) innovative behavior proved not to mediate the influence of transformational leadership head of the unit on individual student performance.

Development of a Simulation-Based Interprofessional Teamwork Assessment Tool

2019 ◽  
Author(s):  
Zia Bismilla ◽  
Tehnaz Boyle ◽  
Karen Mangold ◽  
Wendy Van Ittersum ◽  
Marjorie Lee White ◽  
...  
Keyword(s):  
Assessment Tool ◽  
Interprofessional Teamwork ◽  
Simulation Based

scholarly journals A delirium prevalence audit and a pre and post evaluation of an interprofessional education intervention to increase staff knowledge about delirium in older adults

BMC Nursing ◽  
2021 ◽  
Vol 20 (1) ◽  
Author(s):  
Beverley Ewens ◽  
Karla Seaman ◽  
Lisa Whitehead ◽  
Amanda Towell-Barnard ◽  
Michelle Young
Keyword(s):  
Interprofessional Education ◽  
Assessment Tool ◽  
Subscale Score ◽  
Adverse Outcomes ◽  
Education Intervention ◽  
Chi Square ◽  
Delirium Assessment ◽  
Post Evaluation ◽  
Significant Difference ◽  
Level Of Knowledge

Abstract Background Delirium is more prevalent in older people and estimated to occur in up to 50% of the hospital population. Delirium comprises a spectrum of behaviours, including cognitive and attention deficits, and fluctuating levels of consciousness, often associated with an underlying physiological disturbance. Delirium has been increasingly associated with adverse outcomes. Although often preventable or can at least be mitigated, delirium may not be a standard part of assessment and thus may not be recognized in the early stages when it is most likely to be treated successfully. The aim of this study was to evaluate the level of knowledge of delirium amongst clinicians caring for patients at high risk of developing delirium and to determine whether education can improve clinical assessment of delirium. Methods Two hundred and forty-six case notes were audited before and 149 were reviewed after the education intervention and implementation of a delirium screening tool. Clinicians at the hospital were invited to complete a questionnaire on knowledge of delirium. The questionnaire was based on a validated tool which contained 39 questions about delirium. The questionnaire also contained 28 questions on delirium knowledge. Additional questions were included to gather demographic information specific to the hospital. Descriptive statistics, chi square and independent t-tests were conducted to test for differences in knowledge between the pre and post periods. The Squire Checklist Reporting Guidelines for Quality Improvement Studies informed the preparation of the manuscript. Results The audit demonstrated that the use of a cognitive assessment tool overall increased from 8.5% in pre education to 43% in the post education period. One hundred and fifty-nine staff completed the questionnaire in total, 118 the pre and 41 post. The knowledge subscale score was high pre and post education and no statistically significant difference was observed. The greatest increase in knowledge was related to knowledge of the risk factors subscale. The increase in knowledge (6.8%) was statistically significant. Conclusion An interprofessional approach to delirium education was effective in not only increasing awareness of the factors associated with this syndrome but also increased the use of a delirium assessment tool.

scholarly journals From ideal to real: a qualitative study of the implementation of in situ Interprofessional Simulation-Based Education

2021 ◽  
Author(s):  
Mindy Ju ◽  
Naike Bochatay ◽  
Kathryn Robertson ◽  
James Frank ◽  
Bridget O’Brien ◽  
...  
Keyword(s):  
Interprofessional Education ◽  
Qualitative Case Study ◽  
Psychological Safety ◽  
Considerable Variability ◽  
Interprofessional Teamwork ◽  
Simulation Based ◽  
Competency Based ◽  
The Ideal

Abstract Background: Despite the widespread adoption of interprofessional simulation-based education (IPSE) in healthcare as a means to optimize interprofessional teamwork, data suggest that IPSE may not achieve these intended goals due to a gap between the ideals and the realities of implementation. Methods: We conducted a qualitative case study that used the framework method to understand what and how core principles from guidelines for interprofessional education (IPE) and simulation-based education (SBE) were implemented in existing in situ IPSE programs. We observed simulation sessions and interviewed facilitators and directors at seven programs. Results: We found considerable variability in how IPSE programs apply and implement core principles derived from IPE and SBE guidelines with some principles applied by most programs (e.g., “active learning”, “psychological safety”, “feedback during debriefing”) and others rarely applied (e.g., “interprofessional competency-based assessment”, “repeated and distributed practice”). Through interviews we identified that buy-in, resources, lack of outcome measures, and power discrepancies influenced the extent to which principles were applied. Conclusion: To achieve IPSE’s intended goals of optimizing interprofessional teamwork, programs should transition from designing for the ideal of IPSE to realities of IPSE implementation.

scholarly journals Using computerized assessment in simulated colonoscopy: a validation study

2020 ◽  
Vol 08 (06) ◽  
pp. E783-E791
Author(s):  
Andreas Slot Vilmann ◽  
Christian Lachenmeier ◽  
Morten Bo Søndergaard Svendsen ◽  
Bo Søndergaard ◽  
Yoon Soo Park ◽  
...  
Keyword(s):  
Assessment Tool ◽  
Assessment Tools ◽  
Continuous Assessment ◽  
Computerized Assessment ◽  
3 Dimensional ◽  
Phantom Model ◽  
Significant Difference ◽  
Simulation Based ◽  
Travel Length ◽  
Colon Data

Abstract Background and study aims Patient safety during a colonoscopy highly depends on endoscopist competence. Endoscopic societies have been calling for an objective and regular assessment of the endoscopists, but existing assessment tools are time-consuming and prone to bias. We aimed to develop and gather evidence of validity for a computerized assessment tool delivering automatic and unbiased assessment of colonoscopy based on 3 dimensional coordinates from the colonoscope. Methods Twenty-four participants were recruited and divided into two groups based on experience: 12 experienced and 12 novices. Participants performed twice on a physical phantom model with a standardized alpha loop in the sigmoid colon. Data was gathered directly from the Olympus ScopeGuide system providing XYZ-coordinates along the length of the colonoscope. Five different motor skill measures were developed based on the data, named: Travel Length, Tip Progression, Chase Efficiency, Shaft movement without tip progression, and Looping. Results The experinced had a lower travel length (P < 0.001), tip progression (P < 0.001), chase efficiency (P = 0.001) and looping (P = 0.006), and a higher shaft movement without tip progression (P < 0.001) reaching the cecum compared with the novices. A composite score was developed based on the five measurements to create a combined score of progression, the 3D-Colonoscopy-Progression-Score (3D-CoPS). The 3D-CoPS revealed a significant difference between groups (experienced: 0.495 (SD 0.303) and novices –0.454 (SD 0.707), P < 0.001). Conclusion This study presents a novel, real-time computerized assessment tool for colonoscopy, and strong evidence of validity was gathered in a simulation-based setting. The system shows promising opportunities for automatic, unbiased and continuous assessment of colonoscopy performance.

Development of a Performance Assessment Scale for Simulated Dispatcher-Assisted Cardiopulmonary Resuscitation (Telephone-CPR): A Multi-Center Randomized Simulation-Based Clinical Trial

2021 ◽  
pp. 1-9
Author(s):  
Daniel Aiham Ghazali ◽  
Caroline Delaire ◽  
Emmanuel Blottiaux ◽  
Jean-Yves Lardeur ◽  
Daniel Jost ◽  
...  
Keyword(s):  
Cardiac Arrest ◽  
Cardiopulmonary Resuscitation ◽  
Performance Assessment ◽  
Correlation Coefficient ◽  
Internal Consistency ◽  
Assessment Scale ◽  
Categorical Variables ◽  
Significant Difference ◽  
Simulation Based ◽  
Objective Performance

Abstract Introduction: Dispatchers should be trained to interrogate bystanders with strict protocols to elicit information focused on recognizing cardiac arrest and should provide telephone cardiopulmonary resuscitation (CPR) instructions in all cases of suspected cardiac arrest. While an objective assessment of training outcomes is needed, there is no performance assessment scale for simulated dispatcher-assisted CPR. Study Objective: The aim of the study was to create a valid and reliable performance assessment scale for simulated dispatcher-assisted CPR. Methods: In this prospective, randomized, controlled, multi-centric simulation-based trial (registration number TCTR20210130002), the scale was developed according to the European Resuscitation Council (ERC) and American Heart Association (AHA) Guidelines 2015 and revised by experts. The performance of 48 dispatchers’ telephone-CPR and of 48 bystanders carrying out CPR on a manikin was assessed by two independent evaluators using the scale and using a SkillReporter (PC) software to provide CPR objective performance. Continuous variables were described as mean (SD) and categorical variables as numbers and percentage (%). Comparative analysis between two groups used a Student t-test or a non-parametric test of Mann-Whitney. The internal structure of the scale was evaluated, including internal consistency using α Cronbach coefficient, and reproducibility using intraclass correlation coefficient (ICC) and linear correlation coefficient (R2) calculation. Results: The scale included three different parts: two sections for dispatchers’ (32 items) and bystanders’ CPR performance (15 items) assessment, and a third part recording times. There was excellent internal consistency (α Cronbach coefficient = 0.77) and reproducibility (ICC = 0.93; R² = 0.86). For dispatchers’ performance assessment, α Cronbach coefficient = 0.76; ICC = 0.91; R2 = 0.84. For bystanders’ performance assessment, α Cronbach coefficient = 0.75; ICC = 0.93; R2 = 0.87. Reproducibility was excellent for nine items, good for 19 items, and moderate for 19 items. No item had poor reproducibility. There was no significant difference between dispatch doctors’ and medical dispatch assistants’ performances (33.0 [SD = 4.7] versus 32.3 [SD = 3.2] out of 52, respectively; P = .70) or between trained and untrained bystanders to follow the instructions (14.3 [SD = 2.0] versus 13.9 [SD = 1.8], respectively; P = .64). Objective performance (%) was significantly higher for trained bystanders than for untrained bystanders (67.4 [SD = 14.5] versus 50.6 [SD = 19.3], respectively; P = .03). Conclusion: The scale was valid and reliable to assess performance for simulated dispatcher-assisted CPR. To the authors’ knowledge, no other valid performance tool currently exists. It could be used in simulated telephone-CPR training programs to improve performance.

scholarly journals Development of a Simulation-Based Interprofessional Teamwork Assessment Tool

2019 ◽  
Vol 11 (2) ◽  
pp. 168-176
Author(s):  
Zia Bismilla ◽  
Tehnaz Boyle ◽  
Karen Mangold ◽  
Wendy Van Ittersum ◽  
Marjorie Lee White ◽  
...  
Keyword(s):  
Delphi Method ◽  
Rating Scale ◽  
Assessment Tool ◽  
Ease Of Use ◽  
Validity Evidence ◽  
Interprofessional Teamwork ◽  
Point Rating Scale ◽  
Simulation Based ◽  
Competency Based ◽  
Trainee Competence

ABSTRACT Background  The Accreditation Council for Graduate Medical Education (ACGME) Milestone projects required each specialty to identify essential skills and develop means of assessment with supporting validity evidence for trainees. Several specialties rate trainees on a milestone subcompetency related to working in interprofessional teams. A tool to assess trainee competence in any role on an interprofessional team in a variety of scenarios would be valuable and suitable for simulation-based assessment. Objective  We developed a tool for simulation settings that assesses interprofessional teamwork in trainees. Methods  In 2015, existing tools that assess teamwork or interprofessionalism using direct observation were systematically reviewed for appropriateness, generalizability, adaptability, ease of use, and resources required. Items from these tools were included in a Delphi method with multidisciplinary pediatrics experts using an iterative process from June 2016 to January 2017 to develop an assessment tool. Results  Thirty-one unique tools were identified. A 2-stage review narrowed this list to 5 tools, and 81 items were extracted. Twenty-two pediatrics experts participated in 4 rounds of Delphi surveys, with response rates ranging from 82% to 100%. Sixteen items reached consensus for inclusion in the final tool. A global 4-point rating scale from novice to proficient was developed. Conclusions  A novel tool to assess interprofessional teamwork for individual trainees in a simulated setting was developed using a systematic review and Delphi methodology. This is the first step to establish the validity evidence necessary to use this tool for competency-based assessment.

Sign in / Sign up

Export Citation Format

Share Document