Updated Global Rating Scale standards for 2021

The Joint Advisory Group on Gastrointestinal Endoscopy (JAG) are pleased to launch an updated version of the Global Rating Scale standards

Development and validity evidence of an objective structured assessment of technical skills score for minimally invasive linear-stapled, hand-sewn intestinal anastomoses: the A-OSATS score

Introduction The aim of this study was to develop a reliable objective structured assessment of technical skills (OSATS) score for linear-stapled, hand-sewn closure of enterotomy intestinal anastomoses (A-OSATS). Materials and methods The Delphi methodology was used to create a traditional and weighted A-OSATS score highlighting the more important steps for patient outcomes according to an international expert consensus. Minimally invasive novices, intermediates, and experts were asked to perform a minimally invasive linear-stapled intestinal anastomosis with hand-sewn closure of the enterotomy in a live animal model either laparoscopically or robot-assisted. Video recordings were scored by two blinded raters assessing intrarater and interrater reliability and discriminative abilities between novices (n = 8), intermediates (n = 24), and experts (n = 8). Results The Delphi process included 18 international experts and was successfully completed after 4 rounds. A total of 4 relevant main steps as well as 15 substeps were identified and a definition of each substep was provided. A maximum of 75 points could be reached in the unweighted A-OSATS score and 170 points in the weighted A-OSATS score respectively. A total of 41 anastomoses were evaluated. Excellent intrarater (r = 0.807–0.988, p < 0.001) and interrater (intraclass correlation coefficient = 0.923–0.924, p < 0.001) reliability was demonstrated. Both versions of the A-OSATS correlated well with the general OSATS and discriminated between novices, intermediates, and experts defined by their OSATS global rating scale. Conclusion With the weighted and unweighted A-OSATS score, we propose a new reliable standard to assess the creation of minimally invasive linear-stapled, hand-sewn anastomoses based on an international expert consensus. Validity evidence in live animal models is provided in this study. Future research should focus on assessing whether the weighted A-OSATS exceeds the predictive capabilities of patient outcomes of the unweighted A-OSATS and provide further validity evidence on using the score on different anastomotic techniques in humans.

P.181 High-Fidelity Simulation-Based Microsurgical Training for Neurosurgical Residents

Background: Microsurgical techniques remain a cornerstone of neurosurgical training. Despite this, neurosurgical microvascular case volumes are decreasing as endovascular and minimally invasive options expand. As such, educators are looking towards simulation to supplement operative exposure. We review a single institution’s experience with a comprehensive, longitudinal microsurgical simulation training program, and evaluate its effectiveness. Methods: Consecutive postgraduate year 2 (PGY-2) neurosurgery residents completed a one-year curriculum spanning 17 training sessions divided into 5 modules of increasing fidelity. Both perfused duck wing and live rat femoral vessel training modules were used. Trainee performance was video recorded and blindly graded using the Objective Structured Assessment of Technical Skills Global Rating Scale. Results: Eighteen participants completed 107 microvascular anastomoses during the study. There was significant improvement in six measurable skills during the curriculum. Mean overall score was significantly higher on the fifth attempt compared to the first attempt for all 3 live anastomotic modules (p<0.001). Each module had a different improvement profile across the skills assessed. The greatest improvement was observed during artery-to-artery anastomosis. Conclusions: This high-fidelity microsurgical simulation curriculum demonstrated a significant improvement in the six microneurosurgical skills assessed, supporting its use as an effective teaching model. Transferability to the operative environment is actively being investigated.

A novel approach: Simulating multiple simultaneous encounters to assess multitasking ability in emergency medicine

Study objective The purpose of this feasibility study is to develop and validate a new assessment tool and scoring system for multitasking competency for physicians in-training in a timed simulated setting. The multitasking competency includes ability to appropriately prioritize and implement tasks for different patients who present simultaneously. Methods We designed three single task stations with different levels of difficulty and priority. These skill stations were then combined to create a multitasking simulation scenario. Skill checklists and the global rating scale were utilized to assess the participants’ performance. A multitasking score, multitasking index, and priority score were developed to measure the multitasking ability of participants. Results Thirty-three first-year postgraduate physicians were recruited for this prospective study. The total performance scores were significantly higher for the single-tasking stations than for the multitasking scenario. In terms of the time needed to complete the tasks, the participants spent more time on the multitasking scenario than on the single-tasking scenario. There were significant correlations between the global rating scale and the multitasking score (rho = 0.693, p < 0.001) and between the global rating scale and the multitasking index (rho = 0.515, p < 0.001). The multitasking score, multitasking index, and priority score did not have any significant correlations with the total single-tasking score. Conclusion We demonstrated that the use of a simulated multitasking scenario could be an effective method of assessing multitasking ability and allow assessors to offer better quality feedback.

1005 Acquisition of Surgical Skills in Medical Students Via Telementoring: A Randomised Control Trial

Aim To evaluate the use of telementoring in acquisition of basic suturing skills in medical students. Method This was a single blinded two- armed randomised control trial. Medical students undergoing clinical rotations in their penultimate and final years were invited to participate in this study. Informed consent was obtained. The control group underwent conventional suturing training, with the interventional group undergoing the tutorial in a remote learning setting via live streaming. Pre- and post-test assessment was carried out using validated Global Rating Scale tool. Results Eleven students were enrolled in this study. Participants were comparable at baseline (p=.18) and following the tutorial (p=.29). Participants improved to a statistically significant degree in both the virtual group (p=.02) and the in-person group (p=.001). Conclusions Telementoring is an effective tool in the provision of teaching basic suturing skills in medical students. Research on its use in more complex practical skills is warranted.

What matters for people with brain cancer? Selecting clinical quality indicators for an Australian Brain Cancer Registry

Background The goal of a clinical quality registry is to deliver immediate gains in survival and quality of life by delivering timely feedback to practitioners, thereby ensuring every patient receives the best existing treatment. We are developing an Australian Brain Cancer Registry (ABCR) to identify, describe and measure the impact of the variation and gaps in brain cancer care from the time of diagnosis to the end of life. Methods To determine a set of Clinical Quality Indicators (CQIs) for the ABCR, a database and internet search was used to identify relevant guidelines, which were then assessed for quality using the AGREE II Global Rating Scale. Potential indicators were extracted from 21 clinical guidelines, ranked using a modified Delphi process completed in two rounds by a panel of experts and other stakeholders, and refined by a multidisciplinary Working Group. Results Nineteen key quality reporting domains were chosen, specified by 57 CQIs detailing the specific inclusion and outcome characteristics to be reported. Conclusion The selected CQIs will form the basis for the Australian Brain Cancer Registry, provide a framework for achievable data collection, and specify best practice for patients and health care providers, with a view to improving care for brain cancer patients. To our knowledge the systematic and comprehensive approach we have taken is a world first in selecting the reporting specifications for a brain cancer clinical registry.

Mixed-Methods Analysis of a Validated Arthroscopic Knot-Tying Simulator With New Indirect Visualization Condition

Background: Simulation provides low-risk opportunities for surgical trainees to learn and practice fundamental skills. One simulation tool for orthopaedics is the Arthroscopic Knot (ArK) Trainer, which has been validated as an effective simulation tool across multiple methodologies. Previous studies have investigated the ArK Trainer in its basic form using clear plexiglass, which allows direct visualization of tissue anchors. Purpose: Using a mixed-methods approach, we assessed and compared junior and senior trainees’ Seoul Medical Center (SMC) knot–tying performance under direct and indirect visualization. Study Design: Cross-sectional study. Methods: Fourteen orthopaedic surgery postgraduate trainees at a single medical school were recruited to participate. Trainees tied SMC knots using the Ark Trainer under direct and indirect visualization. A mixed-methods approach was used to evaluate knot-tying proficiency and characterize participants’ approach to knot-tying. Knot-tying proficiency was evaluated using validated tools: a task-specific checklist (TSC), a global rating scale (GRS), and a proficiency scale (PS). Participants’ approach to knot-tying was characterized using Likert-type questionnaires and semistructured interviews. An α level of .10 was set a priori owing to the small pool of trainees. Results: The 14 participants included 7 junior residents (postgraduate years [PGYs] 1 and 2) and 7 senior residents (PGY ≥3), of whom 3 were fellows (PGY 6). Senior trainees outperformed junior trainees on both versions of the ArK Trainer: clear (GRS, P = .055; PS, P = .075) and covered (TSC, P = .05). Overall, participants performed better under direct visualization conditions (GRS, P = .05). In semistructured interviews, significantly more senior trainees discussed relying on haptic cues while tying knots under direct visualization ( P = .021). The majority of trainees agreed that both versions of the ArK Trainer were realistic and appropriate practice formats for their level of training. Conclusion: Senior trainees were significantly more experienced than were junior trainees in arthroscopic skill and outperformed them on both configurations: direct (PS and GRS) and indirect (TSC) visualization. Experienced trainees were significantly more likely to report using tactile cues to aid knot-tying under indirect visualization. It is likely that inexperienced trainees rely more heavily on direct visualization and that the use of tactile cues may be an indicator of knot-tying proficiency. Trainees recommended progression from direct to indirect visualization configurations for inexperienced learners.

The assessment of researchers’ competence in experimental procedures with laboratory animals: A three-step methodology to develop a global rating scale

To conduct animal experiments, researchers must be competent to handle and perform interventions on living animals in compliance with regulations. Laboratory animal science training programmes and licensing bodies therefore need to be able to reliably ensure and certify the professional competence of researchers and technicians. This requires access to assessment strategies which can verify knowledge as well as capturing performative and behavioural dimensions of assessment. In this paper, we describe the process of developing different global rating scales measuring candidates’ competence in a performative assessment. We set out the following sequence, with three crucial phases, in the process of scale development: (a) Item Development, (b) Scale Development and (c) Piloting of the Scale. We note each phase’s different sub-steps. Despite the emergent need to ensure the competence of researchers using animals in scientific procedures, to our best knowledge there are very few species and procedure/skill specific assessment tools for this purpose, and the assessment methodology literature in the field is very limited. This paper provides guidance for those who need to develop and assess proficiency in laboratory animal procedures by setting out a method that can be used to create the required tools and illustrating how competence assessment strategies can be implemented.

Otoskills training during covid-19 pandemic: a before-after study

Background The ongoing COVID-19 pandemic has disrupted the surgical training of residents. There is a real concern that trainees will not be able to meet their training requirements. Low-fidelity surgical simulation appears to be an alternative for surgical training. The educational benefits of repeating ossiculoplasty simulations under a microscope have never been evaluated. With this study we aimed to evaluate the differences in performance scores and on a global rating scale before and after training on an ossiculoplasty simulator. Methods In this quasi-experimental, prospective, single-centre, before-after study with blinded rater evaluation, residents performed five microscopic ossiculoplasty tasks with a difficulty gradient (sliding beads onto rods, the insertion of a partial prosthesis, the insertion of a total prosthesis, and the insertion of a stapedotomy piston under microscopic or endoscopic surgery) before and after training on the same simulator. Performance scores were defined for each task, and total performance scores (score/min) were calculated. All data were collected prospectively. Results Six out of seven intermediate residents and 8/9 novices strongly agreed that the simulator was an effective training device and should be included in the ENT residency program. The mean effect of training was a significant increase in the total performance score (+ 0.52 points/min, [95 % CI, 0.40–0.64], p < 0.001), without a significant difference between novice and intermediate residents. Conclusions This preliminary study shows that techniques for middle-ear surgery can be acquired using a simulator, avoiding any risk for patients, even under lockdown measures.

Objective structured assessment of technical skill in temporal bone dissection: validation of a novel tool

Objective This study developed an assessment tool that was based on the objective structured assessment for technical skills principles, to be used for evaluation of surgical skills in cortical mastoidectomy. The objective structured assessment of technical skill is a well-established tool for evaluation of surgical ability. This study also aimed to identify the best material and printing method to make a three-dimensional printed temporal bone model. Methods Twenty-four otolaryngologists in training were asked to perform a cortical mastoidectomy on a three-dimensional printed temporal bone (selective laser sintering resin). They were scored according to the objective structured assessment of technical skill in temporal bone dissection tool developed in this study and an already validated global rating scale. Results Two external assessors scored the candidates, and it was concluded that the objective structured assessment of technical skill in temporal bone dissection tool demonstrated some main aspects of validity and reliability that can be used in training and performance evaluation of technical skills in mastoid surgery. Conclusion Apart from validating the new tool for temporal bone dissection training, the study showed that evolving three-dimensional printing technologies is of high value in simulation training with several advantages over traditional teaching methods.