Creating Clinical Reasoning Assessment Tools in Different Languages: Adaptation of the Pediatric Emergency Medicine Script Concordance Test to Japanese

Introduction: Clinical reasoning is a crucial skill in the practice of pediatric emergency medicine and a vital element of the various competencies achieved during the clinical training of resident doctors. Pediatric emergency physicians are often required to stabilize patients and make correct diagnoses with limited clinical information, time and resources. The Pediatric Emergency Medicine Script Concordance Test (PEM-SCT) has been developed specifically for assessing physician's reasoning skills in the context of the uncertainties in pediatric emergency practice. In this study, we developed the Japanese version of the PEM-SCT (Jpem-SCT) and confirmed its validity by collecting relevant evidence.Methods: The Jpem-SCT was developed by translating the PEM-SCT into Japanese using the Translation, Review, Adjudication, Pretest, Documentation team translation model, which follows cross-cultural survey guidelines for proper translation and cross-cultural and linguistic equivalences between the English and Japanese version of the survey. First, 15 experienced pediatricians participated in the pre-test session, serving as a reference panel for modifying the test descriptions, incorporating Japanese context, and establishing the basis for the scoring process. Then, a 1-h test containing 60 questions was administered to 75 trainees from three academic institutions. Following data collection, we calculated the item-total correlations of the scores to optimize selection of the best items in the final version of the Jpem-SCT. The reliability of the finalized Jpem-SCT was calculated using Cronbach's α coefficient for ensuring generalizability of the evidence. We also conducted multiple regression analysis of the test score to collect evidence on validity of the extrapolation.Results: The final version of the test, based on item-total correlation data analysis, contained 45 questions. The participant's specialties were as follows: Transitional interns 12.0%, pediatric residents 56.0%, emergency medicine residents 25.3%, and PEM fellows 6.7%. The mean score of the final version of the Jpem-SCT was 68.6 (SD 9.8). The reliability of the optimized test (Cronbach's α) was 0.70. Multiple regression analysis showed that being a transitional intern was a negative predictor of test scores, indicating that clinical experience relates to performance on the Jpem-SCT.Conclusion: This pediatric emergency medicine Script Concordance Test was reliable and valid for assessing the development of clinical reasoning by trainee doctors during residency training.

The Use of a Modified Script Concordance Test in Clinical Rounds to Foster and Assess Clinical Reasoning Skills

The development of clinical reasoning skills is a high priority during clinical service, but an unpredictable case load and limited time for formal instruction makes it challenging for faculty to foster and assess students’ individual clinical reasoning skills. We developed an assessment for learning activity that helps students build their clinical reasoning skills based on a modified version of the script concordance test (SCT). To modify the standard SCT, we simplified it by limiting students to a 3-point Likert scale instead of a 5-point scale and added a free-text box for students to provide justification for their answer. Students completed the modified SCT during clinical rounds to prompt a group discussion with the instructor. Student feedback was positive, and the instructor gained valuable insight into the students’ thought process. A modified SCT can be adopted as part of a multimodal approach to teaching on the clinic floor. The purpose of this article is to describe our modifications to the standard SCT and findings from implementation in a clinical rounds setting as a method of formative assessment for learning and developing clinical reasoning skills.

Assessing the acceptability of script concordance testing: a nationwide study in otolaryngology

Background: Script concordance testing (SCT) is an objective method to evaluate clinical reasoning that assesses the ability to interpret medical information under conditions of uncertainty. Many studies have supported its validity as a tool to assess higher levels of learning, but little is known about its acceptability to major stakeholders. The aim of this study was to determine the acceptability of SCT to residents in otolaryngology – head and neck surgery (OTL-HNS) and a reference group of experts. Methods: In 2013 and 2016, a set of SCT questions, as well a post-test exit survey, were included in the National In-Training Examination (NITE) for OTL-HNS. This examination is administered to all OTL-HNS residents across Canada who are in the second to fifth year of residency. The same SCT questions and survey were then sent to a group of OTL-HNS surgeons from 4 Canadian universities. Results: For 64.4% of faculty and residents, the study was their first exposure to SCT. Overall, residents found it difficult to adapt to this form of testing, thought that the clinical scenarios were not clear and believed that SCT was not useful for assessing clinical reasoning. In contrast, the vast majority of experts felt that the test questions reflected real-life clinical situations and would recommend SCT as an evaluation method in OTL-HNS. Conclusion: Views about the acceptability of SCT as an assessment tool for clinical reasoning differed between OTL-HNS residents and experts. Education about SCT and increased exposure to this testing method are necessary to improve residents’ perceptions of SCT.

Script concordance testing to understand the hypothesis processes of undergraduate nursing students - Multiple case study

Background: Albeit essential to clinical reasoning (CR), strategies for generating student nursing clinical hypotheses at the time of transition to professional practice are underdeveloped. While script concordance testing (SCT) has been shown to be a valid and reliable assessment tool for CR in nursing education, the thought processes including the hypothesis processes involved in choosing an answer is not examined.Methods: A multiple case study was used to understand the complex phenomenon of students’ hypothesis activation and confrontation with the combined use of SCT questions and the think-aloud method. Structured individual interviews were conducted.Results: A total of 18 students, nine first-year and nine third-year students participated in the study. The results show that the students demonstrate certain CR cognitive processes, including early representation of a clinical situation, semantic transformation of data, and hypothesis comparison.Conclusions: Results suggest promoting knowledge articulation aloud and the frequent use of micro-judgments to compare and differentiate hypotheses involving the uncertainty of clinical practice, which underpin learning in successive layers.

Individual versus Team-Based responses to Script Concordance Test in Teaching Clinical Reasoning

Introduction: Although clinical competence is multi-dimensional and should be acquired by each medical student, but most students learn clinical reasoning skills informally in clinical rotations. Accordingly, A prospective quasi-experimental study was conducted aiming to evaluate the merging of Script Concordance Test (SCT) and Team Based Learning (TBL) as a teaching/learning approach in clinical setting for medical students. Methodology: The study ran in three phases. Phase 1 (preparatory phase) involved students’ preparation and preparation of SCT. Phase 2 (implementation phase) included application of individual and team SCT (iSCT and tSCT respectively). Phase 3 (evaluation phase) compared score results and obtained students’ feedback.Results: Significant differences existed when comparing individual students’ response or students’ teams’ responses with experts scores. However, the use of the SCT/TBL approach had improved the clinical reasoning skills of the students in some vignettes and helped the lower achievers through the tSCT. The students found the approach appropriate for teaching or formatively assessing clinical reasoning. It helped them to discuss, correct their mistakes and improve their problem solving and reasoning skills. Conclusion: team-based learning improved students’ responses, especially the lower achievers, to script concordance test. SCT/TBL approach can be used to teach clinical reasoning for undergraduate students.