Abstract
Introduction: Choosing Wisely ® (CW ®)is an initiative of the American Board of Internal Medicine Foundation created to guide the selection of care that is 1) supported by evidence, 2) not duplicative, 3) free from harm, and 4) truly necessary. Between 2013-14, the American Society of Hematology (ASH) published 10 recommendations in accordance with CW ® principles relevant to hematologic care. Previous studies have demonstrated that clinical exposure to non-malignant hematology (NMH) improves trainee understanding of evidence-based, cost effective care as outlined by ASH CW ®. However, dedicated clinical rotations in NMH for internal medicine (IM) residents are not consistently available. Therefore, we created a condensed educational curricular experience using a small group, case-based structure designed to teach the fundamentals of ASH CW ® in NMH to first year IM residents. With a pilot of 6 sessions, we investigated resident baseline knowledge, evaluated the curricular session, and assessed knowledge retention.
Methods: The educational intervention focused on 3 content areas in ASH CW ®: venous thromboembolism (VTE), heparin-induced thrombocytopenia (HIT), and sickle cell disease (SCD) (Table 1). Participants included 21 first year IM residents at a large academic medical center. A 1-hour small group teaching session was scheduled monthly as an assigned didactic for the 2020-21 academic year. A total of 6 sessions were provided, and each session had 2-4 residents assigned. The first 4 sessions were in-person and the final 2 were virtual due to the COVID-19 pandemic. The first author was the instructor at all sessions.
To assess baseline knowledge of the 3 content areas, participants completed an online assessment with 5 case-based multiple-choice questions at the beginning of the session. The instructor then guided participants to again complete the questionnaire together, now using internet access via a personal computer and a recommended list of online resources, including ASH Clinical Practice Guidelines and Pocket Guides. The instructor then led discussion on how each correct answer or guideline recommendation achieves the 4 CW ® principles. At the conclusion of the session, participants completed an online survey to evaluate the educational intervention using a modified Likert scale. To assess knowledge retention, participants received the original online multiple-choice assessment by email 3 months later.
Results: All participants (21/21, 100%) completed the baseline knowledge assessment. The average number of questions correct out of 5 total was 3.3 (67%), with a range of correct answers from 1 to 5. Table 2 includes the content area of each question and the number of correct responses. The question with the lowest total correct (9/21, 43%) addressed the use of transfusion in an uncomplicated pain crisis in SCD.
Seventeen participants (81%) completed the curricular evaluation. All respondents (17/17, 100%) either agreed or strongly agreed 1) that the session filled a gap in their NMH training and 2) that they learned something that would change their clinical practice. Only 1 participant (5%) reported completing a rotation in NMH prior to the session.
Six participants (29%) completed the repeat knowledge assessment at 3 months. All respondents (100%) achieved a perfect score on the multiple choice questions. When asked if the knowledge gained had influenced their clinical practice, 3 (50%) strongly agreed, 2 (33%) agreed, 1 (17%) was neutral, and none disagreed or strongly disagreed.
Conclusion: Our results demonstrate a successful educational pilot to improve the knowledge of ASH CW ® initiatives in NMH for first year IM residents using small group interactive case-based learning. Participants were overwhelmingly receptive to this intervention, expressed high satisfaction and confirmed that the session positively influenced their clinical practice. Although participation in the repeat assessment of knowledge was limited, those that did participate demonstrated high knowledge retention. We intend to expand this pilot initiative by providing the educational session for all incoming IM residents at our institution. We then plan to assess its impact on clinical practice (i.e. use of transfusion in SCD, use of thrombophilia testing, documentation of 4Ts score calculation) to apply the principles of ASH CW ® for improved patient care.
Figure 1 Figure 1.
Disclosures
No relevant conflicts of interest to declare.
Are multiple-choice questions a good tool for the assessment of clinical competence in Internal Medicine?