Resident evaluations in the age of competency-based medical education: faculty perspectives on minimizing burdens

2020 ◽  
pp. 1-6
Author(s):  
Jessica E. Rabski ◽  
Ashirbani Saha ◽  
Michael D. Cusimano
Keyword(s):  
Medical Education ◽  
Mobile Application ◽  
Postgraduate Medical Education ◽  
Professional Activity ◽  
Successful Implementation ◽  
Neurosurgical Training ◽  
Complete Evaluation ◽  
Competency Based Medical Education ◽  
Main Challenge ◽  
Competency Based

OBJECTIVECompetency-based medical education (CBME), an outcomes-based approach to medical education, continues to be implemented across many postgraduate medical education programs worldwide, including a recent introduction into Canadian neurosurgical training programs (July 2019). The success of this educational paradigm shift requires frequent faculty observation and evaluation of residents performing defined tasks of the specialty. A main challenge involves providing residents with frequent performance evaluations and feedback that are feasible for faculty to complete. This study aims to define what is currently happening and what changes are needed to make CBME successful for the certification of neurosurgeons’ competence.METHODSA 55-item questionnaire was emailed nationwide to survey Canadian neurosurgical faculty.RESULTSFifty-two complete responses were received and achieved a distribution highly correlated with the number of faculty neurosurgeons practicing in each Canadian province (Pearson’s r = 0.94). Two-thirds (35/52) of faculty reported currently taking a median of 10 minutes to complete evaluation forms at the end of a resident’s rotation block. Regardless of the faculty’s province of practice (p = 0.50) or years of experience (p = 0.06), they reported 3 minutes (minimum 1 minute, maximum 10 minutes, interquartile range [IQR] 3 minutes) as a feasible amount of time to spend completing an evaluation form following an observation of a resident’s performance of an entrustable professional activity (EPA). If evaluation forms took 3 minutes to complete, 85% of respondents (44/52) would complete EPA evaluations weekly or daily. The faculty recommended 5 minutes as a feasible amount of time to provide oral feedback (minimum 1 minute, maximum 20 minutes, IQR 3.25 minutes), which was significantly higher (p = 0.00099) than their recommended amount of time for completing evaluation forms. The majority of faculty (71%) stated they would prefer to access resident evaluation forms through a mobile application compared to a paper form (12%), an evaluation website (8%), or through a URL link sent via email (10%; p = 0.0032).CONCLUSIONSTo facilitate the successful implementation of CBME into a neurosurgical training curriculum, resident EPA assessment forms should take 3 minutes or less to complete and be accessible through a mobile application.

scholarly journals Black Ice: ways to get a grip on resident co-production within medical education change

2020 ◽  
Author(s):  
Jeffery D. Dagnone ◽  
Samantha Buttemer ◽  
Jena Hall ◽  
Liora Berger ◽  
Kristen Weersink
Keyword(s):  
Medical Education ◽  
Postgraduate Training ◽  
Postgraduate Medical Education ◽  
Change Leadership ◽  
Successful Implementation ◽  
Competency Based Medical Education ◽  
Education Change ◽  
Competency Based ◽  
Queen's University ◽  
National Approach

The Royal College of Physicians and Surgeons of Canada (RCPSC) is transforming its national approach to postgraduate medical education by transitioning all specialty programs to competency based medical education (CBME) curriculums over a seven-year period. Queen’s University, with special permission from the RCPSC, launched CBME curricula for all incoming residents across its 29 specialty programs in July 2017. Resident engagement, empowerment, and co-production through this transition has been instrumental in successful implementation of CBME at Queen’s University. This article aims to use our own experience at Queen’s in the context of current literature and rooted in change leadership theory, to provide a guide for educators, learners, and institutions on how to leverage the interest and enthusiasm of trainees in the transition to CBME in postgraduate training. The following ten tips provides a model for avoiding the “black ice” type pitfalls that can arise with learner involvement, and ensure a smoother transition for other institutions moving forward with CBME implementation.

scholarly journals Building a Shared Mental Model of Competence Across the Continuum: Trainee Perceptions of Subinternships for Residency Preparation

2021 ◽  
Vol 8 ◽  
pp. 238212052110633
Author(s):  
Johannah M. Scheurer ◽  
Cynthia Davey ◽  
Anne G. Pereira ◽  
Andrew P. J. Olson
Keyword(s):  
Medical Education ◽  
Response Rate ◽  
Professional Activity ◽  
University Of Minnesota ◽  
Competency Based Medical Education ◽  
Chi Squared ◽  
Competency Based ◽  
Kappa Agreement ◽  
Physician Competency ◽  
The Continuum

INTRODUCTION Toward a vision of competency-based medical education (CBME) spanning the undergraduate to graduate medical education (GME) continuum, University of Minnesota Medical School (UMMS) developed the Subinternship in Critical Care (SICC) offered across specialties and sites. Explicit course objectives and assessments focus on internship preparedness, emphasizing direct observation of handovers (Core Entrustable Professional Activity, “EPA,” 8) and cross-cover duties (EPA 10). METHODS To evaluate students’ perceptions of the SICC's and other clerkships’ effectiveness toward internship preparedness, all 2016 and 2017 UMMS graduates in GME training ( n = 440) were surveyed regarding skill development and assessment among Core EPAs 1, 4, 6, 8, 9, 10. Analysis included descriptive statistics plus chi-squared and Kappa agreement tests. RESULTS Respondents ( n = 147, response rate 33%) rated the SICC as a rotation during which they gained most competence among EPAs both more (#4, 57% rated important; #8, 75%; #10, 70%) and less explicit (#6, 53%; #9, 69%) per rotation objectives. Assessments of EPA 8 (80% rated important) and 10 (76%) were frequently perceived as important toward residency preparedness. Agreement between importance of EPA development and assessment was moderate (Kappa = 0.40-0.59, all surveyed EPAs). CONCLUSIONS Graduates’ perceptions support the SICC's educational utility and assessments. Based on this and other insight from the SICC, the authors propose implications toward collectively envisioning the continuum of physician competency.

scholarly journals Adapting Feedback to Individual Residents: An Examination of Preceptor Challenges and Approaches

2018 ◽  
Vol 10 (2) ◽  
pp. 168-175 ◽  
Author(s):  
Amanda Roze des Ordons ◽  
Adam Cheng ◽  
Jonathan Gaudet ◽  
James Downar ◽  
Jocelyn Lockyer
Keyword(s):  
Medical Education ◽  
Professional Activity ◽  
Framework Analysis ◽  
Competency Based Medical Education ◽  
Performance Levels ◽  
Skill Sets ◽  
Entrustable Professional Activity ◽  
Competency Based ◽  
Individualize Feedback ◽  
Insight Into

ABSTRACT Background  Feedback conversations between preceptors and residents usually occur in closed settings. Little is known about how preceptors address the challenges posed by residents with different skill sets, performance levels, and personal contexts. Objective  This study explored the challenges that preceptors experienced and approaches taken in adapting feedback conversations to individual residents. Methods  In 2015, 18 preceptors participated in feedback simulations portraying residents with variations in skill, insight, confidence, and distress, followed by debriefing of the feedback conversation with a facilitator. These interactions were recorded, transcribed, and analyzed using thematic and framework analysis. Results  The preceptors encountered common challenges with feedback conversations, including uncertainty in how to individualize feedback to residents and how to navigate tensions between resident- and preceptor-identified goals. Preceptors questioned their ability to enhance skills for highly performing residents, whether they could be directive when residents had insight gaps, how they could reframe the perceptions of the overly confident resident, and whether they should offer support to emotionally distressed residents or provide feedback about performance. Preceptors adapted their approach to feedback, drawing on techniques of coaching for highly performing residents, directing for residents with insight gaps, mediation with overly confident residents, and mentoring with emotionally distressed residents. Conclusions  Examining the feedback challenges preceptors encounter and the approaches taken to adapt feedback to individual residents can provide insight into how preceptors meet the challenges of competency-based medical education, in which frequent, focused feedback is essential for residents to achieve educational milestones and entrustable professional activity expectations.

Competency-based medical education in postgraduate medical education

2010 ◽  
Vol 32 (8) ◽  
pp. 651-656 ◽  
Author(s):  
William F. Iobst ◽  
Jonathan Sherbino ◽  
Olle Ten Cate ◽  
Denyse L. Richardson ◽  
Deepak Dath ◽  
...  
Keyword(s):  
Medical Education ◽  
Postgraduate Medical Education ◽  
Competency Based Medical Education ◽  
Competency Based

scholarly journals Analyzing the Administrative Burden of Competency Based Medical Education

2021 ◽  
pp. 084653712110389
Author(s):  
Kevin Cheung ◽  
Christina Rogoza ◽  
Andrew D. Chung ◽  
Benjamin Yin Ming Kwan
Keyword(s):  
Medical Education ◽  
Postgraduate Medical Education ◽  
Assessment Process ◽  
Full Time ◽  
Equal Distribution ◽  
Competency Based Medical Education ◽  
Quantitative Evidence ◽  
Administrative Burden ◽  
Professional Activities ◽  
Competency Based

Purpose: Postgraduate residency programs in Canada are transitioning to a competency-based medical education (CBME) system. Within this system, resident performance is documented through frequent assessments that provide continual feedback and guidance for resident progression. An area of concern is the perception by faculty of added administrative burden imposed by the frequent evaluations. This study investigated the time spent in the documentation and submission of required assessment forms through analysis of quantitative data from the Queen’s University Diagnostic Radiology program. Methods and Materials: Data regarding time taken to complete Entrustable Professional Activities (EPA) assessments was collected from 24 full-time and part-time radiologists over a period of 18 months. This data was analyzed using SPSS to determine mean time of completion by individuals, departments, and by experience with the assessment process. Results: The average time taken to complete an EPA assessment form was 3 minutes and 6 seconds. Assuming 3 completed EPA assessment forms per week for each resident (n = 12) and equal distribution among all staff, this averaged out to an additional 18 minutes of administrative burden per staff member over a 4 week block. Conclusions: This study investigated the perception by faculty of additional administrative burden for assessment in the CBME framework. The data provided quantitative evidence of administrative burden for the documentation and submission of assessments. The data indicated that the added administrative burden may be reasonable given mandate for CBME implementation and the advantages of adoption for postgraduate medical education.

scholarly journals Sustainability of physical exam skills in a resident-led curriculum in a large internal medicine program with competency based medical education

2018 ◽  
Vol 9 (4) ◽  
pp. e78-92 ◽  
Author(s):  
Don Thiwanka Wijeratne ◽  
Siddhartha Srivastava ◽  
Barry Chan ◽  
Wilma Hopman ◽  
Benjamin Thomson
Keyword(s):  
Internal Medicine ◽  
Medical Education ◽  
Physical Examination ◽  
Peer Assessment ◽  
Physical Exam ◽  
Professional Activity ◽  
Faculty Assessment ◽  
Competency Based Medical Education ◽  
Competency Based ◽  
Faculty Assessments

Background: Competency Based Medical Education (CBME) designates physical examination competency as an Entrustable Professional Activity (EPA). Considerable concern persists regarding the increased time burden CBME may place on educators. We developed a novel physical examination curriculum that shifted the burden of physical examination case preparation and performance assessment from faculty to residents. Our first objective was to determine if participation led to sustainable improvements in physical examination skills. The second objective was to determine if resident peer assessment was comparable to faculty assessment.    Methods: We selected physical exam case topics based on the Objectives of Training in the Specialty of Internal Medicine as prescribed by the Royal College of Physicians and Surgeons of Canada. Internal Medicine residents compiled evidence-based physical exam checklists that faculty reviewed before distribution to all learners. Physical exam practice sessions with whole-group demonstration followed by small-group practice sessions were performed weekly. We evaluated this pilot curriculum with a formative OSCE, during which a resident peer and a faculty member simultaneously observed and assessed examinee performance by .Results: Participation in the novel curriculum practice sessions improved OSCE performance (faculty score mean 78.96 vs. 62.50, p<0.05). Peer assessment overestimated faculty scores (76.2 vs. 65.7, p<0.001), but peer and faculty assessments were highly correlated (R2 = 0.73 (95% CI 0.50-0.87).Conclusion: This novel physical examination curriculum leads to sustainable improvement of physical examination skills. Peer assessment correlated well with the gold standard faculty assessment. This resident-led physical examination curriculum enhanced physical examination skills in a CBME environment, with minimal time commitment from faculty members.

scholarly journals Towards a shared mental model of progressive competence in postgraduate medical education

2018 ◽  
Vol 9 (3) ◽  
pp. e115-118
Author(s):  
Eric Prost
Keyword(s):  
Medical Education ◽  
Mental Model ◽  
Postgraduate Medical Education ◽  
Competency Based Medical Education ◽  
Shared Mental Model ◽  
Competency Based ◽  
The Military ◽  
Postgraduate Medicine

Many professions have hierarchies and a promotion structure. Postgraduate medicine has a tradition of promoting residents based on time spent in a certain specialty. The military, too, may promote its personnel based on factors other than just merit. Both professions have been criticized for divorcing competence from promotion. While Competency-Based Medical Education (CBME) partly solves this problem in medicine, many models of CBME, including the Canadian one, retain distinct stages of training. We urgently need a shared mental model of what a learner in each stage looks like. Some models have been proposed but fall short.

Anesthesia residency training in regional anesthesiology and acute pain medicine: a competency-based model curriculum

2020 ◽  
Vol 45 (8) ◽  
pp. 660-667
Author(s):  
Glenn Woodworth ◽  
Robert B Maniker ◽  
Christina M Spofford ◽  
Ryan Ivie ◽  
Nathalie I Lunden ◽  
...  
Keyword(s):  
Medical Education ◽  
Residency Training ◽  
Acute Pain ◽  
Mobile Application ◽  
Practical Implementation ◽  
Pain Medicine ◽  
Web Based ◽  
Competency Based Medical Education ◽  
Educational Framework ◽  
Competency Based

The Accreditation Council for Graduate Medical Education has shifted to competency-based medical education. This educational framework requires the description of educational outcomes based on the knowledge, skills and behaviors expected of competent trainees. It also requires an assessment program to provide formative feedback to trainees as they progress to competency in each outcome. Critical to the success of a curriculum is its practical implementation. This article describes the development of model curricula for anesthesiology residency training in regional anesthesia and acute pain medicine (core and advanced) using a competency-based framework. We further describe how the curricula were distributed through a shared web-based platform and mobile application.

scholarly journals From prescription to guidance: a European framework for generic competencies

2019 ◽  
Vol 25 (1) ◽  
pp. 173-187 ◽  
Author(s):  
Jessica E. van der Aa ◽  
Anna J. M. Aabakke ◽  
Betina Ristorp Andersen ◽  
Annette Settnes ◽  
Peter Hornnes ◽  
...  
Keyword(s):  
Medical Education ◽  
Change Management ◽  
Postgraduate Medical Education ◽  
Competency Framework ◽  
Competency Based Medical Education ◽  
Generic Competencies ◽  
Obstetrics And Gynaecology ◽  
Competency Based ◽  
Hospital Boards ◽  
Management Theories

Abstract In postgraduate medical education, required competencies are described in detail in existing competency frameworks. This study proposes an alternative strategy for competency-based medical education design, which is supported by change management theories. We demonstrate the value of allowing room for re-invention and creative adaptation of innovations. This new strategy was explored for the development of a new generic competency framework for a harmonised European curriculum in Obstetrics and Gynaecology. The generic competency framework was developed through action research. Data were collected by four European stakeholder groups (patients, nurses, midwives and hospital boards), using a variety of methods. Subsequently, the data were analysed further in consensus discussions with European specialists and trainees in Obstetrics and Gynaecology. These discussions ensured that the framework provides guidance, is specialty-specific, and that implementation in all European countries could be feasible. The presented generic competency framework identifies four domains: ‘Patient-centred care’, ‘Teamwork’, ‘System-based practice’ and ‘Personal and professional development’. For each of these four domains, guiding competencies were defined. The new generic competency framework is supported by European specialists and trainees in Obstetrics and Gynaecology, as well as by their European stakeholders. According to change management theories, it seems vital to allow room for re-invention and creative adaptation of the competency framework by medical professionals. Therefore, the generic competency framework offers guidance rather than prescription. The presented strategy for competency framework development offers leads for implementation of competency-based medical education as well as for development of innovations in postgraduate medical education in general.

scholarly journals Teaching and assessing intra-operative consultations in competency-based medical education: development of a workplace-based assessment instrument

2021 ◽  
Author(s):  
Marcio M. Gomes ◽  
David Driman ◽  
Yoon Soo Park ◽  
Timothy J. Wood ◽  
Rachel Yudkowsky ◽  
...  
Keyword(s):  
Medical Education ◽  
Assessment Tools ◽  
Assessment Instrument ◽  
Professional Activity ◽  
Cultural Issues ◽  
Education Development ◽  
Competency Based Medical Education ◽  
Workplace Based Assessment ◽  
Competency Based ◽  
Validity Theory

AbstractCompetency-based medical education (CBME) is being implemented worldwide. In CMBE, residency training is designed around competencies required for unsupervised practice and use entrustable professional activities (EPAs) as workplace “units of assessment”. Well-designed workplace-based assessment (WBA) tools are required to document competence of trainees in authentic clinical environments. In this study, we developed a WBA instrument to assess residents’ performance of intra-operative pathology consultations and conducted a validity investigation. The entrustment-aligned pathology assessment instrument for intra-operative consultations (EPA-IC) was developed through a national iterative consultation and used clinical supervisors to assess residents’ performance at an anatomical pathology program. Psychometric analyses and focus groups were conducted to explore the sources of evidence using modern validity theory: content, response process, internal structure, relations to other variables, and consequences of assessment. The content was considered appropriate, the assessment was feasible and acceptable by residents and supervisors, and it had a positive educational impact by improving performance of intra-operative consultations and feedback to learners. The results had low reliability, which seemed to be related to assessment biases, and supervisors were reluctant to fully entrust trainees due to cultural issues. With CBME implementation, new workplace-based assessment tools are needed in pathology. In this study, we showcased the development of the first instrument for assessing resident’s performance of a prototypical entrustable professional activity in pathology using modern education principles and validity theory.

Sign in / Sign up

Export Citation Format

Share Document