Programmatic Assessment in Emergency Medicine: Implementation of Best Practices

ABSTRACT Background Programmatic assessment is the intentional collection of key data from multiple sources for both assessment of learning and assessment for learning. Objective We developed a system of programmatic assessment (PA) to identify competency progression (summative) and assessment for learning to assist residents in their formative development. Methods The programmatic assessment was designed iteratively from 2014 through 2016. All assessments were first categorized by competency domain and source of assessment. The number of assessment modalities for each competency domain was collected. These multisource assessments were then mapped by program leadership to the milestones to develop a master PA blueprint. A resident learning management system provided the platform for aggregating formative and summative data, allowing residents and faculty ongoing access to guide learning and assessment. A key component of programmatic assessment was to support resident integration of assessment information through feedback by faculty after shifts and during monthly formal assessments, semiannual resident reviews, and summative judgments by the Clinical Competency Committee. Results Through the PA, the 6 competency domains are assessed through multiple modalities: patient care (22 different assessments), professionalism (18), systems-based practice (17), interprofessional and communication skills (16), medical knowledge (11), and practice-based learning and improvement (6). Each assessment provides feedback to the resident in various formats. Our programmatic assessment has been utilized for more than 2 years with iterative improvements. Conclusions The implementation of programmatic assessment allowed our program to organize diverse, multisourced feedback to drive both formative and summative assessments.

Download Full-text

The Entextualization of Ayurveda as Intellectual Property

International Journal of Cultural Property ◽

10.1017/s0940739112000197 ◽

2012 ◽

Vol 19 (3) ◽

pp. 313-343 ◽

Cited By ~ 3

Author(s):

Matthew Wolfgram

Keyword(s):

Intellectual Property ◽

Global Economy ◽

Medical Knowledge ◽

Linguistic Anthropology ◽

Medical Discourse ◽

Complex Nature ◽

Multiple Sources ◽

The Social ◽

Rights Discourse ◽

The Individual

AbstractThis article documents the practices of pharmaceutical creativity in Ayurveda, focusing in particular on how practitioners appropriate multiple sources to innovate medical knowledge. Drawing on research in linguistic anthropology on the social circulation of discourse—a process calledentextualization—I describe how the ways in which Ayurveda practitioners innovate medical knowledge confounds the dichotomous logic of intellectual property (IP) rights discourse, which opposes traditional collective knowledge and modern individual innovation. While it is clear that these categories do not comprehend the complex nature of creativity in Ayurveda, I also use the concept of entextualization to describe how recent historical shifts in the circulation of discourse have caused a partial entailment of this opposition between the individual and the collectivity. Ultimately, I argue that the method exemplified in this article of tracking the social circulation of medical discourse highlights both the empirical complexity of so-called traditional creativity, and the politics of imposing the categories of IP rights discourse upon that creativity, situated as it often is, at the margins of the global economy.

Download Full-text

Finding Our Voice: Highly Flexible ED for the HyFlex World

To Improve the Academy ◽

10.3998/tia.467 ◽

2021 ◽

Vol 40 (2) ◽

Author(s):

Kristine Larsen ◽

Christina Robinson ◽

Jason A. Melnyk ◽

Jennifer Nicoletti ◽

Amy Gagnon ◽

...

Keyword(s):

Online Learning ◽

Educational Development ◽

Team Approach ◽

Multiple Sources ◽

Multiple Modalities

The COVID-19 pandemic brought about unprecedented changes in our approach to delivering educational development (ED) programming. In this article we discuss how our dual ED centers pivoted during the sudden switch to online learning, highlighting how we overcame challenges such as a small staff, tight timelines, and faculty anxieties. Particularly, we explore how we adapted to the university’s investment in technologically advanced Hybrid-Flexible (HyFlex) classroom spaces and utilized a multi-pronged team approach to provide effective and timely ED to faculty. By identifying key faculty leaders, identifying multiple sources of data, and using multiple modalities, we supported the faculty in their mission to effectively serve their students during this difficult and stressful time. In pivoting from a triage approach to more tactically focused development, the two ED centers discovered that they could more effectively serve faculty (and by extension students) by shattering the structural silos that had previously defined them and instead working as a unified entity.

Download Full-text

Programmatic assessment: From assessment of learning to assessment for learning

Medical Teacher ◽

10.3109/0142159x.2011.565828 ◽

2011 ◽

Vol 33 (6) ◽

pp. 478-485 ◽

Cited By ~ 281

Author(s):

Lambert W. T. Schuwirth ◽

Cees P. M. Van der Vleuten

Keyword(s):

Assessment For Learning ◽

Programmatic Assessment ◽

Assessment Of Learning

Download Full-text

Transforming assessment – Critical reflections around resolving tensions between assessment for learning and of learning

10.14742/ascilite2020.0143 ◽

2020 ◽

Author(s):

Sabina Cerimagic ◽

Priya Khanna

Keyword(s):

Business Education ◽

Medical Curriculum ◽

Curricular Reform ◽

Programmatic Assessment ◽

Graduate Attributes ◽

Summative Assessments ◽

Assessment Systems ◽

Current Assessment ◽

The University ◽

Medical Programs

Current assessment and progression policies and practices in higher education are largely dominated by dichotomy between formative and summative assessments. Given assessments are major drivers of learning, such as dichotomous assessment systems which promote the learning that can only be tested via summative assessments and falls short in fostering complex graduate attributes - such as interdisciplinary competencies. Medical programs, across the globe are now embracing emerging concepts such as programmatic assessment that allows the assessments to serve ‘for, of and as’ learning. Given the most challenging curricular reform in any field, including Business is reforming assessments, this paper critically reflects on how emerging concepts such as programmatic assessment are promising in resolving the tensions created by dichotomy of assessment for vs as learning. The reflections are based on rapid scoping review of programmatic assessment in Medical education, as well as authors’ familiarity with implementation of programmatic assessment in the Sydney Medical School renewed curriculum. There are many lessons that can be learned and carried over from the University of Sydney Medical curriculum to the Business education space.

Download Full-text

Transitioning From a Noon Conference to an Academic Half-Day Curriculum Model: Effect on Medical Knowledge Acquisition and Learning Satisfaction

Journal of Graduate Medical Education ◽

10.4300/jgme-d-13-00185.1 ◽

2014 ◽

Vol 6 (1) ◽

pp. 93-99 ◽

Cited By ~ 13

Author(s):

Duc Ha ◽

Michael Faulx ◽

Carlos Isada ◽

Michael Kattan ◽

Changhong Yu ◽

...

Keyword(s):

Internal Medicine ◽

Knowledge Acquisition ◽

Cleveland Clinic ◽

Medical Knowledge ◽

Learner Satisfaction ◽

Learning Satisfaction ◽

Medical Licensing ◽

Sex Type ◽

Multivariable Analyses ◽

Resident Learning

Abstract Background The academic half-day (AHD) curriculum is an alternative to the traditional noon conference in graduate medical education, yet little is known regarding its effect on knowledge acquisition and resident satisfaction. Objective We investigated the association between the 2 approaches for delivering the curriculum and knowledge acquisition, as reflected by the Internal Medicine In-Training Examination (IM-ITE) scores and assessed resident learning satisfaction under both curricula. Methods The Cleveland Clinic Internal Medicine Residency Program transitioned from the noon conference to the AHD curriculum in 2011. Covariates for residents enrolled from 2004 to 2011 were age; sex; type of medical degree; United States Medical Licensing Examination Step 1, 2 Clinical Knowledge; and IM-ITE-1 scores. We performed univariable and multivariable linear regressions to investigate the association between covariates and IM-ITE-2 and IM-ITE-3 scores. Residents also were surveyed about their learning satisfaction in both curricula. Results Of 364 residents, 112 (31%) and 252 (69%) were exposed to the AHD and the noon conference curriculum, respectively. In multivariable analyses, the AHD curriculum was associated with higher IM-ITE-3 (regression coefficient, 4.8; 95% confidence interval 2.9–6.6) scores, and residents in the AHD curriculum had greater learning satisfaction compared with the noon conference cohort (Likert, 3.4 versus 3.0; P = .003). Conclusions The AHD curriculum was associated with improvement in resident medical knowledge acquisition and increased learner satisfaction.

Download Full-text

Aligning community-engaged research competencies with online training resources across the Clinical and Translational Science Award Consortium

Journal of Clinical and Translational Science ◽

10.1017/cts.2020.538 ◽

2020 ◽

pp. 1-8

Author(s):

Rebecca J. Piasecki ◽

Elisa D. Quarles ◽

Mona N. Bahouth ◽

Anwesha Nandi ◽

Alicia Bilheimer ◽

...

Keyword(s):

Core Competencies ◽

Core Competency ◽

Online Resources ◽

Translational Science ◽

Community Engaged Research ◽

Research Competencies ◽

Resource Classification ◽

Competency Domain ◽

Training Resources ◽

Competency Domains

Abstract Introduction: The extent to which Clinical and Translational Science Award (CTSA) programs offer publicly accessible online resources for training in community-engaged research (CEnR) core competencies is unknown. This study cataloged publicly accessible online CEnR resources from CTSAs and mapped resources to CEnR core competency domains. Methods: Following a search and review of the current literature regarding CEnR competencies, CEnR core competency domains were identified and defined. A systematic review of publicly accessible online CEnR resources from all 64 current CTSAs was conducted between July 2018 and May 2019. Resource content was independently reviewed by two reviewers and scored for the inclusion of each CEnR core competency domain. Domain scores across all resources were assessed using descriptive statistics. Results: Eight CEnR core competency domains were identified. Overall, 214 CEnR resources publicly accessible online from 35 CTSAs were eligible for review. Scoring discrepancies for at least one domain within a resource initially occurred in 51% of resources. “CEnR methods” (50.5%) and “Knowledge and relationships with communities” (40.2%) were the most frequently addressed domains, while “CEnR program evaluation” (12.1%) and “Dissemination and advocacy” (11.2%) were the least frequently addressed domains. Additionally, challenges were noted in navigating CTSA websites to access CEnR resources, and CEnR competency nomenclature was not standardized. Conclusions: Our findings guide CEnR stakeholders to identify publicly accessible online resources and gaps to address in CEnR resource development. Standardized nomenclature for CEnR competency is needed for effective CEnR resource classification. Uniform organization of CTSA websites may maximize navigability.

Download Full-text

Three Goals of Assessment in Medical Education

Ramathibodi Medical Journal ◽

10.33165/rmj.2019.42.3.176195 ◽

2019 ◽

Vol 42 (3) ◽

pp. 76-83

Author(s):

Atipong Pathanasethpong

Keyword(s):

Medical Education ◽

Assessment For Learning ◽

Comprehensive Approach ◽

Learning Assessment ◽

Programmatic Assessment ◽

Education Assessment ◽

Assessment Of Learning ◽

Assessment As Learning ◽

Single Method ◽

Method Of Assessment

Assessment is closely connected to learning since assessment allows a learner’s current competencies to be measured. The results of such a measurement can then be used to produce learning. There are three goals for assessment in medical education: assessment of learning, assessment for learning, and assessment as learning. These three goals serve different purposes and therefore differ in how they are carried out. They also require different approaches for the assessors. It is crucial that all three goals be balanced and attained lest the testing culture wax and the learning culture wane, resulting in a situation in which learners place too much emphasis on passing the tests and not enough emphasis on learning and growing. No single method of assessment can sufficiently achieve all three goals. One comprehensive approach to achieve as well as balance all three goals is to utilize programmatic assessment, in which various methods of assessment are employed based on their strengths and how they can cover each other’s limitations.

Download Full-text

Resident Fuel Levels: Reframing, Assessing, and Addressing Well-Being

Journal of Graduate Medical Education ◽

10.4300/jgme-d-17-00536.1 ◽

2018 ◽

Vol 10 (2) ◽

pp. 198-202 ◽

Cited By ~ 3

Author(s):

Shannon Amerilda Scielzo ◽

David C. Weigle ◽

Salahuddin (Dino) Kazi

Keyword(s):

Well Being ◽

Learning Programs ◽

Program Leadership ◽

Being There ◽

Resident Learning ◽

The Relationship ◽

Academic Year ◽

E Mail ◽

Ongoing Monitoring

ABSTRACT Background To optimize resident learning, programs need to readily assess resident well-being. There is a lack of easy-to-use, acceptable instruments for this task. Objective We created a well-being “fuel gauge,” and assessed the acceptability and feasibility of this weekly electronic communication pipeline for residents to report and discuss their well-being. Methods A well-being fuel gauge assessment was administered weekly over the course of 1 academic year (July 2016 to June 2017) in a large internal medicine residency program. The well-being gauge asked residents to report their fuel levels using a 1 to 5 Likert-type scale (1, empty; 3, half tank; and 5, full tank). Residents who provided low scores (1 or 2) were contacted by program leadership, and the program director sent weekly e-mail updates that addressed residents' comments on their well-being fuel gauge. Results Of 163 residents, 149 (91%) provided data on their well-being fuel gauge, with a 53% average weekly response rate. Fifty-four percent of residents (80 of 149) reported a low score over the course of the year, and 4 residents only used the assessment to report a low score. Comments on average consisted of 280 characters (SD = 357) and were lengthier and more prevalent with lower fuel gauge scores. We analyzed the relationship between scores and comments. Conclusions The well-being fuel gauge was well accepted by most residents and was easy to administer and to oversee by program directors. It facilitated ongoing monitoring of well-being and follow-up to address factors contributing to low well-being.

Download Full-text

How ‘Testing’ Has Become ‘Programmatic Assessment for Learning’

Health Professions Education ◽

10.1016/j.hpe.2018.06.005 ◽

2019 ◽

Vol 5 (3) ◽

pp. 177-184 ◽

Cited By ~ 5

Author(s):

Lambert W.T. Schuwirth ◽

Cees P.M. van der Vleuten

Keyword(s):

Assessment For Learning ◽

Programmatic Assessment

Download Full-text

Developing nurse preceptor competency domain guide tool: A Delphi study

Journal of Nursing Education and Practice ◽

10.5430/jnep.v12n5p33 ◽

2021 ◽

Vol 12 (5) ◽

pp. 33

Author(s):

Mudhar Al Adawi ◽

Ibtisam Al Siyabi ◽

Nasra Al Hashmi ◽

Fatma Mahmood AbdulRasool ◽

Asma Al Harrasi ◽

...

Keyword(s):

Delphi Study ◽

Time Management ◽

Teaching And Learning ◽

Core Competency ◽

Tertiary Hospital ◽

Clinical Practices ◽

Competency Domain ◽

Competency Domains ◽

Nurse Preceptors ◽

Nurse Preceptor

Background and objective: One of the strategies used to prepare novice nurses for their professional journeys in clinical practice is by implementing the preceptorship teaching and learning model. Competencies such as knowledge, experience, abilities, and attributes need to be measured to ensure the desired outcomes of the preceptorship are achieved and consistent. This study aimed to develop a nurse preceptor competency domain guide tool at a tertiary hospital in Oman.Methods: Three-round Delphi iterative design with experts was used to develop the nurse preceptor competency domain guide tool. Following standard measures, eight expert opinions were combined until a group consensus was achieved. The level of consensus within the expert panel was defined as ≥ 75% scoring of items were selected as an essential required competency/item.Results: Eight experts from a main tertiary hospital were included in the panel. Five core competency domains and five subdomains were identified and considered to be relevant for nurse preceptors at the hospital with consensus levels varying from 75% to 100%. A total of 83 descriptive items were identified for the competency guide tool.Conclusions: This study found that the main core competency domains of the tool that nurse preceptors should acquire to be competent preceptors are inter-professional communication skills, appropriate teaching strategies, time management skills, building a learning atmosphere, and coaching critical thinking. This tool would improve nurse preceptors’ performance and equip them with the required prerequisite competencies to professionally start their journey in clinical practices. Follow-up research on tool implementation is highly recommended to evaluate its effectiveness.

Download Full-text