scholarly journals Developing a dashboard for faculty development in competency-based training programs: a design-based research project

2021 ◽  
Author(s):  
Yusuf Yilmaz ◽  
Robert Carey ◽  
Teresa Chan ◽  
Venkat Bandi ◽  
Shisong Wang ◽  
...  
Keyword(s):  
Faculty Development ◽  
Training Programs ◽  
Research Process ◽  
Assessment Data ◽  
Design Based Research ◽  
Program Leaders ◽  
Professional Activities ◽  
Narrative Assessment ◽  
Competency Based ◽  
The University

Background: Canadian specialist residency training programs are implementing a form of competency-based medical education (CBME) that requires frequent assessments of entrustable professional activities (EPAs). Faculty struggle to provide helpful feedback and assign appropriate entrustment scores. CBME faculty development initiatives rarely incorporate teaching metrics. Dashboards could be used to visualize faculty assessment data to support faculty development. Methods: Using a design-based research process, we identified faculty development needs related to CBME assessments and designed a dashboard containing elements (data, analytics, and visualizations) meeting these needs. Data was collected within the emergency medicine residency program at the University of Saskatchewan through interviews with program leaders, faculty development experts, and faculty participating in development sessions. Two investigators thematically analyzed interview transcripts to identify faculty needs that were audited by a third investigator. The needs were described using representative quotes and the dashboard elements designed to address them. Results: Between July 1, 2019 and December 11, 2020 we conducted 15 interviews with nine participants (two program leaders, three faculty development experts, and four faculty members). Three needs emerged as themes from the analysis: analysis of assessments, contextualization of assessments, and accessible reporting. We addressed these needs by designing an accessible dashboard to present contextualized quantitative and narrative assessment data for each faculty member. Conclusions: We identified faculty development needs related to EPA assessments and designed dashboard elements to meet them. The resulting dashboard was used for faculty development sessions. This work will inform the development of CBME assessment dashboards for faculty.

scholarly journals Developing a dashboard to meet the needs of residents in a competency-based training program: A design-based research project

2020 ◽  
Author(s):  
Robert Carey ◽  
Grayson Wilson ◽  
Venkat Bandi ◽  
Debajyoti Mondal ◽  
Lynsey Martin ◽  
...  
Keyword(s):  
Emergency Medicine ◽  
Focus Groups ◽  
Residency Program ◽  
Research Process ◽  
Theory Approach ◽  
Assessment Data ◽  
Design Based Research ◽  
Professional Activities ◽  
Competency Based ◽  
The University

Background: Canadian specialty programs are implementing Competence By Design, a competency-based medical education (CBME) program which requires frequent assessments of entrustable professional activities. To be used for learning, the large amount of assessment data needs to be interpreted by residents, but little work has been done to determine how visualizing and interacting with this data can be supported. Within the University of Saskatchewan emergency medicine residency program, we sought to determine how our residents’ CBME assessment data should be presented to support their learning and to develop a dashboard that meets our residents’ needs. Methods: We utilized a design-based research process to identify and address resident needs surrounding the presentation of their assessment data. Data was collected within the emergency medicine residency program at the University of Saskatchewan via four resident focus groups held over 10 months. Focus group discussions were analyzed using a grounded theory approach to identify resident needs. This guided the development of a dashboard which contained elements (data, analytics, and visualizations) that support their interpretation of the data. The identified needs are described using quotes from the focus groups as well as visualizations of the dashboard elements. Results: Resident needs were classified under three themes: (1) Provide guidance through the assessment program, (2) Present workplace-based assessment data, and (3) Present other assessment data. Seventeen dashboard elements were designed to address these needs. Conclusions: Our design-based research process identified resident needs and developed dashboard elements to meet them. This work will inform the creation and evolution of CBME assessment dashboards designed to support resident learning.

scholarly journals Developing a dashboard to meet Competence Committee needs: a design-based research project

2020 ◽  
Author(s):  
Brent Thoma ◽  
Venkat Bandi ◽  
Robert Carey ◽  
Debajyoti Mondal ◽  
Robert Woods ◽  
...  
Keyword(s):  
Decision Making ◽  
Research Process ◽  
Assessment Data ◽  
Design Based Research ◽  
Educational Environments ◽  
Integral Role ◽  
Learner Assessment ◽  
Workplace Based Assessment ◽  
Competency Based ◽  
The University

Background: Competency-based programs are being adopted in medical education around the world. Competence Committees must visualize learner assessment data effectively to support their decision-making. Dashboards play an integral role in decision support systems in other fields. Design-based research allows the simultaneous development and study of educational environments. Methods: We utilized a design-based research process within the emergency medicine residency program at the University of Saskatchewan to identify the data, analytics, and visualizations needed by its Competence Committee, and developed a dashboard incorporating these elements. Narrative data were collected from two focus groups, five interviews, and the observation of two Competence Committee meetings. Data were qualitatively analyzed to develop a thematic framework outlining the needs of the Competence Committee and to inform the development of the dashboard. Results: The qualitative analysis identified four Competence Committee needs (Explore Workplace-Based Assessment Data, Explore Other Assessment Data, Understand the Data in Context, and Ensure the Security of the Data). These needs were described with narratives and represented through visualizations of the dashboard elements. Conclusions: This work addresses the practical challenges of supporting data-driven decision making by Competence Committees and will inform the development of dashboards for programs, institutions, and learner management systems.

scholarly journals Feasibility of an Entrustable Professional Activity for Pathology Resident Frozen Section Training

2021 ◽  
Vol 8 ◽  
pp. 237428952110417
Author(s):  
Bronwyn H. Bryant
Keyword(s):  
Medical Education ◽  
Frozen Section ◽  
Resident Training ◽  
Entrustable Professional Activities ◽  
Professional Activity ◽  
Clinical Competency ◽  
Professional Activities ◽  
Competency Based ◽  
The University ◽  
Clinical Competency Committee

Entrustable professional activities are an intuitive form of workplace-based assessment that can support competency-based medical education. Many entrustable professional activities have been written and published, but few studies describe the feasibility or implementation of entrustable professional activities in graduate medical education. The frozen section entrustable professional activit was introduced into the pathology residency training at the University of Vermont for postgraduate year 1 at the start of their training in frozen section. The feasibility of the entrustable professional activit was evaluated based on 3 criteria: (a) utilization, (b) support of frozen section training, and (c) generating data to support entrustment decision about residents’ readiness to take call. The entrustable professional activit was well utilized and satisfactory to residents, faculty, pathologists’ assistants, and Clinical Competency Committee members. Most members of the Clinical Competency Committee agreed they had sufficient data and noted higher confidence in assessing resident readiness to take call with the addition of entrustable professional activit to the residents’ assessment portfolio. Residents did not endorse it helped them prepare for call; however, the interruption to frozen section training due to the COVID-19 pandemic was a significant contributing factor. The frozen section entrustable professional activit is a feasible addition to pathology resident training based on utilization, support of training, and generation of data to support entrustment decisions for graduated responsibilities. The implementation and integration of the entrustable professional activit into pathology training at our institution is described with discussion of adjustments for future use.

scholarly journals P122: Narrative assessment of emergency medicine learners: What should we keep as we move to competency-based assessment?

CJEM ◽  
2019 ◽  
Vol 21 (S1) ◽  
pp. S108
Author(s):  
S. Segeren ◽  
L. Shepherd ◽  
R. Pack
Keyword(s):  
Emergency Medicine ◽  
Contextual Information ◽  
Management Plan ◽  
Professional Activities ◽  
Narrative Assessment ◽  
Canmeds Role ◽  
Competency Based ◽  
Qualitative Methodologies ◽  
Iterative Coding ◽  
Narrative Comments

Introduction: For many years, Emergency Medicine (EM) educators have used narrative comments to assess their learners on each shift, either in isolation or combined with some type of Likert scale ranking. Competency based medical education (CBME), soon to be fully implemented throughout Canadian EM educational programs, encourages this type of frequent low-stakes narrative assessment. It is important to understand what information is currently garnered from existing narrative assessments in order to successfully and smoothly transition to the CBME system. The purpose of this study was to explore how one Canadian undergraduate EM program's narrative assessment comments mapped to two competency frameworks: one traditional CanMEDS-based and one competency-based, built on entrustable professional activities (EPAs). Methods: A qualitative and quantitative content analysis of 1925 retrospective, narrative assessments was conducted for the 2015/2016 and 2016/2017 academic years. The unprompted comments were mapped to the Royal College CanMEDS framework and the Association of Faculties of Medicine of Canada EPA Framework. Using an iterative coding process as per accepted qualitative methodologies, additional codes were generated to classify comments and identify themes that were not captured by either framework. Results: 93% and 85% of the unprompted narrative assessments contained comments that mapped to at least one CanMEDS role or EPA competency, respectively. The most common CanMEDS role commented upon was Medical Expert (86%), followed by Communicator, Collaborator and Scholar (all at 23%). The most common EPA competency mentioned related to history and physical findings (62%) followed by management plan (33%), and differential diagnosis (33%). However, 75% of narrative comments contained within the assessments, included ideas that did not fall into either framework but were repeated with frequency to suggest importance. The experiential characteristics of working with a learner were commented upon by 22% of preceptors. Other unmapped themes included contextual information, generalities and platitudes, and directed feedback for next steps to improve. Conclusion: While much of the currently captured data can be mapped to established frameworks, important information for both learner and assessor may be lost by limiting comments to the competencies described within a particular framework, suggesting caution when transitioning to a CBME assessment program.

scholarly journals An analysis of core EPAs reveals a gap between curricular expectations and medical school graduates’ self-perceived level of competence

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Adrian Marty ◽  
Sonia Frick ◽  
Heidi Bruderer Enzler ◽  
Sabine Zundel
Keyword(s):  
Medical Education ◽  
Online Survey ◽  
Management Plan ◽  
Practical Skills ◽  
Professional Activities ◽  
Emergency Situations ◽  
New Curriculum ◽  
Competency Based ◽  
University Of Zurich ◽  
The University

Abstract Background Entrustable Professional Activities (EPAs) are being implemented worldwide as a means to promote competency-based medical education. In Switzerland, the new EPA-based curriculum for undergraduate medical education will be implemented in 2021. The aim of our study was to analyze the perceived, self-reported competence of graduates in 2019. The data represent a pre-implementation baseline and will provide guidance for curriculum developers. Methods Two hundred eighty-one graduates of the Master of Human Medicine program of the University of Zurich who had passed the Federal Licensing Exam in September 2019 were invited to complete an online survey. They were asked to rate their needed level of supervision (“observe only”, “direct, proactive supervision”, “indirect, reactive supervision”) for 46 selected EPAs. We compared the perceived competence with the expected competence of the new curriculum. Results The response rate was 54%. The need for supervision expressed by graduates varied considerably by EPA. The proportion of graduates rating themselves at expected level was high for “history taking”, “physical examination” “and documentation”; medium for “prioritizing differential diagnoses”, “interpreting results” and “developing and communicating a management plan”; low for “practical skills”; and very low for EPAs related to “urgent and emergency care”. Conclusions Currently, there are significant gaps between the expectations of curriculum developers and the perceived competences of students. This is most obvious for practical skills and emergency situations. The new curriculum will either need to fill this gap or expectations might need to be revised.

UPAYA MENINGKATKAN AKTIVITAS DAN HASIL BELAJAR MAHASISWA MELALUI PENERAPAN MODEL PEMBELAJARAN INKUIRI PADA MATA KULIAH KONSEP DASAR IPA 2 DI PROGRAM STUDI PENDIDIKAN GURU SEKOLAH DASAR TAHUN AKADEMIK 2013/2014

2014 ◽  
Vol 3 (1) ◽  
pp. 23
Author(s):  
Mahmud Alpusari
Keyword(s):  
Initial Data ◽  
Learning Outcomes ◽  
Inquiry Learning ◽  
Learning Activities ◽  
Simple Experiment ◽  
Competency Based ◽  
Basic Concepts ◽  
Using Data ◽  
The University ◽  
Daily Test

In line with the competency-based curriculum at the University of Riau, the effort to improvelearning basic concepts of science 2 courses puts emphasis on understanding the concept ofmatter, which is based on students' learning activities through scientific inquiry.Implementation of action research consists of two cycles in PGSD JIP University of Riau onthe odd semester of 2013/2014 with 55 third semester students. Based on the research results,lecturing process by applying the model of inquiry learning, students’ activity increased inwhich in the first cycle all activities are good category except activity I and II are faircategory. Meanwhile students’ activity in first and fourth in cycle II is good category, andvery good category in second, third, fifth, and sixth activity. Temporarily student’s learningoutcomes increased from pre-tests with an average65.45 into 77,0 in daily test I and 77.45onthe daily test II. Improvement from initial data to the first cycle was 11.55, while the datafrom the beginning to the second cycle increased 12 points. In general the improvement ofstudents’learning is possible because the learning model used is inquiry learning so thatlearning becomes active which centered into students by presenting a problem, then studentsare asked to carry out a simple experiment using equipment and tools, using data, arrangingreports, communicating the results of observations based on concepts and learned principles.Keywords: Inquiry, students’ activity, learning outcomes.

scholarly journals Workplace-based assessments of entrustable professional activities in a psychiatry core clerkship: an observational study

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Severin Pinilla ◽  
Alexandra Kyrou ◽  
Stefan Klöppel ◽  
Werner Strik ◽  
Christoph Nissen ◽  
...  
Keyword(s):  
Clinical Encounter ◽  
Entrustable Professional Activities ◽  
Written Feedback ◽  
Professional Activities ◽  
Quality Feedback ◽  
Observational Prospective Cohort Study ◽  
Competency Based ◽  
Patient’S History ◽  
Indirect Supervision ◽  
Clerkship Rotation

Abstract Background Entrustable professional activities (EPAs) in competency-based, undergraduate medical education (UME) have led to new formative workplace-based assessments (WBA) using entrustment-supervision scales in clerkships. We conducted an observational, prospective cohort study to explore the usefulness of a WBA designed to assess core EPAs in a psychiatry clerkship. Methods We analyzed changes in self-entrustment ratings of students and the supervisors’ ratings per EPA. Timing and frequencies of learner-initiated WBAs based on a prospective entrustment-supervision scale and resultant narrative feedback were analyzed quantitatively and qualitatively. Predictors for indirect supervision levels were explored via regression analysis, and narrative feedback was coded using thematic content analysis. Students evaluated the WBA after each clerkship rotation. Results EPA 1 (“Take a patient’s history”), EPA 2 (“Assess physical & mental status”) and EPA 8 (“Document & present a clinical encounter”) were most frequently used for learner-initiated WBAs throughout the clerkship rotations in a sample of 83 students. Clinical residents signed off on the majority of the WBAs (71%). EPAs 1, 2, and 8 showed the largest increases in self-entrustment and received most of the indirect supervision level ratings. We found a moderate, positive correlation between self-entrusted supervision levels at the end of the clerkship and the number of documented entrustment-supervision ratings per EPA (p < 0.0001). The number of entrustment ratings explained 6.5% of the variance in the supervisors’ ratings for EPA 1. Narrative feedback was documented for 79% (n = 214) of the WBAs. Most narratives addressed the Medical Expert role (77%, n = 208) and used reinforcement (59%, n = 161) as a feedback strategy. Students perceived the feedback as beneficial. Conclusions Using formative WBAs with an entrustment-supervision scale and prompts for written feedback facilitated targeted, high-quality feedback and effectively supported students’ development toward self-entrusted, indirect supervision levels.

scholarly journals Preparing for CBME: How often are faculty observing residents?

2020 ◽  
Author(s):  
Sheenagh J K George ◽  
Sarah Manos ◽  
Kenny K Wong
Keyword(s):  
Direct Observation ◽  
Assessment Tools ◽  
Inpatient Setting ◽  
Institutional Research ◽  
Professional Activities ◽  
Inpatient Settings ◽  
Competency Based ◽  
Resident Assessment ◽  
Outpatient Settings ◽  
Common Barriers

Abstract Background The Royal College of Physicians and Surgeons of Canada officially launched ‘Competence by Design’ in July 2017, moving from time-based to outcomes-based training. Transitioning to competency-based medical education (CBME) necessitates change in resident assessment. A greater frequency of resident observation will likely be required to adequately assess whether entrustable professional activities have been achieved. Purpose Characterize faculty and resident experiences of direct observation in a single paediatric residency program, pre-CBME implementation. Qualitatively describe participants’ perceived barriers and incentives to participating in direct observation. Methods Surveys were sent to paediatric residents and faculty asking for demographics, the frequency of resident observation during an average 4-week rotation, perceived ideal frequency of observation, and factors influencing observation frequency. Descriptive data were analyzed. Institutional research ethics board approval was received. Results The response rate was 54% (34/68 faculty and 16/25 residents). When asked the MAXIMUM frequency FACULTY observed a resident take a history, perform a physical examination, or deliver a plan, the median faculty reply was 1, 2, and 3, for outpatient settings and 0, 1, and 2, for inpatient settings. The median RESIDENT reply was 2, 4, and 10 for outpatient settings and 1, 2, and 20 for inpatient settings. When asked the MINIMUM frequency for each domain, the median FACULTY and RESIDENT reply was 0, except for delivering a plan in the inpatient setting. Faculty reported observing seniors delivering the plan more frequently than junior residents. Faculty and resident median replies for how frequently residents should be observed for each domain were the same, three to four, three to four, and five to six times. Four per cent of faculty reported regularly scheduling observations, and 77% of residents regularly ask to be observed. The most common barriers to observation were too many patients to see and both faculty and residents were seeing patients at the same time. Most faculty and resident responders felt that observation frequency could be improved if scheduled at the start of the rotation; faculty were provided a better tool for assessment; and if residents asked to be observed. Conclusions This study provides baseline data on how infrequent faculty observation is occurring and at a frequency lower than what faculty and residents feel is necessary. The time needed for observation competes with clinical service demands, but better scheduling strategies and assessment tools may help.

IMPACT ANALYSIS OF HEC- BASED TRAINING PROGRAMS ON THE PERFORMANCE OF THE UNIVERSITY TEACHERS IN PAKISTAN

2012 ◽  
Vol 01 (06) ◽  
pp. 35-40
Author(s):  
Khalid Hussain Shaikh ◽  
Ikhtiar Ahmed Ghumro ◽  
Asif Ali Shah ◽  
Faiz M. Shaikh ◽  
Tahira Afridi
Keyword(s):  
Professional Development ◽  
Personal Development ◽  
Impact Analysis ◽  
Training Programs ◽  
Effective Teacher ◽  
University Teachers ◽  
Instructional Activities ◽  
Teacher Aides ◽  
The Government ◽  
The University

The current research investigates the HEC based training for the University teachers in Pakistan and its impact on the performance on University teachers. The Higher Education Commission (HEC) in Pakistan seeks to improve the quality of teaching by University teachers. The Commission has initiated different forms of training according to the areas of expertise in order to improve skills and impact on the performance of University teachers. HEC based training plays a crucial role in the personal development among the University Teachers in Pakistan. Survey was conducted from 200 University teachers who have recently got training from HEC skill development or professional development training from HEC from all provinces. Structural questionnaire was design for reliability and accuracy the data. Analysis and evaluation was done by using GENSTAT statistical software. Major findings of the study showed that training should be provided according to discipline and more interactive training should be design for the University teachers. It was revealed that HEC based training not only equipped with knowledge but also improving the confidence level of the University teacher. Moreover due to the government policies, rules and regulations, such as introduction of the Tenure Track System, the and hiring the foreign faculty in various all Public sector universities it also has impact on the performance of students in job market. It revealed that teacher training was beneficial for professional development as well as for teaching performance. It also suggested that improved knowledge, skills and attitudes was necessary for the teacher aides to support the teaching program and facilitate learning and communication. It was further revealed that effective teacher aides required competencies in broad areas of human relations, instructional activities, non-instructional activities, and basic skills. The study concluded that basic and advanced level training is necessary for future training programs in Pakistan and 190 respondents responded to the questionnaires, by producing 95.0% response rate. Among which 70 % were male respondents and 30% were female respondents

The use of teaching methodology in the development of online courses on personal data protection

2021 ◽  
pp. 227-239
Author(s):  
Светлана Александровна Корягина
Keyword(s):  
Educational Institution ◽  
Cognitive Activity ◽  
Personal Data ◽  
Educational Process ◽  
Quality Level ◽  
Teaching Methodology ◽  
Pedagogical Practice ◽  
Professional Activities ◽  
Personal Data Protection ◽  
The University

Основные проблемы современного образования нельзя решать без его развития. Ученые и практики обращают внимание на устаревшие формы и технологии управления работой со студентами. В современных условиях решать проблемы образования традиционными системами управления и методами становится все сложнее, поэтому возникает необходимость в адекватном построении методической оснащенности образовательного процесса в учебном заведении, что актуализирует необходимость обеспечения системы образования методическими  разработками, дидактическими материалами, которые отвечают современным требованиям педагогической практики. Организационными формами методической работы вуза являются: научно-методическая и педагогический совет; аттестация преподавателей; предметные кафедры, творческие лаборатории преподавателей; школа молодого преподавателя; психолого-педагогические и методические семинары; методические школы преподавателей-новаторов; подготовка методических пособий, публикаций в периодических и профессиональных изданиях. Большинство вышеупомянутых видов методического обеспечения являются традиционными, но все они дают возможность внедрять инновации. Однако обращается внимание только на формы и методы организации образовательного процесса, а не на психолого-педагогическое сопровождение познавательной деятельности преподавателей и методическое обеспечение управленческой деятельности как источника качества и результативности их профессиональной деятельности. Реалии выдвигают новые требования к методическому обеспечению образовательного процесса в вузt, требуя менять цели, функции и содержание, превращая уровень его качества в научно-методический. Суть и назначение научно методического обеспечения заключается в создании условий для формирования и личностно-профессионального становления преподавателей, развития их педагогических способностей, творческого потенциала. The main problems of modern education cannot be solved without its development. Scientists and practitioners pay attention to outdated forms and technologies of managing work with students. In modern conditions, it is becoming increasingly difficult to solve the problems of education with traditional management systems and methods, therefore, there is a need for an adequate construction of methodological equipment of the educational process in an educational institution, which actualizes the need to provide the education system with methodological developments, didactic materials that meet modern requirements of pedagogical practice. Organizational forms of methodological work of the university are: scientific and methodological and pedagogical council; certification of teachers; subject departments, creative laboratories of teachers; school of a young teacher; psychological, pedagogical and methodological seminars; methodological schools of innovative teachers; preparation of methodological manuals, publications in periodicals and professional publications. Most of the above-mentioned types of methodological support are traditional, but all of them make it possible to introduce innovations. However, attention is drawn only to the forms and methods of organizing the educational process, and not to the psychological and pedagogical support of the cognitive activity of teachers and methodological support of managerial activities as a source of quality and effectiveness of their professional activities. The realities put forward new requirements for the methodological support of the educational process at the university, requiring changing the goals, functions and content, turning its quality level into a scientific and methodological one. The essence and purpose of scientific and methodological support is to create conditions for the formation and personal and professional development of teachers, the development of their pedagogical abilities, creative potential.

Sign in / Sign up

Export Citation Format

Share Document