Building reliable and generalizable clerkship competency assessments: Impact of ‘hawk-dove’ correction

2021 ◽  
pp. 1-7
Author(s):  
Sally A. Santen ◽  
Michael Ryan ◽  
Marieka A. Helou ◽  
Alicia Richards ◽  
Robert A. Perera ◽  
...  
Keyword(s):  
Competency Assessments

scholarly journals Children's palliative care: examination of a nursing rotation programme

2021 ◽  
Vol 27 (1) ◽  
pp. 20-29
Author(s):  
Lisa Whiting ◽  
Mark Whiting ◽  
Julia Petty ◽  
Michele O'Grady
Keyword(s):  
Palliative Care ◽  
Learning Experience ◽  
Knowledge Exchange ◽  
Comfort Zone ◽  
New Knowledge ◽  
Support Mechanisms ◽  
Hospital Wards ◽  
Competency Assessments ◽  
Insight Into ◽  
Future Plans

Background: An 8-month rotation programme was implemented for five nurses employed in two kinds of children's palliative care environments: hospital wards and hospices. This study reports the views of the nurses completing the rotation. The research drew on appreciative inquiry and involved a pre- and post-rotation interview and questionnaire. Thematic analysis of the interviews revealed seven themes: adjusting to the rotation programme; support mechanisms; being safe; new knowledge and skills; knowledge exchange; misconceptions; future plans. These were supported by the questionnaire findings. Although the nurses identified some frustration at having to undertake competency assessments relating to previously acquired skills, as well as being out of their ‘comfort zone’, all the participants highly recommended the programme. They commented very positively on the support they received and the overall learning experience as well as the new insight into different aspects of care. In addition, they were able to share their newfound knowledge and expertise with others.

Current Models for Inpatient and Emergency Room Ophthalmology Consultation in U.S. Residency Programs

2020 ◽  
Vol 12 (02) ◽  
pp. e171-e174
Author(s):  
Donna H. Kim ◽  
Dongseok Choi ◽  
Thomas S. Hwang
Keyword(s):  
Emergency Department ◽  
Medical Education ◽  
Emergency Room ◽  
Program Directors ◽  
Cross Sectional Survey ◽  
Cross Sectional ◽  
Direct Supervision ◽  
Residency Programs ◽  
Residency Program Directors ◽  
Competency Assessments

Abstract Objective This article examines models of patient care and supervision for hospital-based ophthalmology consultation in teaching institutions. Design This is a cross-sectional survey. Methods An anonymous survey was distributed to residency program directors at 119 Accreditation Council for Graduated Medical Education accredited U.S. ophthalmology programs in the spring of 2018. Survey questions covered consult volume, rotational schedules of staffing providers, methods of supervision (direct vs. indirect), and utilization of consult-dedicated didactics and resident competency assessments. Results Of the 119 program directors, 48 (41%) completed the survey. Programs most frequently reported receiving 4 to 6 consults per day from the emergency department (27, 55.1%) and 4 to 6 consults per day from inpatient services (26, 53.1%). Forty-seven percent of programs reported that postgraduate year one (PGY-1) or PGY-2 residents on a dedicated consult rotation initially evaluate patients. Supervising faculty backgrounds included neuro-ophthalmology, cornea, comprehensive, or a designated chief of service. Staffing responsibility is typically shared by multiple faculty on a daily or weekly rotation. Direct supervision was provided for fewer of emergency room consults (1–30%) than for inpatient consults (71–99%). The majority of programs reported no dedicated didactics for consultation activities (27, 55.1%) or formal assessment for proficiency (33, 67.4%) prior to the initiation of call-related activities without direct supervision. Billing submission for consults was inconsistent and many consults may go financially uncompensated (18, 36.7%). Conclusion The majority of hospital-based ophthalmic consultation at academic centers is provided by a rotating pool of physicians supervising a lower level resident. Few programs validate increased levels of graduated independence using specific assessments.

scholarly journals Advancing Medication Reconciliation in an Outpatient Internal Medicine Clinic through a Pharmacist-Led Educational Initiative

2010 ◽  
Vol 1 (1) ◽  
pp. 11
Author(s):  
Sarah M. Westberg ◽  
Kathrine Beeksma
Keyword(s):  
Internal Medicine ◽  
Nursing Staff ◽  
Medication Reconciliation ◽  
Future Research ◽  
Original Research ◽  
Educational Initiative ◽  
Medicine Clinic ◽  
Internal Medicine Clinic ◽  
The Impact ◽  
Competency Assessments

Objectives: To develop and deliver an effective pharmacist-led educational initiative to clinic staff to advance medication reconciliation in the electronic medical record of an outpatient internal medicine clinic. Methods: An educational initiative designed to improve the ability of nursing staff in medication reconciliation was launched in the outpatient internal medicine clinic of a regional healthcare system. The education was provided by the pharmacist to clinic nursing staff, including registered nurses, licensed practical nurses, and certified medical assistants. The impact of this training was measured through pre-initiation and post-implementation surveys, competency assessments and an audit. Results: The educational initiative was successfully designed and delivered to clinic nursing staff. Assessment of the initiative found that all nursing staff completing competency assessments successfully passed. Pre-initiation- and post-implementation- survey responses on the self-assessed ability to gather and document accurate medication lists did not show significant changes. Informal observations in the clinic indicated that this initiative changed the culture of the clinic, creating increased awareness of the importance of accurate medications and increased emphasis on medication reconciliation. Conclusions: The expertise of pharmacists can be utilized to educate nursing staff on the skills and abilities necessary to gather and document accurate medication lists. This study did not find measurable changes in the accuracy of medication lists in this clinic. Future research is needed to determine the best methods to train health professionals in medication reconciliation to ensure accurate medication lists in the outpatient setting. Type: Original Research

Defensibility of Competency Assessments of Practising Family Physicians

1997 ◽  
pp. 259-262
Author(s):  
J. P. W. Cunnington ◽  
E. Hanna ◽  
J. Turnbull ◽  
T. Kaigas ◽  
G. R. Norman
Keyword(s):  
Family Physicians ◽  
Competency Assessments

scholarly journals Variability in multi-rater competency assessments

1999 ◽  
Vol 25 (1) ◽  
Author(s):  
D. Theron ◽  
G. Roodt
Keyword(s):  
Organisational Structure ◽  
Competency Evaluations ◽  
Competency Assessments

The purpose of this study was to determine if significant differences exist between the multi-rater competency evaluations of employees operating within a flat organisational structure. Sixty-eight marketing employees were each evaluated by a number of raters including themselves, their managers, customers and peers. A competency questionnaire was developed by using the input of the employees who took part in the appraisal. Using paired t- tests significant differences between the various groups of raters were found. These findings and the implications thereof are discussed. Opsomming Die doel van hierdie studie was om te bepaal of daar beduidende verskille bestaan tussen die multi-beoordelaar bevoegdheidsevaluerings van werknemers wat binne 'n plat organisasiestruktuur funksioneer. Agt-en-sestig bemarkingswerknemers is elk beoordeel deur 'n aantal beoordelaars wat die werknemers self, hul bestuurders, kliente en kollegas ingesluit het. 'n Bevoegdheidsvraelys is ontwikkel deur gebruik te maak van die insette van die werknemers wat deel geneem het aan die evaluering. Deur die gebruik van gepaarde t-toetse is gevind dat daar beduidende verskille bestaan tussen sommige van die groepe beoordelaars. Hierdie bevindinge en die implikasies daarvan word bespreek.

The Utility of the MCMI-III in Parental Competency Assessments

2019 ◽  
Vol 19 (3) ◽  
pp. 211-229
Author(s):  
Christina V. Bueno Castellano ◽  
David R. Brandwein ◽  
Richard P. Conti ◽  
Aaron A. Gubi ◽  
Donald R. Marks ◽  
...  
Keyword(s):  
Competency Assessments

scholarly journals Implementation and Evaluation of the Z-Score System for Normalizing Residency Evaluations

2018 ◽  
Vol 128 (1) ◽  
pp. 144-158 ◽  
Author(s):  
Jonathan P. Wanderer ◽  
Getulio R. de Oliveira Filho ◽  
Brian S. Rothman ◽  
Warren S. Sandberg ◽  
Matthew D. McEvoy
Keyword(s):  
Variance Components ◽  
Training Programs ◽  
Clinical Performance ◽  
Core Competency ◽  
Principal Component ◽  
Z Score ◽  
High Stakes ◽  
Score System ◽  
Residency Programs ◽  
Competency Assessments

Abstract Background Assessment of clinical competence is essential for residency programs and should be guided by valid, reliable measurements. We implemented Baker’s Z-score system, which produces measures of traditional core competency assessments and clinical performance summative scores. Our goal was to validate use of summative scores and estimate the number of evaluations needed for reliable measures. Methods We performed generalizability studies to estimate the variance components of raw and Z-transformed absolute and peer-relative scores and decision studies to estimate the evaluations needed to produce at least 90% reliable measures for classification and for high-stakes decisions. A subset of evaluations was selected representing residents who were evaluated frequently by faculty who provided the majority of evaluations. Variance components were estimated using ANOVA. Results Principal component extraction from 8,754 complete evaluations demonstrated that a single factor explained 91 and 85% of variance for absolute and peer-relative scores, respectively. In total, 1,200 evaluations were selected for generalizability and decision studies. The major variance component for all scores was resident interaction with measurement occasions. Variance due to the resident component was strongest with raw scores, where 30 evaluation occasions produced 90% reliable measurements with absolute scores and 58 for peer-relative scores. For Z-transformed scores, 57 evaluation occasions produced 90% reliable measurements with absolute scores and 55 for peer-relative scores. The results were similar for high-stakes decisions. Conclusions The Baker system produced moderately reliable measures at our institution, suggesting that it may be generalizable to other training programs. Raw absolute scores required few assessment occasions to achieve 90% reliable measurements.

scholarly journals Framework and Prototype Development of Mathsci Instruments for Measuring 21st Century Skills in Islamic Context

2021 ◽  
Vol 8 (1) ◽  
pp. 96-107
Author(s):  
Zulfiani Zulfiani ◽  
Iwan Permana Suwarna ◽  
Abdul Muin
Keyword(s):  
21St Century ◽  
21St Century Skills ◽  
Assessment Instrument ◽  
First Century ◽  
Prototype Development ◽  
Competence Evaluation ◽  
Study Results ◽  
Paper And Pencil ◽  
And Mathematics ◽  
Competency Assessments

AbstractThe development of competency assessments that measure 21st century competencies is critical for improving the quality of education. The purpose of this research is to describe a framework and prototype for assessing Math and Science or Math-Sci competency skills in the twenty-first century. The method of development research employed in this study is based on the Akker framework, which entails preliminary research, prototyping, design, and development in a paper and pencil test format. The equipment of the investigation included observation sheets, questionnaires, and tests. This study results in a conceptual framework for the instrument and a verified prototype of the Math-Sci competence evaluation. The Math-Sci competence evaluation is conceptualized around thematic, interdisciplinary questions that integrate three (three) subjects, namely Science (Biology-Physics) and Mathematics, in an Islamic context. Math-Sci, using the ladder analogy (monodisciplinary, interdisciplinary 1, and interdisciplinary 2) to rank students' competency, relates to the thought process of Bloom's taxonomy, the context, and the complexity of the topic. The instrument created was deemed valid and practicable based on the results of expert validation. The development of the Math-Sci competence assessment instrument was the first step toward strengthening assessment for learning and assisting in the improvement of learning through the presentation of integrated contextual problems. AbstrakPengembangan asesmen kompetensi yang membekalkan kecakapan abad 21 sangat urgent untuk memperbaiki proses pembelajaran di sekolah. Tulisan ini bertujuan mendeskripsikan framework dan prototype asesmen kompetensi Math-Sci untuk mengukur keterampilan abad 21. Metode penelitian pengembangan yang digunakan dalam penelitian ini mengacu pada framework Akker yang meliputi penelitian pendahuluan, prototipe yang meliputi desain, dan pengembangan dalam format paper and pencil test. Instrumen penelitian berupa lembar observasi, angket, dan tes. Hasil penelitian diperoleh kerangka konseptual instrumen dan prototipe asesmen kompetensi Math-Sci yang telah divalidasi melalui expert judgment. Kerangka konseptual asesmen kompetensi Math-Sci berupa soal-soal tematik, interdisipliner yang memadukan 3 (tiga) disiplin ilmu yakni IPA (Biologi-Fisika) dan Matematika pada konteks keIslaman. Math-Sci dengan analogi titian anak tangga (monodisiplin, interdisiplin 1, dan  interdisiplin 2) yang mengukur kompetensi peserta didik secara hierarki mengacu pada proses berpikir taksonomi Bloom, konteks dan kompleksitas masalah. Hasil validasi pakar diperoleh instrumen soal yang dikembangkan valid dan layak. Pengembangan instrumen asesmen kompetensi Math-Sci merupakan langkah awal upaya menguatkan asesmen for learning, membantu memperbaiki pembelajaran, bersifat inovatif dengan menghadirkan permasalahan kontekstual integratif.  How to Cite: Zulfiani, Suwarna, I.P., Suwarna, Muin, A. (2021). Framework and Prototype Development of Mathsci Instruments for Measuring 21st Century Skills in Islamic Context. TARBIYA: Journal of Education in Muslim Society, 8(1), 96-107. doi:10.15408/tjems.v8i1.22120.

Competency Assessments

2000 ◽  
Vol 30 (7/8) ◽  
pp. 339-340 ◽  
Author(s):  
Sharon LaDuke
Keyword(s):  
Competency Assessments
Sign in / Sign up

Export Citation Format

Share Document