Times of Medical Education Crisis Require New Evaluation Approaches: Proof of Concept of a System-Based Program Evaluation Model

Purpose of the study: This article reviews the comparative efficacy, theoretical and practical background of three program evaluation models (Stufflebeam’s CIPP model, Kirkpatrick’s model, and outcome-based evaluation models) and their implications in educational programs. The article discusses the strengths and limitations of the three evaluation models. Methodology: Peer-reviewed and scholarly journals were searched for articles related to program evaluation models and their importance. Keywords included program evaluation’, ‘assessment’, ‘CIPP model’, ‘evaluation of educational programs, ‘outcome-based model, and ‘planning’. Articles on Stufflebeam’s CIPP model, Kirkpatrick’s model, and outcome-based evaluation models were particularlyfocused because the review aimed at analysing these three models. The strengths and inadequacies of the three models were weighed and presented. Main Findings: The three models –outcome-based evaluation model, the Kirkpatric model, and the CIPP evaluation model –discussed in this review, have some strengths and weaknesses. Among the compared models, the CIPP model seems more appropriate for its implantation in evaluating educational programs because it is broader, comprehensive, flexible, cost-effective, and feasible. Applications of this study: Like other programs and projects, evaluation of educational programs is necessary to achieve high standards, better outcomes, and meet the objectives. Evaluation is employed before designing a particular educational program or during the already designed program. This review concludes that among different evaluation models, the CIPP evaluation model is more appropriate in evaluating educational programs because it is more comprehensive, efficient, and feasible.Employment of the CIPP model for evaluating educational programs can achieve plausible results about the overall progress of the educational programs. Novelty/Originality of this study: This review highlights the importance of different program evaluation models. It concludes that the CIPP evaluation model offers an excellent mechanism to evaluate educational programs at different stages.

Download Full-text

Information Distribution in Multi-Robot Systems: Utility-Based Evaluation Model

Sensors ◽

10.3390/s20030710 ◽

2020 ◽

Vol 20 (3) ◽

pp. 710 ◽

Cited By ~ 2

Author(s):

Michał Barciś ◽

Agata Barciś ◽

Hermann Hellwagner

Keyword(s):

Evaluation Model ◽

Proof Of Concept ◽

Information Distribution ◽

Unified Framework ◽

Robot Operating System ◽

Robot Systems ◽

System 2 ◽

Aerial Vehicle ◽

Multi Uav ◽

Multi Robot

This work addresses the problem of information distribution in multi-robot systems, with an emphasis on multi-UAV (unmanned aerial vehicle) applications. We present an analytical model that helps evaluate and compare different information distribution schemes in a robotic mission. It serves as a unified framework to represent the usefulness (utility) of each message exchanged by the robots. It can be used either on its own in order to assess the information distribution efficacy or as a building block of solutions aimed at optimizing information distribution. Moreover, we present multiple examples of instantiating the model for specific missions. They illustrate various approaches to defining the utility of different information types. Finally, we introduce a proof of concept showing the applicability of the model in a robotic system by implementing it in Robot Operating System 2 (ROS 2) and performing a simple simulated mission using a network emulator. We believe the introduced model can serve as a basis for further research on generic solutions for assessing or optimizing information distribution.

Download Full-text

Use of an Institutional Template for Annual Program Evaluation and Improvement: Benefits for Program Participation and Performance

Journal of Graduate Medical Education ◽

10.4300/jgme-d-10-00002.1 ◽

2010 ◽

Vol 2 (2) ◽

pp. 160-164 ◽

Cited By ~ 6

Author(s):

Kathryn M. Andolsek ◽

Alisa Nagler ◽

John L. Weinerth

Keyword(s):

Medical Education ◽

Program Evaluation ◽

Graduate Medical Education ◽

Tracking System ◽

University Hospital ◽

Improvement Strategy ◽

Meaningful Improvement ◽

Graduate Medical ◽

Program Adherence ◽

Duke University

Abstract Purpose The Accreditation Council for Graduate Medical Education (ACGME) expects programs to engage in ongoing, meaningful improvement, facilitated in part through an annual process of program assessment and improvement. The Duke University Hospital Office of Graduate Medical Education (OGME) used an institutional practice-based learning and improvement strategy to improve the annual evaluation and improvement of its programs. Methods The OGME implemented several strategies including the development and dissemination of a template for the report, program director and coordinator development, a reminder and tracking system, incorporation of the document into internal reviews, and use of incentives to promote program adherence. Results In the first year of implementation (summer 2005), 27 programs (37%) submitted documentation of their annual program evaluation and improvement to the OGME; this increased to 100% of programs by 2009. A growing number of programs elected to use the template in lieu of written minutes. The number of citations related to required program review and improvement decreased from 12 in a single academic year to 3 over the last 5 years. Conclusion Duke University Hospital's institutional initiative to incorporate practice-based learning and improvement resulted in increased documentation, greater use of a standardized template, fewer ACGME-related citations, and enhanced consistency in preparing for ACGME site visits.

Download Full-text

Advancing Evidence-Based Practice Through Program Evaluation

10.1093/med-psych/9780190609108.001.0001 ◽

2018 ◽

Cited By ~ 3

Author(s):

Julie Q. Morrison ◽

Anna L. Harms

Keyword(s):

Program Evaluation ◽

School Improvement ◽

Change Agents ◽

Evidence Based ◽

School Based ◽

Tier System ◽

Challenges And Opportunities ◽

Education Today ◽

Evaluation Of Evidence ◽

Evaluation Approaches

The demand for results-driven accountability is pervasive in all aspects of education today. The shift in emphasis from compliance to accountability for closing achievement gaps and improving student outcomes creates challenges and opportunities for school-based professionals. School-based professionals have a significant role to play in supporting school improvement efforts and improving outcomes for students in their role as program evaluators. Despite the need for advanced knowledge and skills in program evaluation, resources dedicated to developing the competencies of school-based professionals in evaluation approaches, methods, and tools are scarce. This book addresses this need by serving as a practical resource for school-based professionals seeking to use program evaluation to enhance data-based decision making and accountability as informed by implementation research and within the context of a multi-tier system of supports framework. Throughout this book, school-based professionals are positioned as change agents guiding their school or school districts in the adoption, implementation, and evaluation of evidence-based interventions and systems of support to meet students’ academic and behavioral needs.

Download Full-text

Solving Social Work Assessment Issues: A Program Evaluation

Journal of Baccalaureate Social Work ◽

10.18084/1084-7219.9.2.136 ◽

2004 ◽

Vol 9 (2) ◽

pp. 136-158 ◽

Cited By ~ 2

Author(s):

Shawn Damon Ginther ◽

Mary Schroeder

Keyword(s):

Program Evaluation ◽

Social Work ◽

Student Outcomes ◽

Evaluation Model ◽

Program Assessment ◽

Work Program ◽

Work Assessment ◽

Social Work Programs ◽

Work Education ◽

Social Work Program

Social work programs accredited by the Council on Social Work Education (CSWE) conduct ongoing program assessment to routinely assess student outcomes relative to educational activities. To assist faculty in this effort, this article presents one undergraduate program's holistic social work program evaluation model (SWPEM). The arrangement of each of the SWPEM's eight evaluative elements is reviewed, and its administration procedures are outlined. Arguments for and barriers to program evaluation using the SWPEM are reviewed, and this faculty's experiences with the SWPEM model are discussed.

Download Full-text

Program Evaluation for Internship/Residency Programs in Academic and Research Libraries

College & Research Libraries ◽

10.5860/crl.62.4.307 ◽

2001 ◽

Vol 62 (4) ◽

pp. 307-315 ◽

Cited By ~ 6

Author(s):

Julie Brewer ◽

Mark D. Winston

Keyword(s):

Program Evaluation ◽

Academic Libraries ◽

Evaluation Model ◽

Academic Library ◽

Residency Programs ◽

Research Libraries ◽

Program Coordinators ◽

Effective Program

Academic libraries are turning increasingly to internship/residency programs to enhance their recruitment efforts. Yet, little evaluative information is available to measure the effectiveness of these programs or to justify funding for them. This article outlines the necessary components of an evaluation model for internship/residency programs based on a survey of academic library deans/directors and program coordinators. The study identifies the key evaluation factors that library administrators consider most important for measuring internship/residency programs, as well as the frequency, format, and sources of input for effective program evaluation.

Download Full-text

Building Bridges between Structural and Program Evaluation Approaches to Evaluating Policy

Journal of Economic Literature ◽

10.1257/jel.48.2.356 ◽

2010 ◽

Vol 48 (2) ◽

pp. 356-398 ◽

Cited By ~ 152

Author(s):

James J Heckman

Keyword(s):

Economic Policy ◽

Program Evaluation ◽

Policy Analysis ◽

Structural Approach ◽

Third Way ◽

Evaluation Approach ◽

Alternative Approach ◽

Evaluation Approaches

This paper compares the structural approach to economic policy analysis with the program evaluation approach. It offers a third way to do policy analysis that combines the best features of both approaches. I illustrate the value of this alternative approach by making the implicit economics of LATE explicit, thereby extending the interpretability and range of policy questions that LATE can answer. (JEL C21, E61)

Download Full-text

Two Approaches to Reading Program Evaluation: A Standardized Test and Direct Assessment

Learning Disability Quarterly ◽

10.2307/1510678 ◽

1980 ◽

Vol 3 (4) ◽

pp. 77-87 ◽

Cited By ~ 4

Author(s):

Thomas C. Lovitt ◽

Kathryn Fantasia

Keyword(s):

Program Evaluation ◽

Standardized Test ◽

Statistical Significance ◽

Learning Disabled ◽

Reading Program ◽

Reading Programs ◽

Direct Approach ◽

Ethical Concerns ◽

Direct Assessment ◽

Evaluation Approaches

Pretest and posttest measures of reading were obtained for two groups of learning disabled youngsters for two years. Measures of two types were scheduled to evaluate the program: a standardized test and direct assessment. The standardized test was the Diagnostic Reading Scales, and for the direct assessment passages from the Holt and Lippincott series were used. The two evaluation approaches were compared on: relationship to book levels passed in a year and statistical significance. The data indicated that the approaches were much alike in respect to these features. Nevertheless, the following argument was made that — based on reliability considerations, practical issues, and ethical concerns — teachers might be better off to opt for the direct approach to evaluating their reading programs.

Download Full-text

Implementation of the Mason, DeMers & Middleton consultation and program evaluation model in an Israeli school integration project

International Journal for the Advancement of Counselling ◽

10.1007/bf01407968 ◽

1994 ◽

Vol 17 (2) ◽

pp. 117-127

Author(s):

Yaacov J. Katz ◽

Avraham Ben-Yochanan

Keyword(s):

Program Evaluation ◽

Evaluation Model ◽

School Integration ◽

Integration Project

Download Full-text

EVALUASI PROGRAM PENDIDIKAN KARAKTER DI SEKOLAH DASAR KABUPATEN KULON PROGO

Jurnal Prima Edukasia ◽

10.21831/jpe.v2i2.2721 ◽

2014 ◽

Vol 2 (2) ◽

pp. 223

Author(s):

Stovika Eva Darmayanti ◽

Udik Budi Wibowo

Keyword(s):

Elementary School ◽

Program Evaluation ◽

Elementary Schools ◽

Character Education ◽

Formative Evaluation ◽

Evaluation Model ◽

Monitoring And Evaluation ◽

Facility Management ◽

Research Subjects ◽

The Government

Penelitian ini bertujuan untuk: (1) mengevaluasi ketercapaian program pendidikan karakter pada tingkat sekolah dasar di Kabupaten Kulon Progo, (2) memberikan rekomendasi baik kepada guru, sekolah, maupun pemerintah untuk perbaikan program pendidikan karakter. Jenis penelitian adalah evaluasi program (evaluasi formatif) dengan pendekatan kualitatif. Subjek penelitian meliputi: (1) empat sekolah dasar di Kabupaten Kulon Progo, yaitu SDN 4 Wates, SDN 6 Bendungan, SDN Kriyan, dan SDN Selo; (2) Pengawas SD Kecamatan Kokap dan Pengawas SD Kecamatan Wates; dan (3) Di-nas Pendidikan Kulon Progo. Data dianalisis dengan menggunakan analisis data Milles & Huberman meliputi: reduksi data, penyajian data, dan verifikasi data. Kesimpulan dari penelitian ini adalah: (1) kesiapan sekolah dasar di Kabupaten Kulon Progo untuk mengimplementasikan pendidikan karakter baik, dinilai dari kurikulum yang telah terintegrasi pendidikan karakter, namun masih kurang dalam hal pengelolaan sarana prasarana pendukung dan banyak guru memerlukan lebih banyak pengetahuan dan keterampilan tentang pendidikan karakter; (2) implementasi pendidikan karakter belum tampak pada kegiatan pembelajaran; (3) dukungan dari pemerintah dalam sosialisasi atau pelatihan dirasa masih kurang oleh sekolah; (4) monitoring dan evaluasi pendidikan karakter masih terbatas pada kurikulum dan dilakukan melalui pembinaan pengawas di setiap sekolah; dan (5) kendala yang umum dihadapi sekolah adalah penilaian sikap siswa yang belum terdokumentasi, kurangnya pemahaman guru untuk mengimplementasikan pendidikan karakter, dan tidak adanya sinergi antara pendidikan di sekolah dengan pendidikan di rumah. _________________________________________________________________________________________________________________________________________________________________________________________________________________________________ A PROGRAM EVALUATION OF CHARACTER EDUCATION IN ELEMENTARY SCHOOL OF KULON PROGO REGENCY ABSTRACT This study aims to: (1) evaluate the progress of character education program in the elementary schools in Kulon Progo Regency, and (2) give recommendation to teachers, schools, and the government about the character education program’s improvement. This research used a program evaluation of Scriven’s formative evaluation model with the qualitative approach. The research subjects comprised: (1) four elementary schools in Kulon Progo, i.e. SDN 4 Wates, SDN 6 Bendungan, SDN Kriyan, and SDN Selo, (2) elementary school’s superintendents of Wates and Kokap, and (3) the education department of Kulon Progo. This research used the qualitative data analysis from Miles & Huberman, consisting of data reduction, data display, and verification. The conclusions of this study are that: (1) the readiness of school for implementing character education is good, assessed from their curriculum integrated with character education, but still not good enough in terms of facility management, and the number of teachers in need of more knowledge and skills about character education; (2) the implementation of character education does not happen yet in the classroom learning activities; (3) the sample schools consider that the support from the government is insufficient especially for character education training; (4) the monitoring and evaluation of character education are still limited to the curriculum and done by the school’s superintendents; and (5) the constraints at character education are undocumented students’s behavior assessment, lack of teacher’s understanding to implement the character education, and there is no synergy between education at school and education at home. Keywords: program evaluation, character education, elementary school

Download Full-text