scholarly journals Are Participants Good Evaluators?

2021 ◽  
Author(s):  
Jeffrey Smith ◽  
Alexander Whalley ◽  
Nathaniel Wilcox

Managers of workforce training programs are often unable to afford costly, full-fledged experimental or nonexperimental evaluations to determine their programs’ impacts. Therefore, many rely on the survey responses of program participants to gauge program impacts. Smith, Whalley, and Wilcox present the first attempt to assess such measures despite their already widespread use in program evaluations. They develop a multidisciplinary framework for addressing the issue and apply it to three case studies: the National Job Training Partnership Act Study, the U.S. National Supported Work Demonstration, and the Connecticut Jobs First Program. Each of these studies were subjected to experimental evaluations that included a survey-based participant evaluation measure. The authors apply econometric methods specifically developed to obtain estimates of program impacts among individuals in the studies and then compare these estimates with survey-based participant evaluation measures to obtain an assessment of the surveys’ efficacy. The authors also discuss how their findings fit into the broader literatures in economics, psychology, and survey research.

2021 ◽  
Vol 54 (1) ◽  
pp. 1-38
Author(s):  
Víctor Adrián Sosa Hernández ◽  
Raúl Monroy ◽  
Miguel Angel Medina-Pérez ◽  
Octavio Loyola-González ◽  
Francisco Herrera

Experts from different domains have resorted to machine learning techniques to produce explainable models that support decision-making. Among existing techniques, decision trees have been useful in many application domains for classification. Decision trees can make decisions in a language that is closer to that of the experts. Many researchers have attempted to create better decision tree models by improving the components of the induction algorithm. One of the main components that have been studied and improved is the evaluation measure for candidate splits. In this article, we introduce a tutorial that explains decision tree induction. Then, we present an experimental framework to assess the performance of 21 evaluation measures that produce different C4.5 variants considering 110 databases, two performance measures, and 10× 10-fold cross-validation. Furthermore, we compare and rank the evaluation measures by using a Bayesian statistical analysis. From our experimental results, we present the first two performance rankings in the literature of C4.5 variants. Moreover, we organize the evaluation measures into two groups according to their performance. Finally, we introduce meta-models that automatically determine the group of evaluation measures to produce a C4.5 variant for a new database and some further opportunities for decision tree models.


2016 ◽  
Vol 5 (5) ◽  
pp. 10 ◽  
Author(s):  
Saif Khairat ◽  
Ryan Sandefer ◽  
David Marc ◽  
Lee Pyles

Objective: The purpose of this paper is to review the current state of health information technology (HIT) training programs and identify limitations in workforce expectations and student/trainee level of preparedness. A framework is proposed to build a more effective training program, differentiate HIT and health informatics, and emphasize the critical role of interprofessional collaboration for informatics-related curriculum. We define interprofessionalism as the multi-sector collaborations among academia, industry (Health Care Organizations), and vendors to produce competent informaticians.Methods: Critical review of published HIT and health informatics curricular competencies was conducted, including those published by the Office of the National Coordinator (ONC) for HIT, the American Medical Informatics Association (AMIA), the International Medical Informatics Association (IMIA), and the Council on Accreditation for Health Informatics and Information Management. A review of literature related to HIT and health informatics education and training was also completed.Results: The paper presents a framework for promoting health informatics training with an interprofessional foundation. The core components of the curricular competencies include understanding the healthcare system, biomedical data, computer programming, data analytics, usability, and technology infrastructure. To effectively deliver the content, programs require collaboration between academic institutions, healthcare organizations, and industry vendors.Conclusions: HIT and health informatics-related training programs, in their current form, are not meeting industry needs. The proposed framework addresses the current limitations by providing unique pathways for content delivery by promoting interprofessional collaboration and partnerships between academia and industry.


Author(s):  
Margaret Solomon

This article is about School-Based Initial Teacher Training (SBITT) programs practiced in the USA and the UK. The article briefly discusses how US teacher-training programs began in 1839, as Normal School in New England. They then later became university based traditional teacher-training programs across the country. Then it shows how a gradual change in teacher training came into the U.S. in the 1980s with the introduction of school-based teacher training as an alternative route. Although most teachers in the U.S are still trained in colleges and universities, the paper shows that many states still pursue alternative routes to teacher credentialing and focus on school-based training The next part is a brief narration of the history of school-based teacher training in the UK, which began in the early 19th century. In the later part of 1800s, teacher training was favored at universities in the UK and more colleges were opened to facilitate training teachers at higher education institutions (HEI). In the late 1900s, there was an emergence of School-Based Initial Teacher Training (SBITT) programs developed as a result of a shortage of trained teachers. Finally, a variety of different SBITT programs became the most prominent method of initial teacher training. In 2017–2018, 53% of teachers favored a school-based teacher training program, while 47% preferred a university-based teacher training program


2016 ◽  
Vol 8 (1) ◽  
pp. 68-73 ◽  
Author(s):  
Thomas J. Caruso ◽  
Diane H. Steinberg ◽  
Nancy Piro ◽  
Kimberly Walker ◽  
Rebecca Blankenburg ◽  
...  

ABSTRACT Background Mentors influence medical trainees' experiences through career enhancement and psychosocial support, yet some trainees never receive benefits from involved mentors. Objective Our goals were to examine the effectiveness of 2 interventions aimed at increasing the number of mentors in training programs, and to assess group differences in mentor effectiveness, the relationship between trainees' satisfaction with their programs given the presence of mentors, and the relationship between the number of trainees with mentors and postgraduate year (PGY). Methods In group 1, a physician adviser funded by the graduate medical education department implemented mentorships in 6 residency programs, while group 2 involved a training program with funded physician mentoring time. The remaining 89 training programs served as controls. Chi-square tests were used to determine differences. Results Survey responses from group 1, group 2, and controls were 47 of 84 (56%), 34 of 78 (44%), and 471 of 981 (48%, P = .38), respectively. The percentages of trainees reporting a mentor in group 1, group 2, and the control group were 89%, 97%, and 79%, respectively (P = .01). There were no differences in mentor effectiveness between groups. Mentored trainees were more likely to be satisfied with their programs (P = .01) and to report that faculty supported their professional aspirations (P = .001). Across all programs, fewer first-year trainees (59%) identified a mentor compared to PGY-2 through PGY-8 trainees (84%, P < .001). Conclusions A supported mentorship program is an effective way to create an educational environment that maximizes trainees' perceptions of mentorship and satisfaction with their training programs.


1974 ◽  
Vol 18 (2) ◽  
pp. 172-175
Author(s):  
John T. Dailey ◽  
Evan W. Pickrel

This article traces the historical development of aerial hijackings internationally and domestically and psychological contributions toward their control. As the U.S. hijacking problem increased, the FAA gathered an interdisciplinary study team to derive controls. Selective boarding gate screening procedures were created which used a behavioral profile to identify high risk passengers for search. A library was established for accumulating all available information on hijackings. This information is used for developing new profiles and deriving tactics to teach airline crews for defense against future hijackers. Tactical training programs were developed, their usefulness was demonstrated operationally, and they have been distributed to all U.S. and many foreign carriers. A research program continues for upgrading defenses.


Blood ◽  
2016 ◽  
Vol 128 (22) ◽  
pp. 314-314
Author(s):  
Madeleine M Verhovsek ◽  
Chenchen Hou ◽  
Mona Azzam ◽  
Vicky R. Breakey ◽  
Richard Ward ◽  
...  

Abstract Introduction: In North America, there are a growing number of patients with hemoglobinopathies, including sickle cell disease and thalassemia. As previously identified in needs assessment surveys of Canadian hematology training program directors and fellows (Verhovsek et. al. 2014 - https://ash.confex.com/ash/2014/webprogram/Paper71742.html), hemoglobinopathy learning constitutes an integral component of the hematology residency curriculum, however there are significant differences in volume and case-mix of hemoglobinopathy exposure among training programs. Fellows and program directors expressed strong interest in online e-learning modules to address gaps in clinical and laboratory learning. As such, we developed a set of e-learning modules for hemoglobinopathies that were implemented as a pilot study. We hypothesized that the e-learning-based curriculum would enhance education and standardize curricular exposure between Canadian hematology programs. Methods: Based on the National Hemoglobinopathy Learning Objectives, a curriculum of 12 case-based modules was developed. Modules were implemented as a pilot study in four Canadian hematology training programs using a distributed practice model - two modules every two months over the 12-month academic year. Training programs were selected based on baseline hemoglobinopathy exposure in their curriculum (two programs with high baseline exposure and two programs with low baseline exposure). Fellows completed the modules independently. Learning was supplemented with bimonthly module review sessions facilitated by expert faculty. In programs where the program directors did not identify a local faculty member with content expertise, review sessions were run by web conference. Data were collected regarding the efficacy of the intervention through three main formats: in-person fellow focus groups, online fellow questionnaires, and sets of designated hemoglobinopathy-related questions at the annual National Hematology Online Practice Exam for all Canadian Hematology fellows. Results: From online graded survey responses (1 to 5, "strongly disagree" to "strongly agree"), 97% of respondents indicated the modules were relevant to their particular learning needs (Mean 4.39, SD = 0.55). All survey respondents indicated the case scenarios were realistic, and 95% felt that the e-learning software was easy to use, engaging and offered flexibility of computer-based learning. Responses in focus group feedback sessions mirrored the survey findings: modules were described as "very useful", "realistic", and of high production quality, with participants indicating they felt the issue of variability in case mix could be addressed with online cases. With respect to the expert-led case review sessions, the majority of focus group responses indicated they were valuable and enhanced the learning obtained from online modules, with review sessions being increasingly useful with later modules that were more challenging. In addition, both online survey responses and focus group feedback identified that participants would strongly recommend the modules to their colleagues (97% of survey responses, Mean 4.43, SD = 0.56). Finally, comparing results of the annual National Hematology Online Practice exam, programs that participated in the online hemoglobinopathy modules saw their fellows achieve a higher incremental change in their score from 2015 to 2016, as compared to their counterparts who did not participate in the online modules (increase of 18.5% vs. 14.8%). Conclusion/Implications: In this pilot implementation study, online hemoglobinopathy modules were shown to have high usability and user satisfaction with content. Fellows agreed that the e-learning modules addressed current gaps in curricula and variability in case-mix exposure. Among fellows who have completed the annual National Hematology Online Practice Exam, comparison of the 2015 and 2016 exam scores in participating and non-participating programs indicated improved performance on hemoglobinopathy-related questions. Taken together, our study has shown that our proposed e-learning-based curriculum has been a successful intervention in promoting, standardizing and enhancing education in hemoglobinopathies among Canadian hematology programs. Disclosures Bates: Eli Lilly Canada: Other: Partial salary support through Eli Lilly Canada/May Cohen Chair in Women's Health.


KINDAI ◽  
2020 ◽  
Vol 16 (2) ◽  
pp. 177-200
Author(s):  
Fitri Efendi

Abstrak : Fitri Efendi, Npm. 1811.32202.4732, Analisis Kompetensi, Sarana Prasarana Dan Perencanaan Program Pelatihan Terhadap Kualitas Tenaga Kerja Pada Dinas Tenaga Kerja Dan Transmigrasi Kabupaten Balangan Kalimantan Selatan, Dibawah Bimbingan Rosadiro Cahyono Dan Yudi Rahman, 2020. Tujuan penelitian yang ingin dicapai dalam penelitian ini, adalah Untuk mengetahui dan menganalisis pengaruh kompetensi terhadap kualitas tenaga kerja, Untuk mengetahui dan menganalisis pengaruh sarana prasarana terhadap kualitas tenaga kerja, Untuk mengetahui dan menganalisis pengaruh perencanaan program pelatihan terhadap kualitas tenaga kerja dan untuk mengetahui dan menganalisis pengaruh kompetensi, sarana prasarana dan perencanaan program pelatihan secara simultan terhadap kualitas tenaga kerja pada Dinas Tenaga Kerja Dan Transmigrasi Kabupaten Balangan Kalimantan Selatan. Populasi dari penelitian ini adalah tenaga kerja yang ikut pelatihan di Dinas Tenaga Kerja dan Transmigrasi Kabupaten Balangan Kalimantan Selatan yang berjumlah 224 orang sedangkan sampel dalam penelitian ini sebanyak 56 orang responden. Teknik pengolahan data yang akan dilakukan menggunakan metode kuantitatif dengan analisis regresi berganda dengan perangkat lunak spss. Hasil penelitian hiipotesis pertama menyebutkan secara parsial antara variabel kompetensi berpengaruh signifikan terhadap kualitas tenaga kerja, hipotesis kedua menyebutkan secara parsial antara variabel sarana prasarana berpengaruh signifikan terhadap kualitas tenaga kerja, hipotesis ketiga yang menyebutkan secara parsial antara variabel perencanaan program pelatihan berpengaruh signifikan terhadap kualitas tenaga kerja dan pengujian hipotesis keempat menunjukkan bahwa variabel kompentensi, sarana prasarana dan perencanaan program pelatihan secara simultan berpengaruh signifikan terhadap kualitas tenaga kerja pada Dinas Tenaga Kerja Dan Transmigrasi Kabupaten Balangan Kalimantan Selatan. Kata Kunci : Kompetensi, Sarana Prasarana, Perencanaan Program Pelatihan dan Kualitas Tenaga Kerja Abstract : Fitri Efendi, Npm. 1811.32202.4732, Competency Analysis, Infrastructure Facilities And Planning Of Training Programs To On The Quality Of The Workforce At The Department Manpower And Transmigration Of Balangan District, South Kalimantan, Under The Guidance Of Rosadiro Cahyono Dan Yudi Rahman, 2020. Research objectives to be achieved in this study, are to find out and analyze the effect of competence on the quality of labor, to find out and analyze the influence of infrastructure on the quality of labor, to know and analyze the effect of planning training programs on labor quality and to find out and analyze the effect of competency, infrastructure and planning of a simultaneous training program on the quality of the workforce at the Balangan Manpower and Transmigration Office of South Kalimantan Regency. The population of this study was the workforce who participated in training at the Department of Manpower and Transmigration of Balangan Regency, South Kalimantan, which numbered 224 people, while the sample in this study were 56 respondents. Data processing techniques will be performed using quantitative methods with multiple regression analysis with SPSS software. The results of the first hypotheses research mentioned partially between the competency variables significantly influence labor quality, the second hypothesis partially stated that between infrastructure variables had a significant effect on labor quality, the third hypothesis that partially mentioned training program planning variables had a significant effect on labor quality and testing the fourth hypothesis shows that the competency variables, infrastructure and planning of training programs simultaneously have a significant effect on the quality of the workforce at the Department of Manpower and Transmigration, Balangan Regency, South Kalimantan. Keywords: Competence, Infrastructure, Program Planning Workforce Training and Quality


2003 ◽  
Vol 21 (4) ◽  
pp. 184-189
Author(s):  
H.M. Mathers

Abstract The demographics of the nursery industry in the United States have changed dramatically in the past 10–15 years. Today, the majority of the worker level of nursery staff is Spanish-speaking. Approximately 90% of the Oregon and 60% of the Ohio nursery industry work forces are composed of Hispanic employees. Many of these employees understand little English, and Spanish is their primary language. In Oregon and Ohio 375 and 250 surveys, respectively, were distributed to Spanish-speaking nursery employees. One-hundred-ninety-three surveys were completed in Oregon, as compared to 127 in Ohio. The purpose of these surveys was to determine the backgrounds, experience level and work activities of Hispanic nursery employees, their technical information interests, and what resources were available to these employees. The survey responses came from 3 sources: Interviews at training programs, on-farm visits and mailed surveys in a newsletter format. The three sources were pooled and analyzed. The survey targeted nursery workers filling primarily manual labor positions; however, we found this staff level was predominantly Hispanic. Also, the majority of those surveyed were Hispanic employees who had risen to supervisor or crew leader level, such as propagation area supervisors. Plant identification (95%) and weed control (90%) were the foremost technical topics of interest of Ohio surveyed employees. This was significantly different from the survey results in Oregon where 81% chose insect control, 77% weed control, and 66% disease control as leading topics of interest. In Oregon, plant identification rated only 45%. Ninety-three percent of Ohio and 97% of Oregon newsletter readers wanted to receive future issues of the Spanish language newsletter. This finding was consistent with responses from nursery visits and training programs in the two states where respondents indicated their positive reception to receiving technical information in Spanish.


Sign in / Sign up

Export Citation Format

Share Document