scholarly journals Development and Initial Validation of a Program Director's Evaluation Form for Medical School Graduates

2015 ◽  
Vol 180 (suppl_4) ◽  
pp. 97-103 ◽  
Author(s):  
Ting Dong ◽  
Steven J. Durning ◽  
William R. Gilliland ◽  
Kimberly A. Swygert ◽  
Anthony R. Artino
Keyword(s):  
Medical School ◽  
Core Competencies ◽  
Reliability And Validity ◽  
Internal Consistency Reliability ◽  
Survey Method ◽  
Good Reliability ◽  
Grade Point ◽  
Evaluation Form ◽  
Five Factors ◽  
And Performance

ABSTRACT Background: In the early 1990s, our group of interdepartmental academicians at the Uniformed Services University (USU) developed a PGY-1 (postgraduate year 1) program director evaluation form. Recently, we have revised it to better align with the core competencies established by the Accreditation Council for Graduate Medical Education. We also included items that reflected USU's military-unique context. Purpose: To collect feasibility, reliability, and validity evidence for our revised survey. Method: We collected PGY-1 data from program directors (PD) who oversee the training of military medical trainees. The cohort of the present study consisted of USU students graduating in 2010 and 2011. We performed exploratory factor analysis (EFA) to examine the factorial validity of the survey scores and subjected each of the factors identified in the EFA to an internal consistency reliability analysis. We then performed correlation analysis to examine the relationship between PD ratings and students' medical school grade point averages (GPAs) and performance on U.S. Medical Licensing Examinations Step assessments. Results: Five factors emerged from the EFA–—Medical Expertise, Military-unique Practice, Professionalism, System-based Practice, and Communication and Interpersonal Skills.” The evaluation form also showed good reliability and feasibility. All five factors were more strongly associated with students' GPA in the initial clerkship year than the first 2 years. Further, these factors showed stronger correlations with students' performance on Step 3 than other Step Examinations. Conclusions: The revised PD evaluation form seemed to be a valid and reliable tool to gauge medical graduates' first-year internship performance.

scholarly journals Development and Initial Validation of a Program Director's Evaluation Form for Third-Year Residents

2015 ◽  
Vol 180 (suppl_4) ◽  
pp. 104-108 ◽  
Author(s):  
Anthony R. Artino ◽  
Ting Dong ◽  
David F. Cruess ◽  
William R. Gilliland ◽  
Steven J. Durning
Keyword(s):  
Medical School ◽  
Internal Consistency ◽  
Core Competencies ◽  
Reliability And Validity ◽  
Internal Consistency Reliability ◽  
Grade Point ◽  
Medical Expertise ◽  
Usmle Step ◽  
And Performance ◽  
Evaluation Survey

ABSTRACT Background: Using a previously developed postgraduate year (PGY)-1 program director's evaluation survey, we developed a parallel form to assess more senior residents (PGY-3). The PGY-3 survey, which aligns with the core competencies established by the Accreditation Council for Graduate Medical Education, also includes items that reflect our institution's military-unique context. Purpose: To collect feasibility, reliability, and validity evidence for the new PGY-3 evaluation. Methods: We collected PGY-3 data from program directors who oversee the education of military residents. The current study's cohort consisted of Uniformed Services University of the Health Sciences students graduating in 2008, 2009, and 2010. We performed exploratory factor analysis (EFA) to examine the internal structure of the survey and subjected each of the factors identified in the EFA to an internal consistency reliability analysis. We then performed correlation analysis to examine the relationships between PGY-3 ratings and several outcomes: PGY-1 ratings, cumulative medical school grade point average (GPA), and performance on U.S. Medical Licensing Examinations (USMLE) Step 1, Step 2 Clinical Knowledge, and Step 3. Results: Of the 510 surveys we distributed, 388 (76%) were returned. Results from the EFA suggested four factors: “Medical Expertise,” “Professionalism,” “Military-unique Practice,” and “Systems-based Practice.” Scores on these four factors showed good internal consistency reliability, as measured by Cronbach's α (α ranged from 0.92 to 0.98). Further, as expected, “Medical Expertise” and “Professionalism” had small to moderate correlations with cumulative medical school GPA and performance on the USMLE Step examinations. Conclusions: The new program director's evaluation survey instrument developed in this study appears to be feasible, and the scores that emerged have reasonable evidence of reliability and validity in a sample of third-year residents.

scholarly journals Psychometric evaluation of the Chinese version of the Snizek-revised Hall’s Professionalism Inventory Scale

2019 ◽  
Vol 47 (3) ◽  
pp. 1154-1168 ◽  
Author(s):  
Xiaoqian Chen ◽  
Qi Yu ◽  
Feifei Yu ◽  
Yixiang Huang ◽  
Lingling Zhang
Keyword(s):  
Factor Analysis ◽  
Confirmatory Factor Analysis ◽  
Internal Consistency ◽  
Convergent Validity ◽  
Discriminant Validity ◽  
Reliability And Validity ◽  
Internal Consistency Reliability ◽  
Good Reliability ◽  
Five Factors ◽  
Confirmatory Factor

Objective This study was performed to assess the reliability and validity of the Chinese version of the Snizek-revised Hall’s Professionalism Inventory Scale (C-SR-HPIS). Methods Exploratory factor analysis and confirmatory factor analysis were used to evaluate the construct validity of the C-SR-HPIS. The average variance extracted (AVE) and square root of the AVE were calculated and correlation analyses were performed to test the convergent validity and discriminant validity, respectively. Cronbach’s alpha (α) coefficient was used to test the internal consistency reliability. Results Data for 355 clinical nurses in mainland China were collected. Five factors were extracted, accounting for 58.86% of the total explained variance, and 20 items were selected for the C-SR-HPIS. The confirmatory factor analysis suggested good fitness of the modified model. The AVE was acceptable for convergent validity. The square roots of the AVE of the five factors were larger than their correlation coefficients with other factors, showing suitable discriminant validity. Cronbach’s α coefficient of internal consistency reliability of the overall scale was 0.76, indicating good reliability of the scale. Conclusions This study demonstrated good reliability and validity of the C-SR-HPIS and provides a quantitative tool for the assessment of nursing professionalism in China.

scholarly journals The Influence of Information Interaction Behavior on Value Co-Creation Business Model of Online Education Enterprises Performances from the Perspective of Supply Chain

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Jingjing Lv ◽  
Nan Wang ◽  
Shaoxin Xiang
Keyword(s):  
Supply Chain ◽  
Online Education ◽  
Business Model ◽  
Questionnaire Survey ◽  
Positive Impact ◽  
Reliability And Validity ◽  
Survey Method ◽  
Information Interaction ◽  
And Performance ◽  
The Impact

This paper analyzes the impact of information interaction ability on the value co-creation business model of online education enterprises from the perspective of supply chain. Integrate the perspective of supply chain and summarize the content of supply chain capability and performance. This paper analyzes the connotation and shortcomings of information interaction ability, combines the connotation of value co-creation to maximize the advantages of information interaction, assumes the impact of information interaction on value co-creation through questionnaire survey method, uses software to analyze the reliability and validity of data, and proves that the data are reasonable. Information interaction has a positive impact on the value co-creation business model of online education enterprises.

scholarly journals The Feasibility, Reliability, and Validity of a Program Director???s (Supervisor???s) Evaluation Form for Medical School Graduates

2005 ◽  
Vol 80 (10) ◽  
pp. 964-968 ◽  
Author(s):  
Steven J. Durning ◽  
Louis N. Pangaro ◽  
Linda L. Lawrence ◽  
Donna Waechter ◽  
John McManigle ◽  
...  
Keyword(s):  
Medical School ◽  
Reliability And Validity ◽  
Program Director ◽  
Evaluation Form

scholarly journals Development and preliminary validation of the competency model for rural general practitioner in China

2020 ◽  
Author(s):  
Yun Wei ◽  
Yanli Liu ◽  
Xuexue Zhao ◽  
Guanghui Jin ◽  
Yali Zhao ◽  
...  
Keyword(s):  
General Practitioner ◽  
Pilot Study ◽  
Rural Areas ◽  
Panel Discussion ◽  
Reliability And Validity ◽  
Internal Consistency Reliability ◽  
Competency Model ◽  
Face Validity ◽  
Validity And Reliability ◽  
Good Reliability

Abstract Background There is a shortage of rural general practitioners (GPs) in China. Training programs have been initiated to meet the needs of GP workforce in the rural areas; however, there is an absence of validated tool to assess their competencies. Objective This study aimed to develop a competency model for rural general practitioner (CMRGP) after training in China and to examine its validity and reliability. Methods A multistage process was adopted to develop the CMRGP comprised literature review, panel discussion and expert consultation, and the initial version of CMRGP was reduced from 10 domains and 77 items to 7 domains and 54 items. A pilot study was conducted among 202 rural GPs for the psychometric evaluation and application of the initial version of CMRGP, in which a questionnaire on the importance of items and self-evaluation was completed by the GPs. Results In the pilot study, 132 completed questionnaires (65.3%) were returned. Acceptability and face validity of the CMRGP were supported by high importance scores of the items, in which 52 out of 54 items achieved score higher than 4.00 (possible score from 0 to 5). Factor analysis supported the construct validity. After the modification, the final version of CMRGP contained 6 domains and 47 items. Good reliability was supported by internal consistency reliability (Cronbach’s α was 0.98) and split-half reliability (Spearman–Brown coefficient was 0.99). Conclusions The CMRGP demonstrated good reliability and validity. Pilot study showed its potential for application in the rural general practice and training program.

Pre-admission Grades And Student Performance: The Malaysian Medical School Experience

2012 ◽  
Vol 5 (3) ◽  
pp. 145
Author(s):  
Basir Abidin ◽  
Noor A Suliman ◽  
Salwa M Din ◽  
Norhafizah A Manan
Keyword(s):  
Medical School ◽  
Student Performance ◽  
Medical Schools ◽  
Natural Sciences ◽  
High Achievers ◽  
Grade Point ◽  
Common Assessments ◽  
Preclinical Phase ◽  
Medical Program ◽  
And Performance

Prior academic achievement is often considered the best predictor and therefore the preadmission criteria for highly competitive medical schools. Most studies that advocate this viewpoint analyzed cohorts based on results of a central examination taken by students who come from various pre-university setups and backgrounds. Far less is known about students who come from a common setup, sit for common assessments, and the effect on their achievement in medical schools. This study sets out to investigate the correlation and association of preadmission grades and various summative results during preclinical year medical program. The association of the whole group is investigated. The association of subgroups (low, intermediate, and high achievers) to preclinical achievement was also investigated to get insight about the consistency. As a group, the pre-university performance (especially in natural sciences) has moderate to high correlations to various results in preclinical phase. As subgroups: low achievers are less consistent and predictable; high achievers are best correlated to various results of preclinical phase; while intermediate achievers are in between. The cumulative grade point and performance in natural sciences in a common pre-university program can serve not only as a predictor of performance, but also as an indicator for consistency of performance in preclinical phase of medical programs. This information may be of use to medical school admission and selection committees.

Factors affecting usage of e-resources: scale development and validation

2017 ◽  
Vol 69 (1) ◽  
pp. 64-75 ◽  
Author(s):  
Ruchi Jain Garg ◽  
Vinod Kumar ◽  
Vandana
Keyword(s):  
Convergent Validity ◽  
Discriminant Validity ◽  
Reliability And Validity ◽  
Internal Consistency Reliability ◽  
Test Reliability ◽  
Equation Modeling ◽  
Content Type ◽  
Postgraduate Students ◽  
Factors Affecting ◽  
Five Factors

Purpose The purpose of this paper is to develop and validate a scale involving the factors affecting usage of e-resources. Design/methodology/approach The present study identifies five factors affecting the usage of e-resources: training modes, awareness, influencers, utilitarian benefits, and experiential and hedonic benefits. To generate measurement items, two focus group discussions were carried out which resulted in 20 items. To empirically ensure reliability and validity of scale, data were collected from 347 postgraduate students actively using e-resources. To test reliability of scale, internal consistency reliability and construct reliability were examined. The construct validity of scale (which includes convergent validity and discriminant validity) was verified through confirmatory factor analysis using structure equation modeling. Findings Findings of the study present a statistically reliable and valid scale consisting of five factors and 18 items. Originality/value The present study is one of the distinctive studies on introducing a scale employing factors affecting usage of e-resources.

scholarly journals Objective predictors of intern performance

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Amanda C. Filiberto ◽  
Lou Ann Cooper ◽  
Tyler J. Loftus ◽  
Sonja S. Samant ◽  
George A. Sarosi ◽  
...  
Keyword(s):  
Medical Students ◽  
Medical School ◽  
Core Competencies ◽  
Cohort Analysis ◽  
The United States ◽  
Grade Point ◽  
Class Rank ◽  
Residency Programs ◽  
Usmle Step 1 ◽  
Usmle Step

Abstract Background Residency programs select medical students for interviews and employment using metrics such as the United States Medical Licensing Examination (USMLE) scores, grade-point average (GPA), and class rank/quartile. It is unclear whether these metrics predict performance as an intern. This study tested the hypothesis that performance on these metrics would predict intern performance. Methods This single institution, retrospective cohort analysis included 244 graduates from four classes (2015–2018) who completed an Accreditation Council for Graduate Medical Education (ACGME) certified internship and were evaluated by program directors (PDs) at the end of the year. PDs provided a global assessment rating and ratings addressing ACGME competencies (response rate = 47%) with five response options: excellent = 5, very good = 4, acceptable = 3, marginal = 2, unacceptable = 1. PDs also classified interns as outstanding = 4, above average = 3, average = 2, and below average = 1 relative to other interns from the same residency program. Mean USMLE scores (Step 1 and Step 2CK), third-year GPA, class rank, and core competency ratings were compared using Welch’s ANOVA and follow-up pairwise t-tests. Results Better performance on PD evaluations at the end of intern year was associated with higher USMLE Step 1 (p = 0.006), Step 2CK (p = 0.030), medical school GPA (p = 0.020) and class rank (p = 0.016). Interns rated as average had lower USMLE scores, GPA, and class rank than those rated as above average or outstanding; there were no significant differences between above average and outstanding interns. Higher rating in each of the ACGME core competencies was associated with better intern performance (p < 0.01). Conclusions Better performance as an intern was associated with higher USMLE scores, medical school GPA and class rank. When USMLE Step 1 reporting changes from numeric scores to pass/fail, residency programs can use other metrics to select medical students for interviews and employment.

scholarly journals Benefits and Feasibility of Using Videos to Assess Medical School Applicants’ Empathetic Abilities in Multiple Mini Interviews

2020 ◽  
Author(s):  
Kyong-Jee Kim ◽  
Nam Young Lee ◽  
Bum Sun Kwon
Keyword(s):  
Construct Validity ◽  
Medical School ◽  
Video Clip ◽  
Reliability And Validity ◽  
Perceived Difficulty ◽  
And Performance ◽  
Candidate Perceptions ◽  
Relationship Of ◽  
Doctor Communication ◽  
The Relationship

Abstract Purpose We sought to evaluate the feasibility and benefits of using video-based scenarios in Multiple Mini Interviews (MMIs) to assess candidate’s empathic abilities by investigating candidate perceptions and the acceptability, fairness, reliability, and validity of the test. Methods The study sample was candidates for admission interviews held in the MMI format at a medical school in South Korea. In this six-station MMI, one station included a 2-min video clip of a patient-doctor communication scenario to assess candidate emphatic abilities, whereas paper-based scenarios were used in the other stations. Candidate’s perceptions and acceptability of using the video-based scenario in the empathy station were examined using a 41-item post-MMI questionnaire. Fairness of the test was assessed by means of differences in candidate perceptions and performance across different demographics or backgrounds. Construct validity was assessed by examining the relationship of candidate performances in the empathy station with those in other stations. The G-coefficient was analyzed to estimate the reliability of the test. Results Eighty-two questionnaires were returned, a 97.6% response rate. Candidates showed overall positive perceptions of the video-based scenario and they found it authentic and interesting. The test was fair as there were no differences in candidates’ perceptions of the patient-doctor relationship presented in the video clip and neither in their performance nor in their perceived difficulty of the station across demographics or backgrounds. Construct validity was established as candidate performance in the empathy station was not associated with that of any other stations. The G-coefficient was 0.74. Conclusions The present study demonstrates that the video-based scenario is a feasible tool to assess candidate’s empathy in the MMI.

scholarly journals Development of the Russian Version of the Questionnaire for the Problematic Use of Social Networks

2018 ◽  
Vol 26 (3) ◽  
pp. 33-55
Author(s):  
N.A. Sirota ◽  
D.V. Moskovchenko ◽  
V.M. Yaltonsky ◽  
A.V. Yaltonskaya
Keyword(s):  
Social Networks ◽  
Online Communication ◽  
Reliability And Validity ◽  
Negative Consequences ◽  
Cognitive Emotion Regulation ◽  
Good Reliability ◽  
Russian Version ◽  
Five Factors ◽  
Personal Anxiety ◽  
Good Internal Consistency

We present the results of modification and approbation of the Problematic Facebook (Social Networks) Use Scale on the Russian sample. The scale assesses various aspects of social networks use including preferring online communication among other types, using social networks to regulate emotions, constant thinking about social networks and frequent compulsive social network visits and their negative consequences. The study involved 900 people who filled out electronic forms posted in the leading Russian social networks — Facebook, Vkontakte, Instagram. The mean age was 28.6 ± 7.5. It is shown that the Russian version of Problematic Social Networks Use Scale has good internal consistency. It consists of five factors and has good reliability and validity scores. Significant differences in cognitive emotion regulation strategies, positive and negative affect and personal anxiety in the low- and high-use groups confirm the external validity of the scale.

Sign in / Sign up

Export Citation Format

Share Document