scholarly journals Development and Initial Validation of a Program Director's Evaluation Form for Third-Year Residents

2015 ◽  
Vol 180 (suppl_4) ◽  
pp. 104-108 ◽  
Author(s):  
Anthony R. Artino ◽  
Ting Dong ◽  
David F. Cruess ◽  
William R. Gilliland ◽  
Steven J. Durning
Keyword(s):  
Medical School ◽  
Internal Consistency ◽  
Core Competencies ◽  
Reliability And Validity ◽  
Internal Consistency Reliability ◽  
Grade Point ◽  
Medical Expertise ◽  
Usmle Step ◽  
And Performance ◽  
Evaluation Survey

ABSTRACT Background: Using a previously developed postgraduate year (PGY)-1 program director's evaluation survey, we developed a parallel form to assess more senior residents (PGY-3). The PGY-3 survey, which aligns with the core competencies established by the Accreditation Council for Graduate Medical Education, also includes items that reflect our institution's military-unique context. Purpose: To collect feasibility, reliability, and validity evidence for the new PGY-3 evaluation. Methods: We collected PGY-3 data from program directors who oversee the education of military residents. The current study's cohort consisted of Uniformed Services University of the Health Sciences students graduating in 2008, 2009, and 2010. We performed exploratory factor analysis (EFA) to examine the internal structure of the survey and subjected each of the factors identified in the EFA to an internal consistency reliability analysis. We then performed correlation analysis to examine the relationships between PGY-3 ratings and several outcomes: PGY-1 ratings, cumulative medical school grade point average (GPA), and performance on U.S. Medical Licensing Examinations (USMLE) Step 1, Step 2 Clinical Knowledge, and Step 3. Results: Of the 510 surveys we distributed, 388 (76%) were returned. Results from the EFA suggested four factors: “Medical Expertise,” “Professionalism,” “Military-unique Practice,” and “Systems-based Practice.” Scores on these four factors showed good internal consistency reliability, as measured by Cronbach's α (α ranged from 0.92 to 0.98). Further, as expected, “Medical Expertise” and “Professionalism” had small to moderate correlations with cumulative medical school GPA and performance on the USMLE Step examinations. Conclusions: The new program director's evaluation survey instrument developed in this study appears to be feasible, and the scores that emerged have reasonable evidence of reliability and validity in a sample of third-year residents.

scholarly journals Development and Initial Validation of a Program Director's Evaluation Form for Medical School Graduates

2015 ◽  
Vol 180 (suppl_4) ◽  
pp. 97-103 ◽  
Author(s):  
Ting Dong ◽  
Steven J. Durning ◽  
William R. Gilliland ◽  
Kimberly A. Swygert ◽  
Anthony R. Artino
Keyword(s):  
Medical School ◽  
Core Competencies ◽  
Reliability And Validity ◽  
Internal Consistency Reliability ◽  
Survey Method ◽  
Good Reliability ◽  
Grade Point ◽  
Evaluation Form ◽  
Five Factors ◽  
And Performance

ABSTRACT Background: In the early 1990s, our group of interdepartmental academicians at the Uniformed Services University (USU) developed a PGY-1 (postgraduate year 1) program director evaluation form. Recently, we have revised it to better align with the core competencies established by the Accreditation Council for Graduate Medical Education. We also included items that reflected USU's military-unique context. Purpose: To collect feasibility, reliability, and validity evidence for our revised survey. Method: We collected PGY-1 data from program directors (PD) who oversee the training of military medical trainees. The cohort of the present study consisted of USU students graduating in 2010 and 2011. We performed exploratory factor analysis (EFA) to examine the factorial validity of the survey scores and subjected each of the factors identified in the EFA to an internal consistency reliability analysis. We then performed correlation analysis to examine the relationship between PD ratings and students' medical school grade point averages (GPAs) and performance on U.S. Medical Licensing Examinations Step assessments. Results: Five factors emerged from the EFA–—Medical Expertise, Military-unique Practice, Professionalism, System-based Practice, and Communication and Interpersonal Skills.” The evaluation form also showed good reliability and feasibility. All five factors were more strongly associated with students' GPA in the initial clerkship year than the first 2 years. Further, these factors showed stronger correlations with students' performance on Step 3 than other Step Examinations. Conclusions: The revised PD evaluation form seemed to be a valid and reliable tool to gauge medical graduates' first-year internship performance.

scholarly journals Objective predictors of intern performance

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Amanda C. Filiberto ◽  
Lou Ann Cooper ◽  
Tyler J. Loftus ◽  
Sonja S. Samant ◽  
George A. Sarosi ◽  
...  
Keyword(s):  
Medical Students ◽  
Medical School ◽  
Core Competencies ◽  
Cohort Analysis ◽  
The United States ◽  
Grade Point ◽  
Class Rank ◽  
Residency Programs ◽  
Usmle Step 1 ◽  
Usmle Step

Abstract Background Residency programs select medical students for interviews and employment using metrics such as the United States Medical Licensing Examination (USMLE) scores, grade-point average (GPA), and class rank/quartile. It is unclear whether these metrics predict performance as an intern. This study tested the hypothesis that performance on these metrics would predict intern performance. Methods This single institution, retrospective cohort analysis included 244 graduates from four classes (2015–2018) who completed an Accreditation Council for Graduate Medical Education (ACGME) certified internship and were evaluated by program directors (PDs) at the end of the year. PDs provided a global assessment rating and ratings addressing ACGME competencies (response rate = 47%) with five response options: excellent = 5, very good = 4, acceptable = 3, marginal = 2, unacceptable = 1. PDs also classified interns as outstanding = 4, above average = 3, average = 2, and below average = 1 relative to other interns from the same residency program. Mean USMLE scores (Step 1 and Step 2CK), third-year GPA, class rank, and core competency ratings were compared using Welch’s ANOVA and follow-up pairwise t-tests. Results Better performance on PD evaluations at the end of intern year was associated with higher USMLE Step 1 (p = 0.006), Step 2CK (p = 0.030), medical school GPA (p = 0.020) and class rank (p = 0.016). Interns rated as average had lower USMLE scores, GPA, and class rank than those rated as above average or outstanding; there were no significant differences between above average and outstanding interns. Higher rating in each of the ACGME core competencies was associated with better intern performance (p < 0.01). Conclusions Better performance as an intern was associated with higher USMLE scores, medical school GPA and class rank. When USMLE Step 1 reporting changes from numeric scores to pass/fail, residency programs can use other metrics to select medical students for interviews and employment.

Abstract 2173: Abbreviated 7-Item NIHSS for Inpatient Acute Stroke Management

Stroke ◽  
2012 ◽  
Vol 43 (suppl_1) ◽  
Author(s):  
Jennifer Parreira ◽  
Julie Nauser
Keyword(s):  
Acute Stroke ◽  
Internal Consistency ◽  
Recurrent Stroke ◽  
Reliability And Validity ◽  
Internal Consistency Reliability ◽  
Neurological Status ◽  
Criterion Validity ◽  
Suitable Alternative ◽  
Motor Movement ◽  
Left And Right

Background: Each year, nearly 800,000 people experience a new or recurrent stroke. To assess changes in neurological status during hospitalization, nurses are required to complete the 15-item National Institute of Health Stroke Scale (NIHSS), which is time consuming and therefore, may not be completed as thoroughly as it should. To ensure these patients are being effectively and adequately assessed, there is a need for an abbreviated NIHSS, which can be completed in a shorter time frame. Purpose: The purpose of this study was to determine the merits of an abbreviated NIHSS using seven items considered to be key to determining changes in neurological status. Methods: Over a six month period, 100 scores from the 15-item NIHSS were collected on a sample of 92 acute care stroke patients who had a mean age of 64.2; 52% were female. Seven items were calculated, which included level of consciousness, left and right arm motor movement, left and right leg motor movement, speech fluency, and speech clarity. To assess internal consistency reliability of the 7-item NIHSS, Cronbach's α was computed. To assess criterion validity, the 7-item NIHSS and 15-item NIHSS were correlated using Pearson's r . Results: The 15-item NIHSS ranged from 1 to 37, with a mean of 8.6. The 7-item NIHSS ranged from 0 to 24, with a mean of 4.9. For the 7-item NIHSS, Cronbach's α was .82 and Pearson's r was .96 (p<.001), when correlated with the 15-item NIHSS. Conclusions: Assessment of the acute stroke patient using the 15-item NIHSS is time-consuming, and therefore, may be underutilized. An abbreviated 7-item NIHSS demonstrated satisfactory evidence of internal consistency reliability and criterion validity, when correlated with the 15-item NIHSS, suggesting the 7-item version may be a suitable alternative to the longer version. More psychometric testing is warranted to fully evaluate reliability and validity of the 7-item NIHSS.

Assessment of Reliability and Validity of the Turkish Version of Heartland Forgiveness Scale

2012 ◽  
Vol 111 (2) ◽  
pp. 575-584 ◽  
Author(s):  
Asli Bugay ◽  
Ayhan Demir ◽  
Raquel Delevi
Keyword(s):  
Internal Consistency ◽  
Satisfaction With Life ◽  
Pearson Correlation ◽  
Reliability And Validity ◽  
Internal Consistency Reliability ◽  
Criterion Validity ◽  
Satisfaction With Life Scale ◽  
Turkish Version ◽  
Response Scale ◽  
Three Factor

The current study investigated the reliability and validity of the Turkish version of the Heartland Forgiveness Scale (HFS). The Turkish version of the HFS, the Ruminative Response Scale (RRS), and the Satisfaction with Life Scale (SWLS) were administered to 796 Turkish university students (430 women, 366 men) with a mean age of 20.6 yr. ( SD = 2.1). Internal consistency reliability was calculated and Cronbach alpha coefficients ranged between .71 and .82. Criterion validity (Pearson correlation) between the HFS and other scales ranged from −.09 to .34. Additionally, a confirmatory factor analysis was conducted to assess the fit of the standard three-factor solution of the HFS. The results suggested that the Turkish version of the HFS had adequate internal consistency, criterion validity, and reflected the standard three-factor structure, indicating that it can be reliably used to measure forgiveness among a Turkish sample.

scholarly journals Validity and reliability of the Indonesian version of tinnitus handycap inventory

2015 ◽  
Vol 24 (1) ◽  
pp. 36-42
Author(s):  
Jenny E. Bashiruddin ◽  
Widayat Alviandi ◽  
Alvin Reinaldo ◽  
Eka D. Safitri ◽  
Yupitri Pitoyo ◽  
...  
Keyword(s):  
Internal Consistency ◽  
Reliability And Validity ◽  
Internal Consistency Reliability ◽  
Psychometric Validation ◽  
Validity And Reliability ◽  
Functional Scale ◽  
Cross Sectional ◽  
Validity Test ◽  
Psychometric Instrument ◽  
High Internal Consistency

Background: To translate and assess the validity and reliability of the Indonesian version of Tinnitus Handycap Inventory (THI) as an psychometric instrument for evaluating the quality of life in tinnitus patients. This instrument will support the clinicians to determine the appropriate tinnitus management for them.Methods: A cross-sectional psychometric validation study was performed to assess the internal consistency, reliability and validity of the Indonesian version of THI in 50 subjective tinnitus patients at ENT outpatient clinic of Cipto Mangunkusumo Hospital between May-August 2010. 25 question items of original THI were translated, back-translated and validated using the transcultural validation by WHO.Results: The validity test demonstrated a significant correlation in the emotional and the catastrophic scale whilst there was no significant correlation in the functional scale for item F2 and particularly for item F15. Nevertheless, the validity test on the functional scale showed a good result. This study also showed high internal consistency and reliability for the total scale (Cronbach-α = 0.91)Conclusion: The evaluation result indicated that the reliability of adapted Indonesian version of the THI in our study is relatively high and could be applied in clinical examination or further otolaryngology study by both specialists and general physicians.

The Modified Interpersonal Relationship Scale: Reliability and Validity,

1993 ◽  
Vol 73 (3_part_1) ◽  
pp. 995-1004
Author(s):  
Jane L. Garthoeffner ◽  
Carolyn S. Henry ◽  
Linda C. Robinson
Keyword(s):  
Interpersonal Relationships ◽  
Internal Consistency ◽  
Concurrent Validity ◽  
Interpersonal Relationship ◽  
Reliability And Validity ◽  
Internal Consistency Reliability ◽  
Self Report ◽  
Varimax Rotation ◽  
Scale Reliability

This study was designed to evaluate a modification of the Interpersonal Relationship Scale and to establish subscales representing dimensions of intimacy (N = 356). The initial self-report scale was tested for internal consistency reliability. Next, subscales were identified using principal components factoring with varimax rotation. Internal consistency reliability and concurrent validity of the modified over-all scale and subscales were examined. The modified scale and subscales provided reliable and valid measures of the quality of interpersonal relationships in young adults.

Turkish Validation of the Standardized Cosmesis and Health Nasal Outcomes Survey

2019 ◽  
Vol 35 (04) ◽  
pp. 397-399 ◽  
Author(s):  
Sercan Gode ◽  
Arin Ozturk ◽  
Mustafa Sahin ◽  
Veysel Berber ◽  
Fazil Apaydin
Keyword(s):  
Internal Consistency ◽  
Repeated Measures ◽  
Reliability And Validity ◽  
Internal Consistency Reliability ◽  
Control Group ◽  
Turkish Version ◽  
Cronbach’S Alpha ◽  
Cronbach's Alpha ◽  
Significant Difference ◽  
Turkish Validation

AbstractThe objective of this study is to provide a valid and reliable Turkish version of the original Standardized Cosmesis and Health Nasal Outcomes Survey (SCHNOS). The SCHNOS questionnaire was translated from English to Turkish using standardized guidelines. Participants completed the questionnaire twice, with an interval of 7 to 10 days. Reliability and validity analyses were performed based on these answers. Differences between the rhinoplasty and control groups, the internal consistency of the instrument (Cronbach's alpha coefficient), and the strength of association between the two repeated measures (Pearson's correlation coefficient) were analyzed. Of the 188 included individuals (106 females, 82 males; mean age 28 ± 8.4 years), 97 were in the rhinoplasty group and 91 were in the control group. The mean total SCHNOS scores were 0.6 ± 0.4 and 29.4 ± 8.9 in the control and rhinoplasty groups, respectively, and there was a statistically significant difference in total scores between the groups (p < 0.001). The internal consistency reliability of the scale was found to be highly significant (Cronbach's alpha = 0.96, with a lower 95% confidence interval of 0.89). The test–retest correlation value was 0.95 for the total score, and each item had a strong test–retest correlation, ranging from 0.92 to 0.96. The Turkish version of the SCHNOS is a valid and reliable scale for evaluating both functional and cosmetic outcomes in Turkish-speaking patients undergoing rhinoplasty.

Assessing Nursing Student Perceptions of the Clinical Learning Environment: Refinement and Testing of the SECEE Inventory

2009 ◽  
Vol 17 (3) ◽  
pp. 232-246 ◽  
Author(s):  
Kari Sand-Jecklin
Keyword(s):  
Learning Environment ◽  
Student Perceptions ◽  
Internal Consistency ◽  
Nursing Student ◽  
Student Evaluation ◽  
Reliability And Validity ◽  
Internal Consistency Reliability ◽  
Student Evaluations ◽  
Clinical Learning ◽  
Clinical Learning Environment

The Student Evaluation of Clinical Education Environment (SECEE) instrument was developed to provide information about the quality of the student clinical learning environment to assist clinical agencies, nursing faculty, and administers in selecting clinical sites that best promote student learning. The SECEE Version 3 was used in all clinical courses at a large mid-Atlantic university from 2001 to 2005. Data from more than 2,700 inventories were analyzed to assess instrument reliability and validity. Internal consistency reliability based on coefficient alpha was .94, with subscale alphas ranging from .82 to .94. Comparisons by analysis of variance revealed significant differences in student evaluations based on both clinical sites and clinical faculty. Confirmatory factor analysis supported the predetermined three factors (subscales) and subscale item content, except for two items. The SECEE Version 3 demonstrated strong internal consistency reliability and the ability to discriminate between student evaluations of distinct clinical sites and faculty and provided practically useful information to both faculty and agency staff.

Psychometric Evaluation of the Nursing Stress Scale (NSS) Among Chinese Nurses in Taiwan

2007 ◽  
Vol 15 (2) ◽  
pp. 133-144 ◽  
Author(s):  
Mei-Hua Lee ◽  
William L. Holzemer ◽  
Julia Faucett
Keyword(s):  
Internal Consistency ◽  
Stress Reduction ◽  
Medical Center ◽  
Reliability And Validity ◽  
Psychometric Evaluation ◽  
Internal Consistency Reliability ◽  
Chinese Version ◽  
Stress Scale ◽  
Nursing Stress ◽  
Test Retest Reliability

The purpose of this study was to translate the Nursing Stress Scale (NSS) into Chinese and test its reliability and validity among Chinese nurses in Taiwan. Potential participants were asked to self-administer a Chinese version of the NSS. The agreement estimation was used to determine the equivalence of the meaning between the Chinese and original English versions and was rated by five bilingual nurses as 92% accurate for the 34 items. The test-retest reliability for the NSS at 2 weeks was .71 (p = .022, n = 10). Internal consistency reliability and factor analysis were tested with 770 nurses from 65 inpatient units at a medical center in Taiwan. The internal consistency of the Chinese version of the NSS for an overall coefficient alpha is .91 for the total scale, and ranges from .67 to .79 for the subscales. The Chinese version of the NSS explains 53.77% of the variance in work stressors among Chinese nurses in Taiwan. Overall, the Chinese version of the NSS is internally consistent but may not be stable over 2 weeks. There was adequate evidence of the reliability and validity of the NSS-Chinese as an instrument appropriate to measure work stress among Chinese nurses. The translated NSS could be a useful tool for examining the frequency and major sources of stress experienced by Chinese nurses in hospital settings, and for the development of appropriate interventions for stress reduction.

Measuring Normative Beliefs and Shared Behavioral Expectations in Organizations: The Reliability and Validity of the Organizational Culture Inventory

1993 ◽  
Vol 72 (3_suppl) ◽  
pp. 1299-1330 ◽  
Author(s):  
Robert A. Cooke ◽  
Janet L. Szumal
Keyword(s):  
Organizational Culture ◽  
Internal Consistency ◽  
Reliability And Validity ◽  
Temporal Consistency ◽  
Normative Beliefs ◽  
Behavioral Expectations ◽  
Scale Scores ◽  
Organizational Culture Inventory ◽  
And Performance ◽  
And Behavior

The Organizational Culture Inventory measures 12 sets of normative beliefs or shared behavioral expectations associated with three general types of cultures, Constructive, Passive-Defensive, and Aggressive-Defensive These cultural norms are hypothesized to influence the thinking and behavior of organizational members, their motivation and performance, and their satisfaction and stress. As components of organizational culture, behavioral expectations are considered to be shared and enduring in nature. Tests of three types of reliability—internal consistency, interrater, and test-retest—and two types of validity—construct and criterion-related—on data provided by 4,890 respondents indicate that the inventory is a dependable instrument for assessing the normative aspects of culture. Obtained alpha coefficients support the internal consistency of the scales; tests for interrater agreement show that significant variance in individuals' responses is explained by their organizational membership; and tests for differences across time show the temporal consistency of scale scores. Factor analysis results provide general support for the construct validity of the scales, most of which were related to both individual and organizational criteria as predicted.

Sign in / Sign up

Export Citation Format

Share Document