scholarly journals Potential Impact of Pass/Fail Scores on USMLE Step 1: Predictors of Excellence in Obstetrics and Gynecology Residency Training

2021 ◽  
Vol 8 ◽  
pp. 238212052110374
Author(s):  
Sejal Tamakuwala ◽  
Joshua Dean ◽  
Katherine J. Kramer ◽  
Adib Shafi ◽  
Sarah Ottum ◽  
...  
Keyword(s):  
Student Performance ◽  
Clinical Performance ◽  
Predictive Ability ◽  
Obstetrics And Gynecology ◽  
Resident Performance ◽  
Application Service ◽  
Usmle Step 1 ◽  
Usmle Step ◽  
Clerkship Grade ◽  
Residency Application

AIM The study aims to determine resident applicant metrics most predictive of academic and clinical performance as measured by the Council of Resident Education in Obstetrics and Gynecology (CREOG) examination scores and Accreditation Council for Graduate Medical Education (ACGME) clinical performance (Milestones) in the aftermath of United States Medical Licensing Examination Scores (USMLE) Step 1 becoming a pass/fail examination. METHODS In this retrospective study, electronic and paper documents for Wayne State University Obstetrics and Gynecology residents matriculated over a 5-year period ending July 2018 were collected. USMLE scores, clerkship grade, and wording on the letters of recommendation as well as Medical Student Performance Evaluation (MSPE) were extracted from the Electronic Residency Application Service (ERAS) and scored numerically. Semiannual Milestone evaluations and yearly CREOG scores were used as a marker of resident performance. Statistical analysis on residents (n = 75) was performed using R and SPSS and significance was set at P < .05. RESULTS Mean USMLE score correlated with CREOG performance and, of all 3 Steps, Step 1 had the tightest association. MSPE and class percentile also correlated with CREOGs. Clerkship grade and recommendation letters had no correlation with resident performance. Of all metrics provided by ERAS, none taken alone, were as useful as Step 1 scores at predicting performance in residency. Regression modeling demonstrated that the combination of Step 2 scores with MSPE wording restored the predictive ability lost by Step 1. CONCLUSIONS The change of USMLE Step 1 to pass/fail may alter resident selection strategies. Other objective markers are needed in order to evaluate an applicant’s future performance in residency.

scholarly journals Examining the Factors Influencing Applicants’ Placement on One General Surgery Program’s Rank Order List

2022 ◽  
pp. 000313482110680
Author(s):  
Alexandra Hahn ◽  
Jessica Gorham ◽  
Alaa Mohammed ◽  
Brian Strollo ◽  
George Fuhrman
Keyword(s):  
Student Performance ◽  
Rank Order ◽  
Univariate Analysis ◽  
Geographic Location ◽  
Geographic Region ◽  
Rank List ◽  
Usmle Step ◽  
Clerkship Grade ◽  
Residency Application ◽  
Letters Of Recommendation

Purpose Surgery residency applications include variables that determine an individual’s rank on a program’s match list. We performed this study to determine which residency application variables are the most impactful in creating our program’s rank order list. Methods We completed a retrospective examination of all interviewed applicants for the 2019 match. We recorded United States Medical Licensing Examinations (USMLE) step I and II scores, class quartile rank from the Medical Student Performance Evaluation (MSPE), Alpha Omega Alpha (AOA) membership, geographic region, surgery clerkship grade, and grades on other clerkships. The MSPE and letters of recommendation were reviewed by two of the authors and assigned a score of 1 to 3, where 1 was weak and 3 was strong. The same two authors reviewed the assessments from each applicant’s interview and assigned a score from 1-5, where 1 was poor and 5 was excellent. Univariate analysis was performed, and the significant variables were used to construct an adjusted multivariate model with significance measured at P < .05. Results Univariate analysis for all 92 interviewed applicants demonstrated that USMLE step 2 scores ( P = .002), class quartile rank ( P = .004), AOA status ( P = .014), geographic location ( P < .001), letters of recommendation ( P < .001), and interview rating ( P < .001) were significant in predicting an applicant’s position on the rank list. On multivariate analysis only USMLE step 2 ( P = .018) and interview ( P < .001) remained significant. Conclusion USMLE step 2 and an excellent interview were the most important factors in constructing our rank order list. Applicants with a demonstrated strong clinical fund of knowledge that develop a rapport with our faculty and residents receive the highest level of consideration for our program.

scholarly journals Surveying Obstetrics and Gynecology Residents About Their Residency Applications, Interviews, and Ranking

2021 ◽  
Vol 13 (2) ◽  
pp. 257-265
Author(s):  
Karen E. George ◽  
Gregory M. Gressel ◽  
Tony Ogburn ◽  
Mark B. Woodland ◽  
Erika Banks
Keyword(s):  
United States ◽  
The United States ◽  
Obstetrics And Gynecology ◽  
Application Process ◽  
Factors Associated ◽  
Usmle Step 1 ◽  
Usmle Step ◽  
Residency Application

ABSTRACT Background Residency applications have increased in the last decade, creating growing challenges for applicants and programs. Objective We evaluated factors associated with application and match into obstetrics and gynecology residency. Methods During the annual in-training examination administered to all obstetrics and gynecology residents in the United States, residents were surveyed on the residency application process. Results Ninety-five percent (5094 of 5347) residents responded to the survey. Thirty-six percent reported applying to 30 or fewer programs, 26.7% applied to more than 31 programs, and 37.1% opted not to answer this question. Forty-nine percent of residents received honors in their obstetrics and gynecology clerkship and 37.1% did not. The majority of residents (88.6%) reported scoring between 200 and 250 on USMLE Step 1. Eighty-six percent matched into one of their top 5 programs. The only factor associated with matching in residents' top 5 programs was receiving honors in their clerkship (OR 1.29; 95% CI 1.08–1.54; P &lt; .005). The only factor associated with matching below the top 5 programs was a couples match (OR 0.56; 95% CI 0.43–0.72; P &lt; .001). In choosing where to apply, residents identified program location and reputation as the most important factors, while for ranking, location and residency culture were the most important. Conclusions Most obstetrics and gynecology residents reported matching into their top 5 choices. Receiving an honors grade in the clerkship was the only factor associated with matching in applicants' top 5 programs. Location was the most important factor for applying to and ranking of programs.

The State of the Otolaryngology Match: A Review of Applicant Trends, “Impossible” Qualifications, and Implications

2017 ◽  
Vol 156 (6) ◽  
pp. 985-990 ◽  
Author(s):  
Sarah N. Bowe ◽  
Cecelia E. Schmalbach ◽  
Adrienne M. Laury
Keyword(s):  
The Past ◽  
Application Service ◽  
Research Publications ◽  
Medical Licensing ◽  
Usmle Step 1 ◽  
Licensing Examination ◽  
Usmle Step ◽  
And Performance ◽  
Residency Application ◽  
The Relationship

Objective This State of the Art Review aims (1) to define recent qualifications of otolaryngology resident applicants by focusing on United States Medical Licensing Examination (USMLE) scores, Alpha Omega Alpha (AOA) status, and research/publications and (2) to summarize the current literature regarding the relationship between these measures and performance in residency. Data Sources Electronic Residency Application Service, National Residency Matching Program, PubMed, Ovid, and GoogleScholar. Review Methods Electronic Residency Application Service and National Residency Matching Program data were analyzed to evaluate trends in applicant numbers and qualifications. Additionally, a literature search was performed with the aforementioned databases to identify relevant articles published in the past 5 years that examined USMLE Step 1 scores, AOA status, and research/publications. Conclusions Compared with other highly competitive fields over the past 3 years, the only specialty with decreasing applicant numbers is otolaryngology, with the rest remaining relatively stable or slightly increased. Additionally, USMLE Step 1 scores, AOA status, and research/publications do not reliably correlate with performance in residency. Implications for Practice The consistent decline in applications for otolaryngology residency is concerning and reflects a need for change in the current stereotype of the “ideal” otolaryngology applicant. This includes consideration of additional selection measures focusing on noncognitive and holistic qualities. Furthermore, otolaryngology faculty should counsel medical students that applying in otolaryngology is not “impossible” but rather a feasible and worthwhile endeavor.

A comparison of student performance on discipline-specific versus integrated exams in a medical school course

2013 ◽  
Vol 37 (4) ◽  
pp. 370-376 ◽  
Author(s):  
Andrew R. Thompson ◽  
Mark W. Braun ◽  
Valerie D. O'Loughlin
Keyword(s):  
Medical School ◽  
Student Performance ◽  
Bloom's Taxonomy ◽  
Integrated Instruction ◽  
Bloom’S Taxonomy ◽  
Significant Difference ◽  
Usmle Step 1 ◽  
Usmle Step ◽  
Exam Scores ◽  
The Impact

Curricular reform is a widespread trend among medical schools. Assessing the impact that pedagogical changes have on students is a vital step in review process. This study examined how a shift from discipline-focused instruction and assessment to integrated instruction and assessment affected student performance in a second-year medical school pathology course. We investigated this by comparing pathology exam scores between students exposed to traditional discipline-specific instruction and exams (DSE) versus integrated instruction and exams (IE). Exam content was controlled, and individual questions were evaluated using a modified version of Bloom's taxonomy. Additionally, we compared United States Medical Licensing Examination (USMLE) step 1 scores between DSE and IE groups. Our findings indicate that DSE students performed better than IE students on complete pathology exams. However, when exam content was controlled, exam scores were equivalent between groups. We also discovered that the integrated exams were composed of a significantly greater proportion of questions classified on the higher levels of Bloom's taxonomy and that IE students performed better on these questions overall. USMLE step 1 exam scores were similar between groups. The finding of a significant difference in content complexity between discipline-specific and integrated exams adds to recent literature indicating that there are a number of potential biases related to curricular comparison studies that must be considered. Future investigation involving larger sample sizes and multiple disciplines should be performed to explore this matter further.

scholarly journals Correlation of Ophthalmology Residency Application Characteristics with Subsequent Performance in Residency

2021 ◽  
Vol 13 (02) ◽  
pp. e151-e157
Author(s):  
Brett M. Gudgel ◽  
Andrew T. Melson ◽  
Justin Dvorak ◽  
Kai Ding ◽  
R. Michael Siatkowski
Keyword(s):  
Medical School ◽  
Language Processing ◽  
Resident Performance ◽  
Subsequent Performance ◽  
University Of Oklahoma ◽  
Usmle Step 1 ◽  
Application Characteristics ◽  
Number Of Publications ◽  
Residency Application ◽  
Core Clerkship

Abstract Purpose Only from reviewing applications, it is difficult to identify which applicants will be successful ophthalmology residents. The change of the USMLE Step 1 scoring to “Pass/Fail” removes another quantitative metric. We aimed to identify application attributes correlated with successful residency performance. This study also used artificial intelligence (AI) to evaluate letters of recommendation (LOR), the Dean's letter (MSPE), and personal statement (PS). Design Retrospective analysis of application characteristics versus residency performance was conducted. Participants Residents who graduated from the Dean McGee Eye Institute/University of Oklahoma Ophthalmology residency from 2004 to 2019 were included in this study. Methods Thirty-four attributes were recorded from each application. Residents were subjectively ranked into tertiles and top and bottom deciles based on residency performance by faculty present during their training. The Ophthalmic Knowledge Assessment Program (OKAP) examination scores were used as an objective performance metric. Analysis was performed to identify associations between application attributes and tertile/decile ranking. Additional analysis used AI and natural language processing to evaluate applicant LORs, MSPE, and PS. Main Outcome Measures Characteristics from residency applications that correlate with resident performance were the primary outcome of this study. Results Fifty-five residents and 21 faculty members were included. A grade of “A” or “Honors” in the obstetrics/gynecology (OB/GYN) clerkship and the presence of a home ophthalmology department were associated with ranking in the top tertile but not the top decile. Mean core clerkship grades, medical school ranking in the top 25 U.S. News and World Report (USNWR) primary care rankings, and postgraduate year (PGY)-2 and PGY-3 OKAP scores were predictive of being ranked in both the top tertile and the top decile. USMLE scores, alpha-omega-alpha (AOA) status, and number of publications did not correlate with subjective resident performance. AI analysis of LORs, MSPE, and PS did not identify any text features that correlated with resident performance. Conclusions Many metrics traditionally felt to be predictive of residency success (USMLE scores, AOA status, and research) did not predict resident success in our study. We did confirm the importance of core clerkship grades and medical school ranking. Objective measures of success such as PGY-2 and PGY-3 OKAP scores were associated with high subjective ranking.

scholarly journals Early assessment with a virtual reality haptic simulator predicts performance in clinical practice

2019 ◽  
Vol 6 (5) ◽  
pp. 274-278
Author(s):  
Loulwa M Al-Saud ◽  
Faisal Mushtaq ◽  
Richard P Mann ◽  
Isra'a Mirghani ◽  
Ahmed Balkhoyor ◽  
...  
Keyword(s):  
Virtual Reality ◽  
Student Performance ◽  
Skill Acquisition ◽  
Clinical Training ◽  
Clinical Performance ◽  
Predictive Ability ◽  
Early Assessment ◽  
Medical Specialties ◽  
Data Driven Approach ◽  
The Relationship

BackgroundPrediction of clinical training aptitude in medicine and dentistry is largely driven by measures of a student’s intellectual capabilities. The measurement of sensorimotor ability has lagged behind, despite being a key constraint for safe and efficient practice in procedure-based medical specialties. Virtual reality (VR) haptic simulators, systems able to provide objective measures of sensorimotor performance, are beginning to establish their utility in facilitating sensorimotor skill acquisition, and it is possible that they may also inform the prediction of clinical performance.MethodsA retrospective cohort study examined the relationship between student performance on a haptic VR simulator in the second year of undergraduate dental study with subsequent clinic performance involving patients 2 years later. The predictive ability was tested against a phantom-head crown test (a traditional preclinical dental assessment, in the third year of study).ResultsVR scores averaged across the year explained 14% of variance in clinic performance, while the traditional test explained 5%. Students who scored highly on this averaged measure were ~10 times more likely to be high performers in the clinical crown test. Exploratory analysis indicated that single-trial VR scores did not correlate with real-world performance, but the relationship was statistically significant and strongest in the first half of the year and weakened over time.ConclusionsThe data demonstrate the potential of a VR haptic simulator to predict clinical performance and open up the possibility of taking a data-driven approach to identifying individuals who could benefit from support in the early stages of training.

scholarly journals Analysis of the 1990–2007 neurosurgery residency match: does applicant gender affect neurosurgery match outcome?

2018 ◽  
Vol 129 (2) ◽  
pp. 282-289 ◽  
Author(s):  
Susan R. Durham ◽  
Katelyn Donaldson ◽  
M. Sean Grady ◽  
Deborah L. Benzil
Keyword(s):  
Medical School ◽  
San Francisco ◽  
Female Gender ◽  
Application Process ◽  
Significant Difference ◽  
Successful Match ◽  
Usmle Step 1 ◽  
Usmle Step ◽  
Underrepresentation Of Women ◽  
Residency Application

OBJECTIVEWith nearly half of graduating US medical students being female, it is imperative to understand why females typically make up less than 20% of the neurosurgery applicant pool, a number that has changed very slowly over the past several decades. Organized neurosurgery has strongly indicated the desire to overcome the underrepresentation of women, and it is critical to explore whether females are at a disadvantage during the residency application process, one of the first steps in a neurosurgical career. To date, there are no published studies on specific applicant characteristics, including gender, that are associated with match outcome among neurosurgery resident applicants. The purpose of this study is to determine which characteristics of neurosurgery residency applicants, including gender, are associated with a successful match outcome.METHODSDe-identified neurosurgical resident applicant data obtained from the San Francisco Fellowship and Residency Matching Service for the years 1990–2007 were analyzed. Applicant characteristics including gender, medical school attended, year of application, United States Medical Licensing Exam (USMLE) Step 1 score, Alpha Omega Alpha (AOA) status, and match outcome were available for study.RESULTSOf the total 3426 applicants studied, 473 (13.8%) applicants were female and 2953 (86.2%) were male. Two thousand four hundred forty-eight (71.5%) applicants successfully matched. USMLE Step 1 score was the strongest predictor of match outcome with scores > 245 having an OR of 20.84 (95% CI 10.31–42.12) compared with those scoring < 215. The mean USMLE Step 1 score for applicants who successfully matched was 233.2 and was 210.8 for those applicants who did not match (p < 0.001). Medical school rank was also associated with match outcome (p < 0.001). AOA status was not significantly associated with match outcome. Female gender was associated with significantly lower odds of matching in both simple (OR 0.59, 95% CI 0.48–0.72) and multivariate analyses (OR 0.57, 95% CI 0.34–0.94 CI). USMLE Step 1 scores were significantly lower for females compared to males with a mean score of 230.1 for males and 221.5 for females (p < 0.001). There was no significant difference in medical school ranking or AOA status when stratified by applicant gender.CONCLUSIONSThe limited historical applicant data from 1990–2007 suggests that USMLE Step 1 score is the best predictor of match outcome, although applicant gender may also play a role.

scholarly journals The Effects of Pass/Fail USMLE Step 1 Scoring on the Otolaryngology Residency Application Process

2020 ◽  
Author(s):  
Khodayar Goshtasbi ◽  
Mehdi Abouzari ◽  
Tjoson Tjoa ◽  
Sonya Malekzadeh ◽  
Naveen D. Bhandarkar
Keyword(s):  
Application Process ◽  
Usmle Step 1 ◽  
Usmle Step ◽  
Residency Application

scholarly journals Ophthalmology Program Director Perspectives of Scoring Step 1 Pass/Fail

2020 ◽  
Vol 12 (02) ◽  
pp. e251-e254
Author(s):  
Saif A. Hamdan ◽  
Alan T. Makhoul ◽  
Brian C. Drolet ◽  
Jennifer L. Lindsey ◽  
Janice C. Law
Keyword(s):  
Well Being ◽  
The United States ◽  
Application Process ◽  
Medical Graduates ◽  
Medical Licensing ◽  
Usmle Step 1 ◽  
Licensing Examination ◽  
Usmle Step ◽  
Residency Application ◽  
The Impact

Abstract Background Scoring for the United States Medical Licensing Examination (USMLE) Step 1 was recently announced to be reported as binary as early as 2022. The general perception among program directors (PDs) in all specialties has largely been negative, but the perspective within ophthalmology remains uncharacterized. Objective This article characterizes ophthalmology residency PDs' perspectives regarding the impact of pass/fail USMLE Step 1 scoring on the residency application process. Methods A validated 19-item anonymous survey was electronically distributed to 111 PDs of Accreditation Council for Graduate Medical Education-accredited ophthalmology training programs. Results Fifty-six PDs (50.5%) completed the survey. The median age of respondents was 48 years and the majority were male (71.4%); the average tenure as PD was 7.1 years. Only 6 (10.7%) PDs reported the change of the USMLE Step 1 to pass/fail was a good idea. Most PDs (92.9%) indicated that this will make it more difficult to objectively compare applicants, and many (69.6%) did not agree that the change would improve medical student well-being. The majority (82.1%) indicated that there will be an increased emphasis on Step 2 Clinical Knowledge (CK) scores, and many (70.4%) felt that medical school reputation will be more important in application decisions. Conclusion Most ophthalmology PDs who responded to the survey do not support binary Step 1 scoring. Many raised concerns regarding shifted overemphasis on Step 2 CK, uncertain impact on student well-being, and potential to disadvantage certain groups of medical students including international medical graduates. These concerns highlight the need for reform in the ophthalmology application process.

scholarly journals Publication misrepresentation among neurosurgery residency applicants: an increasing problem

2016 ◽  
Vol 124 (1) ◽  
pp. 193-198 ◽  
Author(s):  
Heather M. Kistka ◽  
Arash Nayeri ◽  
Li Wang ◽  
Jamie Dow ◽  
Rameela Chandrasekhar ◽  
...  
Keyword(s):  
Negative Binomial ◽  
Demographic Data ◽  
Univariate Analysis ◽  
Negative Binomial Regression ◽  
Medical Schools ◽  
Scientific Integrity ◽  
Advanced Degrees ◽  
Application Service ◽  
Usmle Step 1 ◽  
Residency Application

OBJECT Misrepresentation of scholarly achievements is a recognized phenomenon, well documented in numerous fields, yet the accuracy of reporting remains dependent on the honor principle. Therefore, honest self-reporting is of paramount importance to maintain scientific integrity in neurosurgery. The authors had observed a trend toward increasing numbers of publications among applicants for neurosurgery residency at Vanderbilt University and undertook this study to determine whether this change was a result of increased academic productivity, inflated reporting, or both. They also aimed to identify application variables associated with inaccurate citations. METHODS The authors retrospectively reviewed the residency applications submitted to their neurosurgery department in 2006 (n = 148) and 2012 (n = 194). The applications from 2006 were made via SF Match and those from 2012 were made using the Electronic Residency Application Service. Publications reported as “accepted” or “in press” were verified via online search of Google Scholar, PubMed, journal websites, and direct journal contact. Works were considered misrepresented if they did not exist, incorrectly listed the applicant as first author, or were incorrectly listed as peer reviewed or published in a printed journal rather than an online only or non-peer-reviewed publication. Demographic data were collected, including applicant sex, medical school ranking and country, advanced degrees, Alpha Omega Alpha membership, and USMLE Step 1 score. Zero-inflated negative binomial regression was used to identify predictors of misrepresentation. RESULTS Using univariate analysis, between 2006 and 2012 the percentage of applicants reporting published works increased significantly (47% vs 97%, p < 0.001). However, the percentage of applicants with misrepresentations (33% vs 45%) also increased. In 2012, applicants with a greater total of reported works (p < 0.001) and applicants from unranked US medical schools (those not ranked by US News & World Report) were more likely to have erroneous citations (p = 0.038). CONCLUSIONS The incidence of legitimate and misrepresented scholarly works reported by applicants to the authors’ neurosurgery residency program increased during the past 6 years. Misrepresentation is more common in applicants from unranked US medical schools and those with a greater number of reported works on their application. This trend is concerning in a profession where trustworthiness is vital. To preserve integrity in the field, programs should consider verifying citations prior to submitting their rank lists.

Sign in / Sign up

Export Citation Format

Share Document