Evaluation of vignette-type examination items for testing medical physiology.

1993 ◽  
Vol 264 (6) ◽  
pp. S11 ◽  
Author(s):  
R G Carroll
Keyword(s):  
Item Analysis ◽  
Initial Step ◽  
The United States ◽  
National Board ◽  
Question Types ◽  
Clinical Vignettes ◽  
Medical Licensing ◽  
High Discrimination ◽  
Difficulty Index ◽  
Medical Examiners

Step 1 of the United States Medical Licensing Examination has now replaced the National Board of Medical Examiners part 1 examination as the initial step in medical licensure. The new examination format places an increased emphasis on clinical vignettes to test basic physiological concepts. Suggestions from a question-writing workshop were incorporated into cardiovascular items in a midcourse physiology examination. Item analysis indicated that the overall examination difficulty was not different from examinations administered in the preceding years. Within the examination, however, the vignette questions had significantly higher discrimination indexes than other question types. The slope of the difficulty-to-discrimination relationship was significantly higher in 1992 than in previous years. This means that questions of equivalent difficulty were more discriminating in 1992 than in previous years. Finally, this report suggests a method of identifying questions with both high discrimination and high difficulty index for the development of an (informal) test item bank. The clinical vignette question types are a valuable method of examining basic physiological principles.

scholarly journals Enhancing COMLEX-USA: Evidence-Based Redesign of the Osteopathic Medical Licensure Examination Program

2018 ◽  
Vol 104 (3) ◽  
pp. 11-18
Author(s):  
Dorothy T. Horber ◽  
John R. Gimpel
Keyword(s):  
The United States ◽  
The State ◽  
Evidence Based ◽  
Licensure Examination ◽  
National Board ◽  
Medical Licensing ◽  
Level 3 ◽  
Competency Based ◽  
Licensing Examination ◽  
Medical Examiners

ABSTRACT To ensure the Comprehensive Osteopathic Medical Licensing Examination of the United States of America (COMLEX-USA) remains relevant and current in meeting the needs of the state licensing boards and other constituents, the National Board of Osteopathic Medical Examiners (NBOME) has developed a new blueprint for an enhanced, competency-based examination program to be implemented with the COMLEX-USA Level 3 examination in late 2018. This article summarizes the evidence-based design processes on which the new blueprint is built, how it differs from the previous blueprint, and the evidence supporting its validity for the primary and intended purpose of COMLEX-USA — osteopathic physician licensure. It concludes with the changes being implemented by the NBOME to ensure COMLEX-USA remains current and meets the needs of its stakeholders, the state licensing boards.

scholarly journals Examining concurrent validity between COMLEX-USA Level 2-Cognitive Evaluation and COMLEX-USA Level 2-Performance Evaluation

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Brandon Craig ◽  
Xiaolin Wang ◽  
Jeanne Sandella ◽  
Tsung-Hsun Tsai ◽  
David Kuo ◽  
...  
Keyword(s):  
Performance Evaluation ◽  
Concurrent Validity ◽  
Pearson Correlation ◽  
The United States ◽  
National Board ◽  
Cognitive Evaluation ◽  
Competency Domain ◽  
Medical Examiners ◽  
Pearson Correlations ◽  
Level 2

Abstract Context The Comprehensive Osteopathic Medical Licensing Examination of the United States of America (COMLEX-USA) is a three level examination used as a pathway to licensure for students in osteopathic medical education programs. COMLEX-USA Level 2 includes a written assessment of Fundamental Clinical Sciences for Osteopathic Medical Practice (Level 2-Cognitive Evaluation [L2-CE]) delivered in a computer based format and separate performance evaluation (Level 2-Performance Evaluation [L2-PE]) administered through live encounters with standardized patients. L2-PE was designed to augment L2-CE. It is expected that the two examinations measure related yet distinct constructs. Objectives To explore the concurrent validity of L2-CE with L2-PE. Methods First attempt test scores were obtained from the National Board of Osteopathic Medical Examiners database for 6,639 candidates who took L2-CE between June 2019 and May 2020 and matched to the students’ L2-PE scores. The sample represented all colleges of osteopathic medicine and 97.5% of candidates who took L2-CE during the complete 2019–2020 test cycle. We calculated disattenuated correlations between the total score for L2-CE, the L2-CE scores for the seven competency domains (CD1 through CD7), and the L2-PE scores for the Humanistic Domain (HM) and Biomedical/Biomechanical Domain (BM). All scores were on continuous scales. Results Pearson correlations ranged from 0.10 to 0.88 and were all statically significant (p<0.01). L2-CE total score was most strongly correlated with CD2 (0.88) and CD3 (0.85). Pearson correlations between the L2-CE competency domain subscores ranged from 0.17 to 0.70, and correlations which included either HM or BM ranged from 0.10 to 0.34 with the strongest of those correlations being between BM and L2-CE total score (0.34) as well as between HM and BM (0.28).The largest increase between corresponding Pearson and disattenuated correlations was for pairs of scores with lower reliabilities such as CD5 and CD6, which had a Pearson correlation of 0.17 and a disattenuated correlation of 0.68. The smallest increase in correlations was observed in pairs of scores with larger reliabilities such as L2-CE total score and HM, which had a Pearson correlation of 0.23 and a disattenuated correlation of 0.28. The reliability of L2-CE was 0.87, 0.81 for HM, and 0.73 for BM. The reliabilities for the L2-CE competency domain scores ranged from 0.22 to 0.74. The small to moderate correlations between the L2-CE total score and the two L2-PE support the expectation that these examinations measure related but distinct constructs. The correlations between L2-PE and L2-CE competency domain subscores reflect the distribution of items defined by the L2-PE blueprint, providing evidence that the examinations are performing as designed. Conclusions This study provides evidence supporting the validity of the blueprints for constructing COMLEX-USA Levels 2-CE and 2-PE examinations in concert with the purpose and nature of the examinations.

scholarly journals Identifying Students at Risk of Failing the USMLE Step 2 Clinical Skills Examination

2019 ◽  
Vol 51 (6) ◽  
pp. 483-499
Author(s):  
Susan Rosenthal ◽  
Stefani Russo ◽  
Katherine Berg ◽  
Joseph Majdan ◽  
Jennifer Wilson ◽  
...  
Keyword(s):  
Student Performance ◽  
Performance Metrics ◽  
Clinical Skills ◽  
Students At Risk ◽  
National Board ◽  
Medical College ◽  
Subject Examination ◽  
Medical Licensing ◽  
Faculty Ratings ◽  
Medical Examiners

Background and Objectives: New standards announced in 2017 could increase the failure rate for Step 2 Clinical Skills (CS). The purpose of this study was to identify student performance metrics associated with risk of failing. Methods: Data for 1,041 graduates of one medical school from 2014 through 2017 were analyzed, including 30 (2.9%) failures. Metrics included Medical College Admission Test, United States Medical Licensing Examination Step 1, and clerkship National Board of Medical Examiners (NBME) Subject Examination scores; faculty ratings in six clerkships; and scores on an objective structured clinical examination (OSCE). Bivariate statistics and regression were used to estimate risk of failing. Results: Those failing had lower Step 1 scores, NBME scores, faculty ratings, and OSCE scores (P&lt;.02). Students with four or more low ratings were more likely to fail compared to those with fewer low ratings (relative risk [RR], 12.76, P&lt;.0001). Logistic regression revealed other risks: low surgery NBME scores (RR 3.75, P=.02), low pediatrics NBME scores (RR 3.67, P=.02), low ratings in internal medicine (RR 3.42, P=.004), and low OSCE Communication/Interpersonal Skills (RR 2.55, P=.02). Conclusions: Certain medical student performance metrics are associated with risk of failing Step 2 CS. It is important to clarify these and advise students accordingly.

Standards of an Examination (cut points), Construction of Scenarios, Measurement of lmpacts in the Decision Making Process

2018 ◽  
Vol 7 (9) ◽  
pp. 31-36
Author(s):  
Raja Subhiyah
Keyword(s):  
The United States ◽  
Standard Errors ◽  
Decision Making Process ◽  
National Board ◽  
Cut Points ◽  
The Us ◽  
Medical License ◽  
License Examination ◽  
Medical Examiners ◽  
The Impact

The National Board of Medical Examiners (NBME®) applies rigorous examinations to determine the impact on candidates taking the United States Medical License Examination (USMLE®) for the purpose of licensing physicians in the US. UU UU These standards apply to all levels of exam development, administration and qualification. The standards apply to the following processes:• Validity of the inferences of the articles: content: the content tested must be appropriate, asking the correct questions, the format of the element, or the design of the test and the design, or the process by which the test is performed, the documentation• Accuracy of the scores: reliability, focalization, information on the cutting edge, standard errors• Determination and application of cut points: methods, modified Angoff procedure, misclassification error.The main focus is on the last standard, although the first two levels are also briefly discussed. Different methods to establish an approval standard will be discussed and the method used for USMLE will be described. Errors of misclassification are also presented and how to minimize them.

Pass/Fail Scoring of USMLE Step 1 and the Need for Residency Selection Reform

2020 ◽  
pp. 019459982095116
Author(s):  
Parsa P. Salehi ◽  
Babak Azizzadeh ◽  
Yan Ho Lee
Keyword(s):  
Neck Surgery ◽  
The United States ◽  
Unintended Consequences ◽  
Application Process ◽  
National Board ◽  
Student Stress ◽  
Medical Graduates ◽  
Residency Selection ◽  
State Medical Boards ◽  
Medical Examiners

The Federation of State Medical Boards and the National Board of Medical Examiners recently announced a change in the United States Medical Licensing Examination Step 1 scoring convention to take effect, at the earliest, on January 1, 2022. There are many reasons for this change, including decreasing medical student stress and incentivizing students to learn freely without solely focusing on Step 1 performance. The question remains how this will affect the future of the otolaryngology–head and neck surgery match. By eradicating Step 1 grades, other factors, such as research, may garner increased importance in the application process. Such a shift may discriminate against students from less well-known medical schools, international medical graduates, and students from low socioeconomic backgrounds, who have fewer academic resources and access to research. Residency programs should try to anticipate such unintended consequences of the change and work on solutions heading into 2022.

scholarly journals Correlation of National Board of Medical Examiners Scores with United States Medical Licensing Examination Step 1 and Step 2 Scores

2012 ◽  
Vol 87 (10) ◽  
pp. 1348-1354 ◽  
Author(s):  
Christopher M. Zahn ◽  
Aaron Saguil ◽  
Anthony R. Artino ◽  
Ting Dong ◽  
Gerald Ming ◽  
...  
Keyword(s):  
United States ◽  
National Board ◽  
Medical Licensing ◽  
Licensing Examination ◽  
Medical Examiners
Sign in / Sign up

Export Citation Format

Share Document