Standard setting in an objective structured clinical examination: use of global ratings of borderline performance to determine the passing score

Introduction: Clinical reasoning involves critical thinking and decision-making in clinical situations. It can be evaluated using Objective structured clinical examination (OSCE), which measures clinical skills associated with the development of clinical reasoning. Objective: To describe the implementation of an OSCE to evaluate the clinical skills associated with the development of clinical reasoning in physical therapy students, and to determine their level of satisfaction with this strategy. Materials and methods: Cross-sectional descriptive study carried out in 159 physiotherapy students from Universidad Andres Bello, Chile, enrolled in the Reasoning in Physical therapy course. The OSCE had 11 stations and a student satisfaction survey was administered. Data normality was determined using the Shapiro-Wilk test. Descriptive statistics (percentages, medians, and interquartile ranges (IQR)) were used for data analysis. Results: The median global score was 142 points (IQR:132-150) and 61.1% of the students obtained a passing score (≥ 134 points). Stations in which most students had a passing score were S3, S5 and S7 (with standardized patients): 78.62%, 96.85% and 85.53%, respectively. Regarding the satisfaction survey, 36.48% and 59.12% of the students agreed and strongly agreed with using tools that assess their clinical skills. Conclusions: The OSCE was successfully designed and implemented to evaluate the clinical skills associated with the the development of clinical reasoning in the study population, and most of them reported a high level of satisfaction with its use; this confirms OSCE is an excellent methodology to train and evaluate physical therapy students.

Download Full-text

The peritoneal dialysis orders objective structured clinical examination (OSCE): A formative assessment for nephrology fellows

Peritoneal Dialysis International ◽

10.1177/08968608211000542 ◽

2021 ◽

pp. 089686082110005

Author(s):

Lisa K Prince ◽

Brian C Y’Barbo ◽

Robert Nee ◽

and Christina M Yuan ◽

Keyword(s):

Peritoneal Dialysis ◽

Clinical Examination ◽

Item Difficulty ◽

Standard Of Care ◽

Objective Structured Clinical Examination ◽

Evidence Based ◽

Home Dialysis ◽

Bacterial Peritonitis ◽

Ultrafiltration Failure ◽

Passing Score

Background: Peritoneal dialysis (PD) management is a fundamental nephrology skill, especially with the recent emphasis on home dialysis. We report a prospective multicentre cohort study of a formative objective structured clinical examination (OSCE) assessing competence in managing PD-associated bacterial peritonitis, using the unified model of construct validity. Methods: The OSCE was developed by the principal investigators and reviewed by two subject matter experts. The test committee (eight nephrologists and one PD nurse) assessed test item difficulty/relevance and determined passing score. There were 22 test items (7 evidence-based/standard-of-care questions). Passing score was 16/22 (73%). No item had median relevance less than ‘important’, and all were easy to medium difficulty. Content validity index was 0.91. Preliminary validation (16 board-certified volunteers): mean score was 19 ± 2, with 94% (15/16) passing. Kappa = 0.85 [95% confidence interval (CI) 0.77–0.94]. Cronbach’s α = 0.70. Results: Eighty-seven fellows (16 programmes) were tested; 67% passed. Fellows scored significantly less than validators: 17 ± 3 versus 19 ± 2, p < 0.001 [95% CI 1.2–3.6]. Eighty-six per cent of evidence-based/standard-of-care questions were answered correctly by validators versus 54% by fellows; p < 0.001. Ninety-three per cent of fellows recognized that sufficient criteria were present to diagnose peritonitis, but only 17% correctly indicated all three. Seventy-seven per cent recognized peritonitis-associated ultrafiltration failure, but only 17% prescribed 21 days of antibiotic treatment for gram-negative peritonitis. Eighty-five per cent of fellows surveyed agreed/strongly agreed that the OSCE was useful in self-assessing proficiency. Second-year in-training examination and OSCE scores were positively correlated (Pearson’s r = 0.57, p < 0.00). Conclusions: The OSCE may be used to formatively assess fellow proficiency in managing PD-associated peritonitis.

Download Full-text