Previous attempts to predict resident clinical performance based solely on measures of cognitive skills have been uniformly unsuccessful. For the past 8 years, a formative residency evaluation system has been used that includes yearly comprehensive oral in-training examinations (OITEs) assessing each resident's performance in the three areas of professional competence: cognitive, psychomotor, and affective. The results of these examinations and scores received on the written in-training examination (WITE) given by the American Board of Pediatrics were compared with faculty ratings received during the subsequent year of residency. No significant correlation was found at any level of training between WITE scores and clinical performance. Analysis based on clinical setting did not improve these results. Oral intraining examination scores, however, were highly correlated with clinical performance ratings. In addition, with oral in-training examination scores, the "problem" interns—those whose clinical performance rating placed them in the lower 10% of interns—were predicted with a high degree of significance, sensitivity, and specificity. Predictions based on WITEs were not significant. Simultaneous evaluation of all three areas of professional competence should be done when predictions of resident performance are attempted. The OITE is a powerful formative evaluation tool, providing valuable learning experiences as residents are objectively assessed while they perform patient-centered tasks that represent critical skills in the practice of medicine. It allows early detection and possible amelioration of future problems in the clinical performance of a resident.
The OSCE as a formative evaluation tool for substance abuse teaching