Medical Simulation as a Competency-Based Assessment within Physician Assistant Education

Simulation-based practices are widely utilized in medical education and are known to be a safe and effective way to train and assess learners, improve provider confidence and competency, and improve patient safety. Competency-based initiatives are being more broadly utilized to assess learner proficiency in health professions education. Recent publication of competencies expected of new graduate physician assistants, and updated accreditation requirements which include assessment of learner competencies in non-knowledge based domains, have led to the creation of this simulation-based summative assessment of learner competency in communication and patient care skills for Physician Assistant students. The purpose of this quantitative study was to identify if this simulation assessment had appropriate construct validity and rater consistency, and to identify if correlation existed between learner performance on the simulation exam and in required Supervised Clinical Training Experiences for measures of communication skills and patient care skills. While raters for the simulation assessment had minimal variability, measures of internal consistency did not achieve suitable thresholds for patient care skills. Communication skills assessment was able to achieve the minimum suitable threshold for internal consistency with minor revisions. No correlation was noted between exam performance for communication skills or patient care skills and clinical practice ratings. Several key areas exist which may explain these results including the rating scale for the simulation exam which utilized checklists and not global rating scales, faculty raters with broad and diverse clinical backgrounds, observation-related factors on the part of the student, and the high-complexity and multidimensional nature of provider-patient interactions.