scholarly journals A Document Analysis of Nationally Available Faculty Assessment Forms of Resident Performance

2021 ◽  
Vol 13 (6) ◽  
pp. 833-840
Author(s):  
Judith C. French ◽  
Lily C. Pien

ABSTRACT Background Written feedback by faculty of resident performance is valuable when it includes components based on assessment for learning. However, it is not clear how often assessment forms include these components for summative and formative feedback. Objective To analyze prompts used in forms for faculty assessment of resident performance, guided by best practices in survey research methodology, self-regulation theory, and competency-based assessment. Methods A document analysis, which is a qualitative approach used to analyze content and structure of texts, was completed on assessment forms nationally available in MedHub. Due to the number of forms available, only internal medicine and surgery specialties were included. A document summary form was created to analyze the assessments. The summary form guided researchers through the analysis. Results Forty-eight forms were reviewed, each from a unique residency program. All forms provided a textbox for comments, and 54% made this textbox required for assessment completion. Eighty-three percent of assessments placed the open textbox at the end of the form. One-third of forms contained a simple prompt, “Comments,” for the narrative section. Fifteen percent of forms included a box to check if the information on the form had been discussed with the resident. Fifty percent of the assessments were unclear if they were meant to be formative or summative in nature. Conclusions Our document analysis of assessment forms revealed they do not always follow best practices in survey design for narrative sections, nor do they universally address elements deemed important for promotion of self-regulation and competency-based assessment.

2020 ◽  
Vol 34 (3) ◽  
pp. 153-167
Author(s):  
John R. Lauck ◽  
Stephen J. Perreault ◽  
Joseph R. Rakestraw ◽  
James S. Wainberg

SYNOPSIS Auditing standards require external auditors to inquire of client-employees regarding their knowledge of actual or suspected fraud (PCAOB 2010b; AICPA 2016). However, the extant literature provides little guidance on practical methods that auditors can employ to increase the likelihood of fraud disclosure and improve audit quality. Drawing upon best practices in the whistleblowing literature and psychological theories on self-regulation, we experimentally test the efficacy of two practical strategies that auditors can employ during the fraud inquiry process: actively promoting statutory whistleblower protections and strategically timing their fraud inquiries. Our results indicate that auditors are more likely to elicit client-employee fraud disclosures by actively promoting statutory whistleblower protections and strategically timing the fraud inquiry to take place in the afternoon, when client-employee self-regulation is more likely to be depleted. These two audit inquiry strategies should be of considerable interest to audit practitioners, audit committees, and those concerned with improving audit quality. Data Availability: From the authors by request.


2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Severin Pinilla ◽  
Alexandra Kyrou ◽  
Stefan Klöppel ◽  
Werner Strik ◽  
Christoph Nissen ◽  
...  

Abstract Background Entrustable professional activities (EPAs) in competency-based, undergraduate medical education (UME) have led to new formative workplace-based assessments (WBA) using entrustment-supervision scales in clerkships. We conducted an observational, prospective cohort study to explore the usefulness of a WBA designed to assess core EPAs in a psychiatry clerkship. Methods We analyzed changes in self-entrustment ratings of students and the supervisors’ ratings per EPA. Timing and frequencies of learner-initiated WBAs based on a prospective entrustment-supervision scale and resultant narrative feedback were analyzed quantitatively and qualitatively. Predictors for indirect supervision levels were explored via regression analysis, and narrative feedback was coded using thematic content analysis. Students evaluated the WBA after each clerkship rotation. Results EPA 1 (“Take a patient’s history”), EPA 2 (“Assess physical & mental status”) and EPA 8 (“Document & present a clinical encounter”) were most frequently used for learner-initiated WBAs throughout the clerkship rotations in a sample of 83 students. Clinical residents signed off on the majority of the WBAs (71%). EPAs 1, 2, and 8 showed the largest increases in self-entrustment and received most of the indirect supervision level ratings. We found a moderate, positive correlation between self-entrusted supervision levels at the end of the clerkship and the number of documented entrustment-supervision ratings per EPA (p < 0.0001). The number of entrustment ratings explained 6.5% of the variance in the supervisors’ ratings for EPA 1. Narrative feedback was documented for 79% (n = 214) of the WBAs. Most narratives addressed the Medical Expert role (77%, n = 208) and used reinforcement (59%, n = 161) as a feedback strategy. Students perceived the feedback as beneficial. Conclusions Using formative WBAs with an entrustment-supervision scale and prompts for written feedback facilitated targeted, high-quality feedback and effectively supported students’ development toward self-entrusted, indirect supervision levels.


Author(s):  
Silvana Chambers ◽  
Kim Nimon ◽  
Paula Anthony-McMann

This paper presents best practices for conducting survey research using Amazon Mechanical Turk (MTurk). Readers will learn the benefits, limitations, and trade-offs of using MTurk as compared to other recruitment services, including SurveyMonkey and Qualtrics. A synthesis of survey design guidelines along with a sample survey are presented to help researchers collect the best quality data. Techniques, including SPSS and R syntax, are provided that demonstrate how users can clean resulting data and identify valid responses for which workers could be paid.


Sign in / Sign up

Export Citation Format

Share Document