Family Medicine Clerkship Assessment Analysis: Application and Implications

Introduction: Clerkship assessment structures should consist of a systematic process that includes information from exam and assignment data to legitimize student grades and achievement. Analyzing student performance across assessments, rather than on a single assignment, provides a more accurate picture to identify academically at-risk students. This paper presents the development and implications of a structured approach to assessment analysis for the Family Medicine Clerkship at Florida International University Herbert Wertheim College of Medicine. Methods: The assessment analysis included a table presenting the distribution of all assessment performance results for 166 clerkship students from April 2018 to June 2019. A correlation table showed linear relationships between performance on all graded activities. We conducted a Pearson analysis (r), coefficient of determination (r2), multiple regression analysis, and reliability of performance analysis. Results: Performance on one assessment—the core skills quiz—yielded a statistically significant correlation (r=.409, r2= .16, P<.001) with the final clerkship grade. The reliability of performance analyses showed low performers (<-1.7 SD), had both a low mean quiz score (59.6) and final grade (83). Top performers (>-1.7 SD) had both a high mean quiz score (88.5) and final grade (99.6). This was confirmed by multiple regression analysis. Conclusion: The assessment analysis revealed a strong linear relationship between the core skills quiz and final grade; this relationship did not exist for other assignments. In response to the assessment analysis, the clerkship adjusted the grading weight of its assignments to reflect their utility in differentiating academic performance and implemented faculty development regarding grading for multiple assignments.

Eksplorasi Hasil Belajar Matematika melalui Penerapan Sistem Pembelajaran Daring (SIPEDAR) di Masa Pandemi Covid-19

This study aims to describe student learning outcomes through the application of the online learning system (SIPEDAR) during the COVID-19 pandemic and to find out what features are used in learning through SIPEDAR such as assignment (assignment), attendance (presence), bigbluebottonBN, choice, databases, external tools, feedback, forums, glossaries, quizzes, chats, quizzes, lessons, scrom packages, surveys, workshops, books, files, folders, IMS content packages, labels, pages, and URLs. This type of research is qualitative research. The subjects in this study were third semester students who took Mathematics at Elementary School Class High in the elementary school teacher education study program at Universitas Sarjanawiyata Tamansiswa. The instruments used were in the form of multiple choice questions, true false statements, and descriptions. The results of the study were the average quiz score and the average assignment score in class 3A, namely 83.27 and 72.90. The average quiz score and the average assignment score in the 3F class were 83.56 and 70.67. Student learning outcomes in both classes are included in the good category so that learning using SIPEDAR is carried out well and can be modified with other e-learning platforms.

Does the MMPI-2 College Maladjustment Scales Affect the Mini-Quiz Score of Students in the Medical Study Program of Universitas Muhammadiyah Yogyakarta

A student, especially a new student, will make adjustments to their environment in their first year of college, from high school to higher education. During the adjustment process, freshman experience various kinds of changes in the Higher Education environment. These changes both academically and non-academically. This research is a quantitative analytic observational research with cross sectional approach. The population used in this study were new students of the 2018 class of the Medical Study Program at Universitas Muhammadiyah Yogyakarta. There were 150 new student batch 2018 as population of the study. Based on the results and discussion, it can be concluded that there is no significant relationship between the MMPI-2 College Maladjustment Scales (Mt) and the mini-quiz score. It is suggested to have further research with more mini-quiz scores in blocks, research on factors that affect learning outcomes, and involve other aspects of learning outcomes, not only mini-quiz scores but also final block evaluation scores, tutorial scores and practicum scores.

Teaching During Covid: The Effectiveness of the HyFlex Classroom in a 300 Level Statistics Class

This study compares a 300 level statistics class taught using a HyFlex classroom to the same class taught one year earlier in the traditional, face to face method. While one quiz score was better for the face to face class, the HyFlex class did better than the traditional class on one quiz and final the examination. Every other point of comparison showed no difference between the classes.

A93 VOLUME OF EMR EXPOSURE IN TRAINING IS CORRELATED WITH POLYPECTOMY COGNITIVE COMPETENCE AMONGST RECENT GASTROENTEROLOGY GRADUATES

Background Competence in performing polypectomy is increasingly appreciated as a colonoscopy quality metric, as incomplete resection can lead to post-colonoscopy colorectal cancer, particularly for polyps removed using piecemeal endoscopic mucosal resection (EMR). The relationship between training experiences and cognitive competence in polypectomy has not been previously described. Aims We aimed to examine associations between training and assessment experiences, self-reported comfort, and cognitive competence in polypectomy amongst recent graduates of Canadian gastroenterology training programs. Methods An online survey was distributed to recent GI graduates (≤5 years in independent practice). The survey comprised 4 sections: (1) demographics; (2) training and assessment experiences in colonoscopy, polypectomy, and EMR; (3) self-reported comfort in performing aspects of polypectomy outlined in the Direct Observation of Polypectomy Skills Assessment Tool; and (4) performance on a 22-item multiple choice quiz intended to assess cognitive competence in polypectomy (items and correct responses to which were determined a priori based on agreement of two experts). Data was analyzed using descriptive statistics and associations between predictors (demographics, training/assessment experiences, self-reported comfort) and outcomes (quiz score) were assessed using multiple linear regression. Results There were 28 survey respondents, comprising 13 (46%) who trained in advanced endoscopy, 5 (18%) in hepatology, 2 (7%) in motility, 1 (4%) in IBD, 1 (4%) in nutrition, and 6 (21%) with no advanced training. This cohort had a mean (SD) duration in independent practice of 29.0 (18.4) months. Their mean (SD) annual volume of colonoscopy, polypectomy, and EMR in independent practice was 530 (221), 182 (76), 28 (16), respectively and they had completed 525 (203) colonoscopies, 146 (92) polypectomies, and 23 (20) EMRs in their prior training. Their mean (SD) quiz score was 71.9% (13.2%). ANOVA revealed significant score differences based on fellowship history, with those trained in advanced endoscopy achieving the highest scores (81.1%, P=0.01). Multiple linear regression revealed that the number of EMRs completed during training was significantly correlated with quiz performance (B=0.60, P=0.03). Conclusions EMR experience during training appears to be associated with cognitive competence in polypectomy in independent practice. These results suggest increasing exposure to EMR in training may improve polypectomy quality amongst practicing endoscopists. Funding Agencies CAG

Design and Evaluation of Scholarship and Research Orientation Sessions in Academic Pharmacy Practice Experiences

Objective: Evaluate the efficacy of educating academic Advance Pharmacy Practice Experiential (APPE) students on scholarship and research (S&R). We hypothesized that academic APPE students gain more knowledge in scholarship and research in comparison to non-academic APPE students. Methods: As a component of academic APPE rotation, all academic APPE students assigned by the office of experiential education per year (the intervention group) (n = 11) attended S&R sessions during the first week of their academic APPE rotation. Student volunteers from the same graduating class were recruited as the control group (n = 12). However, 1 student from the intervention group and 2 students from the control group were excluded from the analysis, leaving 10 students from each group. These sessions were taken in small groups of 1 to 3 once in their assigned rotation. S&R sessions included many topics that were based on a S&R manual. Upon completion of the sessions, a hard copy of a 10-question knowledge-based quiz was given to both groups at the same time. APPE students’ knowledge about S&R were assessed via grading the quiz. The anonymity of the quiz was maintained by requesting students only to mention whether they are from intervention group or controlled group. Results: While control group averaged 49% ± 4.58 on the quiz, the intervention group averaged 71.5% ± 5.77. Statistical analysis was performed using SPSS software. Based on independent sample t-test, students who took the S&R session show significant increase in their quiz score. Conclusion: Compared to the control group, S&R sessions helped academic APPE students to gain knowledge on S&R. The students also perceived that S&R sessions increased their interest in S&R.

Use of a CPD Plan Template with SMART Goals as Part of a Diabetes Pharmacotherapy Module

Objective: To evaluate the effectiveness of a Continuing Professional Development (CPD) plan template used in Fall of 2017 on quality of SMART goal development and student quiz scores. Innovation: The gap in time from when pharmacology is taught and when it is applied has contributed to poor student retention and performance in the diabetes pharmacotherapy course. To address this gap, the diabetes pharmacotherapy learning sequence was redesigned and included a self-assessment (pre-test), and the completion of a “CPD plan template”, which involved writing 1-3 SMART goals for each question missed on the pre-test. Following sequence completion, students took an identical post-quiz. Pre- and post- quiz scores were compared. Quality of CPD plan SMART goals was evaluated. Key Findings: The CPD plan template was completed by 98% of students. The majority, 62.5% of students, wrote SMART goals at the intermediate or good level, while 37.5% were evaluated as needs improvement. The average pre-quiz score was 7.4 points and average post-quiz score was 17.1 points with an average improvement of 9.8 points (p<0.0001). There was a statistically significant improvement for top 25% post-quiz scoring students who wrote “good” SMART goals compared to those who wrote goals needing improvement (p= 0.002). For students scoring in the lowest 25%, students with goals needing improvement scored higher than those with intermediate quality goals (p< 0.04). Next Steps: It may be beneficial to introduce CPD to students sooner, as well as teach students more intentionally how to create and use SMART goals to improve learning. Finally, instructor follow up with students regarding use of their plan during a learning sequence may have additional benefit. Article Type: Note

Can an Online Educational Module Improve Medical Trainee Confidence and Knowledge of Coagulation?

Background: Coagulation has notoriously been a topic that medical trainees find challenging to learn. A lack of understanding around coagulation has led to widespread inappropriate ordering of commonly used coagulation tests, including the prothrombin time (PT) and the activated partial thromboplastin time (aPTT). Despite these tests being validated for specific clinical indications, they are frequently ordered as screening tests in unselected patients, and often ordered together, suggesting a gap in physician understanding of coagulation and appropriate testing. To explore this further, we conducted a root-cause-analysis survey of 10 medical trainees. 70% of the surveyed trainees did not feel comfortable with their knowledge regarding coagulation and the appropriate use of coagulation tests. Trainees attributed their suboptimal knowledge to the manner in which coagulation is taught in training programs. Furthermore, they identified a scarcity of practical resources on coagulation and expressed interest in a web-accessible resource. Methods: We created an educational module on coagulation testing for trainees, available online at www.coagtesting.com. This module was created with the intent of simplifying the teaching of coagulation, with a focus on emphasizing clinically-relevant concepts. The module was evaluated at the University of Toronto with 50 participating medical trainees (11 medical students, 39 internal medicine residents [14 PGY1, 15 PGY2, 10 PGY3 residents]). Participation in our study included completing a validated knowledge pre-quiz on coagulation, completion of the educational module, and then the post-quiz following the module. To assess longer term knowledge retention, participants were asked to repeat the knowledge quiz three months following their initial participation. Our educational intervention was evaluated according to the Kirkpatrick Model, a framework for learning evaluation, with educational outcomes organized into four ranked levels (level 1, reaction; level 2, learning; level 3, behaviour; level 4, results). The primary objective of this study was to determine if the module improves trainee knowledge of coagulation, as indicated by their quiz results (level 2, learning). The secondary objective was to evaluate if the module has an influence on trainee ordering practices as assessed by a follow-up survey (level 3, behaviour). Results: The median pre-module quiz score was 67% (range 24% - 86%) with an increase of 24% to a median post-module quiz score of 91% (range 64% - 100%). Notably, in the pre-module quiz, 94% of trainees overestimated the sensitivity and specificity of the PT and aPTT in detecting a bleeding disorder, and 44% of trainees underestimated the cost of a PT test. 80% of trainees described increased confidence regarding their knowledge of coagulation and the use of coagulation tests following completion of the module. In addition, we have demonstrated sustained knowledge acquisition with a 3-month post-quiz median score of 89% (n=15, range 67%-100%). 100% of trainees who completed the 3-month follow-up survey (n=15) felt that the educational module had a positive influence on their practice and 87% of trainees were more likely to consider the sensitivity, specificity, and cost of a lab test prior to ordering it. In the seven months since the module was launched, it has been completed by over 2,000 unique visitors worldwide, with use in Canada, the United States, the United Kingdom, Australia, France, and Saudi Arabia according to data from our website host. Furthermore, several visitors to the website have re-visited the module multiple times. Conclusion: We have successfully demonstrated a significant increase in trainee knowledge and confidence regarding coagulation and appropriate use of coagulation tests with our educational intervention. Using the expertise of medical educators and incorporating feedback from trainees, we have employed a novel approach to the teaching of coagulation to maximize its approachability and clinical relevance. Our module also incorporates education on the cost of coagulation testing and appropriate use of these tests thus in line with the tenets of Choosing Wisely. The degree to which trainees have been utilizing and re-referring to our educational module worldwide emphasizes the need for this resource and its importance in bridging a large gap in medical training. Disclosures Sholzberg: Novartis: Honoraria; Amgen: Honoraria, Research Funding.

PENERAPAN METODE DISKUSI UNTUK MENINGKATKAN KOGNITIF SISWA KELAS VII SMPN 7 KUNTODARUSALAM

This study aims to determine the application of discussion methods to improve cognitive grade VII students SPMN 7 Kuntodalamalam, Research data retrieval carried out from January to February 2017 in Class VII 7 Kuntodarussalam Middle School Academic Year 2017/2018. The research subjects were 32 students consisting of 10 female students and 22 male students. Data collection includes: the value of discussion, Quiz, PR and Block Exams. Using descriptive analysis. The results obtained from 6 indicators in the discussion process in cycle I and sikulus II were highest in indicator 1 (attention to discussion material) with an average of 97.97% in very good category. in the first cycle of indicator 5 (Respecting the suggestions and opinions of fellow discussion participants) with an average of 61.09% (category is very lacking), in the second cycle the lowest was on indicator 6 (Concluding material) with an average of 64.13% ( category is very lacking). The average value of the discussion of meeting students from the first cycle and the second cycle the lowest value at the first meeting with a value of 76.35 (lack of category) and a value of 82.06 (sufficient category), the highest in the second cycle of meeting 2 with a value of 79.53 (sufficient category), while in the second cycle the highest at meeting III with a value of 82.06 (sufficient category). the average quiz score in the first cycle and the second cycle the lowest Quiz 3 with a value of 80.8 (sufficient category), the highest on quiz 6 with a value of 84.4 (good category). the value of homework assignments for students is the lowest in the first cycle with a value of 88.7 and the second cycle the value of 91.3 is very good. Nilau UB obtained from the first cycle is the lowest with the highest value of 81.9 and Cycle II with a value of 84.2 with a good category. It can be concluded that the application of the discussion method can improve the cognitive grade VII of Kuntodarusalam 7th Middle School.Keywords: metode diskusi,kognitif

The Hand Surgeon Consultation Improves Patient Knowledge in a Hand Surgery Mission to Honduras

Background: The purpose of this study was to assess impact of the surgeon consultation and informed consent process on patient education in an international hand surgery mission compared with a US academic hand surgery practice. These two groups were selected to evaluate communication difficulties in a surgical mission setting compared with standard of care in a high-income country. Methods: A multi-part survey was administered to patients presenting to a hand surgery mission during March 2012 and new patients of a university hand center in a 3-month period during 2011. Surveys were administered prior to and following surgeon consultation with one fellowship-trained hand surgeon. The survey assessed knowledge of basic hand anatomy, physiology, disease, individual diagnosis, and surgical risks. Results: 71 patients participated in the study (university n=36, mission n=35). Pre-consultation quiz score averaged 58% in the university group versus 27% in the mission group. Post-consultation quiz scores averaged 62% in the university group versus 40% in the mission group. Only the mission group’s quiz score increase was statistically significant. 93% of the university group reported learning about their condition and diagnosis, but only 40% demonstrated correct insight into their diagnosis. In the mission group, 73% reported learning about their condition and diagnosis while 53% demonstrated correct insight into their diagnosis. Although all consultations involved discussion of surgical risks, only 62% of the university group and 52% of the mission group recalled discussing surgical risks. Conclusions: The hand surgeon consultation was more effective in improving hand knowledge in the surgery mission group compared to in a university hand practice. This suggests that the surgeon consultation should be pursued despite communication barriers in surgical missions. However, the discrepancy between patient perception of knowledge gains and correct insight into diagnosis, and the deficit of patient retention of surgical risks need to be improved.