Progress testing and its effect on medical student motivation

Assessment has a pivotal role in medical education, as it can direct student learning. Motivation is an essential factor that determines learning assessment results. One of the aims of assessment is to determine the level of student knowledge, one of which can be measured by a progress test. This study aimed to determine the relationship between progress test and student motivation. This study used a cross-sectional observational analytic method. It was conducted in 2017 at Faculty of Medicine Universitas Sebelas Maret (FM UNS), Indonesia. The respondents consisted of 253 students from batch 2014, 2015, and 2016. The sample was selected by stratified random sampling. The instrument used to assess academic motivation was the academic motivation scale (AMS). The research data were analyzed using the Pearson correlation test. There is a significant positive correlation between progress test results and academic motivation (r=0.500; p=0.000). However, there is no significant difference in motivation level based on gender (p=0.889) and student cohort (p=0.533). In the progress test score, there are significant differences based on gender (p=0.014) and grade-point average (p=0.000). However, there is no significant difference in the progress test scores based on the student batch (p=0.212). The results support that progress test is useful assessment method to support medical student’s motivation.

Medical Students’ Perspectives on Online Proctoring During Remote Digital Progress Test

Remote teaching and assessment are essential for current education. During online examination, online proctoring is often used as a surveillance tool. Little is known about student perceptions on online proctoring. Using an online questionnaire, we found that medical students worry most about unjustified invalidation of their exam due to unstable internet connection, background noise or webcam issues, and privacy issues. It is important to be aware of these worries as they may influence test results.

Short Term Effect of the COVID-19 Pandemic On Progress Test Outcomes and Emotional State of Medical Students: A Quantitative and Qualitative Analysis

Background The COVID-19 pandemic has been the source of many challenges for medical students worldwide. The authors examined short-term effects on knowledge gain as well as shifts in learning behavior and study-related emotional states.Method The development of knowledge gain was measured comparing the outcomes of shared questions within Progress Test (PT) pairs. The authors used mixed-effect regression models and compared the absolute variations in the percentage of correct answers per subject. Three successive test pairs were analyzed in this manner: PT36-PT41 (both conducted before the pandemic), PT37-PT42 (PT37 took place before the pandemic; PT42 was conducted from April 2020 onwards) and PT38-PT43 (PT38 was administered before the pandemic; PT43 started in November 2020). A survey including closed and open-ended questions was also carried out in January 2021 with the purpose of assessing the learning behavior and emotional state of participants. Open-ended responses were analyzed using Latent Dirichlet Allocation. Results The most recent test of each PT-pair showed a higher mean score compared to the previous test in the same pair (PT36-PT41: 2.53 (95% CI: 1.31-3.75), PT37-PT42: 3.72 (2.57-4.88), PT38-PT43: 5.59 (4.37-6-81)). Analogously, an increase in the share of correct answers was observed for most medical disciplines, with Epidemiology showing the most remarkable upsurge.N=2,715 students from eleven different German-speaking faculties participated in the survey. Respondents were mostly positive towards online lectures, which were perceived as clearly beneficial, allowing for more time and flexibility. On the other hand, the suspension of practical lessons and alleged communicational and organizational shortcomings were seen as the main disadvantages. 28% of the students did not perceive negative impacts on their emotional state regarding their studies, however, 20% of the surveyed students found it difficult to cope with the lack of social contacts, with an additional 8% of them claiming to feel lonely, demotivated or abandoned.Conclusion: Overall, PT performance improved during the pandemic. Students see advantages in online lectures, but disadvantages in the cancellation of practical lectures; they miss their former social interactions and some even show signs of emotional distress.

Progress testing as a pattern of excellence for the assessment of medical students’ knowledge: concepts, history, and perspective

Progress test has been created with the necessity of an assessment method align with problem-based learning. Although it was specifically created to overcome the limitations of traditional assessment for problem-based learning, nowadays is used by different type of curricula. In this paper, we first present the basic assumptions, the history, benefit and challenges of the progress test. Progress test overcomes many limitations of traditional assessment, such as validity and reliability. However, the implementation of progress test is a logistical challenge. In addition, we discuss the limitation of progress test when used as summative assessment, which may not always be aligned with constructivist theory. When adding feedback and methods of analysis that considers multiple testing, progress test is then align with constructivist theory. Finally, the use of progress test’s sub scores may lack validity because of the low number of items; thus, pass/fail decision should not be based on the sub scores, but only on general scores.

Using Markov chain model to evaluate medical students’ trajectory on progress tests and predict USMLE step 1 scores---a retrospective cohort study in one medical school

Background Medical students must meet curricular expectations and pass national licensing examinations to become physicians. However, no previous studies explicitly modeled stages of medical students acquiring basic science knowledge. In this study, we employed an innovative statistical model to characterize students’ growth using progress testing results over time and predict licensing examination performance. Methods All students matriculated from 2016 to 2017 in our medical school with USMLE Step 1 test scores were included in this retrospective cohort study (N = 358). Markov chain method was employed to: 1) identify latent states of acquiring scientific knowledge based on progress tests and 2) estimate students’ transition probabilities between states. The primary outcome of this study, United States Medical Licensing Examination (USMLE) Step 1 performance, were predicted based on students’ estimated probabilities in each latent state identified by Markov chain model. Results Four latent states were identified based on students’ progress test results: Novice, Advanced Beginner I, Advanced Beginner II and Competent States. At the end of the first year, students predicted to remain in the Novice state had lower mean Step 1 scores compared to those in the Competent state (209, SD = 14.8 versus 255, SD = 10.8 respectively) and had more first attempt failures (11.5% versus 0%). On regression analysis, it is found that at the end of the first year, if there was 10% higher chance staying in Novice State, Step 1 scores will be predicted 2.0 points lower (95% CI: 0.85–2.81 with P < .01); while 10% higher chance in Competent State, Step 1scores will be predicted 4.3 points higher (95% CI: 2.92–5.19 with P < .01). Similar findings were also found at the end of second year medical school. Conclusions Using the Markov chain model to analyze longitudinal progress test performance offers a flexible and effective estimation method to identify students’ transitions across latent stages for acquiring scientific knowledge. The results can help identify students who are at-risk for licensing examination failure and may benefit from targeted academic support.

ANALYSIS OF PROGRESS TEST RESULTS IN MEDICAL FACULTY STUDENTS

Background: Assessment of learning outcomes is an important evaluation material to show how the teaching and learning process has been carried out. It can be obtained from formative and summative assessment, then students are given feedback from these results. One method for formative evaluation is a progress test. During the implementation at the HKBP Nommensen University Faculty of Medicine, there had never been an analysis of the results of the Progress Test activity. This study was conducted for analysing of the results of the Progress Test held in the even semester of the 2018/2019 academic year. Methods: This study used an observational descriptive design with cross sectional method. The sample were all students of the Faculty of Medicine who were actively studying in the even semester of the 2018/2019 Academic Year totaling 215 subjects. Item analysis was done on the questions for basic and clinical medicine category by the level of difficulty and the discrimination index based on students’ study period. Results: Students passing rate that attended the progress test in this study were very low. However, the score achieved by the students increased along with the length of students’ study period. Item analysis resulted the difficulty level majority at the medium level, and the discrimination index majority at the poor level for both the basic and clinical medicine science category. Conclusion: Progress testing can be used as a tool to help curriculum designer see the development of students’ knowledge skills both individually and in population.

Chirurgische Aufklärung: Klar geregelt durch das Patientenrechtegesetz – deutliche Unsicherheit bei Medizinstudierenden

Zusammenfassung Hintergrund Das chirurgische Aufklärungsgespräch stellt eine komplexe Herausforderung dar und ist als Lernziel im Nationalen Kompetenzbasierter Lernzielkatalog Medizin verankert. Die wenigen bestehenden Lehrformate sind uneinheitlich und aufwändig; insbesondere juristische Implikationen nehmen im Studium wenig Raum ein, obwohl sie mit Inkrafttreten des Patientenrechtegesetzes vermehrt in den Fokus gerückt sind und u. a. bei Regressverfahren eine wichtige Rolle spielen. Ziel der Arbeit Ziel war eine Ermittlung des Wissensstandes von Medizinstudierenden zu den rechtlichen Aspekten des chirurgischen Aufklärungsgespräches auf Basis einer juristischen Analyse des Patientenrechtegesetzes. Dieses wurde auf Implikationen für die Lehre im Medizinstudium überprüft. Material und Methoden Nach Analyse von Gesetz und Rechtsprechung wurden Multiple-Choice-Fragen zu den rechtlichen Aspekten des chirurgischen Aufklärungsgespräches erstellt und im Sinne einer Querschnittsanalyse im Progress Test Medizin platziert. Es erfolgte die deskriptive statistische Auswertung der Ergebnisse bei Berliner Medizinstudierenden. Ergebnisse Es wurden die Antworten von 2625 (Wintersemester 2018/19) und 2409 (Sommersemester 2019) Berliner Studierenden ausgewertet. Bei den Fragen zur Art aufzuklärender Prozeduren sowie der Bedenkzeit nahm die Anzahl Studierender, die die Frage korrekt beantwortete, über die Zeit zu, erreichte jedoch nicht den Vergleichswert aller Fragen des Progress Test Medizin. Bei den Fragen zu den notwendigen Inhalten wählten, unabhängig vom Ausbildungsstand, zwischen 30 und 60 % die korrekte Antwort, eine Zunahme korrekter Antworten über die Zeit war nicht zu sehen. Diskussion In der vorliegenden Arbeit konnte gezeigt werden, dass bei Medizinstudierenden über alle Semester hinweg Unsicherheiten bezüglich der juristischen Aspekte des Aufklärungsgespräches bestehen. Der gesetzliche Rahmen lässt allerdings Raum für neue Lehrformate wie der hier erstmals vorgestellten „Co-Aktion“: der Studierende führt die Aufklärung eines Patienten selbstständig, unter Aufsicht und Verantwortung des behandelnden Arztes, durch.