An Objective Structured Clinical Examination for Medical Student Radiology Clerkships: Reproducibility Study

Background Objective structured clinical examinations (OSCEs) are a useful method to evaluate medical students’ performance in the clerkship years. OSCEs are designed to assess skills and knowledge in a standardized clinical setting and through use of a preset standard grading sheet, so that clinical knowledge can be evaluated at a high level and in a reproducible way. Objective This study aimed to present our OSCE assessment tool designed specifically for radiology clerkship medical students, which we called the objective structured radiology examination (OSRE), with the intent to advance the assessment of clerkship medical students by providing an objective, structured, reproducible, and low-cost method to evaluate medical students’ radiology knowledge and the reproducibility of this assessment tool. Methods We designed 9 different OSRE cases for radiology clerkship classes with participating third- and fourth-year medical students. Each examination comprises 1 to 3 images, a clinical scenario, and structured questions, along with a standardized scoring sheet that allows for an objective and low-cost assessment. Each medical student completed 3 of 9 random examination cases during their rotation. To evaluate for reproducibility of our scoring sheet assessment tool, we used 5 examiners to grade the same students. Reproducibility for each case and consistency for each grader were assessed with a two-way mixed effects intraclass correlation coefficient (ICC). An ICC below 0.4 was deemed poor to fair, an ICC of 0.41 to 0.60 was moderate, an ICC of 0.6 to 0.8 was substantial, and an ICC greater than 0.8 was almost perfect. We also assessed the correlation of scores and the students’ clinical experience with a linear regression model and compared mean grades between third- and fourth-year students. Results A total of 181 students (156 third- and 25 fourth-year students) were included in the study for a full academic year. Moreover, 6 of 9 cases demonstrated average ICCs more than 0.6 (substantial correlation), and the average ICCs ranged from 0.36 to 0.80 (P<.001 for all the cases). The average ICC for each grader was more than 0.60 (substantial correlation). The average grade among the third-year students was 11.9 (SD 4.9), compared with 12.8 (SD 5) among the fourth-year students (P=.005). There was no correlation between clinical experience and OSRE grade (−0.02; P=.48), adjusting for the medical school year. Conclusions Our OSRE is a reproducible assessment tool with most of our OSRE cases showing substantial correlation, except for 3 cases. No expertise in radiology is needed to grade these examinations using our scoring sheet. There was no correlation between scores and the clinical experience of the medical students tested.

Download Full-text

An Objective Structured Clinical Examination for Medical Student Radiology Clerkships: Reproducibility Study (Preprint)

10.2196/preprints.15444 ◽

2019 ◽

Author(s):

Pedro Vinícius Staziaki ◽

Rutuparna Sarangi ◽

Ujas N Parikh ◽

Jeffrey G Brooks ◽

Christina Alexandra LeBedis ◽

...

Keyword(s):

Medical Student ◽

Medical Students ◽

Clinical Experience ◽

Assessment Tool ◽

Low Cost ◽

Intraclass Correlation ◽

School Year ◽

Reproducibility Study ◽

Substantial Correlation ◽

High Level

BACKGROUND Objective structured clinical examinations (OSCEs) are a useful method to evaluate medical students’ performance in the clerkship years. OSCEs are designed to assess skills and knowledge in a standardized clinical setting and through use of a preset standard grading sheet, so that clinical knowledge can be evaluated at a high level and in a reproducible way. OBJECTIVE This study aimed to present our OSCE assessment tool designed specifically for radiology clerkship medical students, which we called the objective structured radiology examination (OSRE), with the intent to advance the assessment of clerkship medical students by providing an objective, structured, reproducible, and low-cost method to evaluate medical students’ radiology knowledge and the reproducibility of this assessment tool. METHODS We designed 9 different OSRE cases for radiology clerkship classes with participating third- and fourth-year medical students. Each examination comprises 1 to 3 images, a clinical scenario, and structured questions, along with a standardized scoring sheet that allows for an objective and low-cost assessment. Each medical student completed 3 of 9 random examination cases during their rotation. To evaluate for reproducibility of our scoring sheet assessment tool, we used 5 examiners to grade the same students. Reproducibility for each case and consistency for each grader were assessed with a two-way mixed effects intraclass correlation coefficient (ICC). An ICC below 0.4 was deemed poor to fair, an ICC of 0.41 to 0.60 was moderate, an ICC of 0.6 to 0.8 was substantial, and an ICC greater than 0.8 was almost perfect. We also assessed the correlation of scores and the students’ clinical experience with a linear regression model and compared mean grades between third- and fourth-year students. RESULTS A total of 181 students (156 third- and 25 fourth-year students) were included in the study for a full academic year. Moreover, 6 of 9 cases demonstrated average ICCs more than 0.6 (substantial correlation), and the average ICCs ranged from 0.36 to 0.80 (P<.001 for all the cases). The average ICC for each grader was more than 0.60 (substantial correlation). The average grade among the third-year students was 11.9 (SD 4.9), compared with 12.8 (SD 5) among the fourth-year students (P=.005). There was no correlation between clinical experience and OSRE grade (−0.02; P=.48), adjusting for the medical school year. CONCLUSIONS Our OSRE is a reproducible assessment tool with most of our OSRE cases showing substantial correlation, except for 3 cases. No expertise in radiology is needed to grade these examinations using our scoring sheet. There was no correlation between scores and the clinical experience of the medical students tested.

Download Full-text

Perfectionism is related with academic stress in medical student

European Psychiatry ◽

10.1016/j.eurpsy.2017.01.1207 ◽

2017 ◽

Vol 41 (S1) ◽

pp. S690-S690 ◽

Cited By ~ 1

Author(s):

C. Kim ◽

C. Seockhoon ◽

L. Suyeon ◽

Y. Soyoun ◽

P. Boram

Keyword(s):

Logistic Regression ◽

Regression Analysis ◽

Medical Student ◽

Medical Students ◽

Logistic Regression Analysis ◽

Stress Level ◽

Academic Stress ◽

Pearson’S Correlation ◽

Pearson's Correlation ◽

High Level

ObjectiveEspecially medical students set high academic standard, and try hard to achieve because they are requested to studying medicine perfectly. Multidimensional perfectionism scale (MPS) is used to measure perfectionism in medical student. The purpose of the present study was to investigate the relationships between stress and various factors including perfectionism.MethodsWe enrolled 159 medical students from University of Ulsan college of Medicine. We used the Medical Stress scale (MSS), the Multidimensional Perfectionism Scale(MPS), the patient health questionnaire-9(PHQ-9), the academic motivation scale (AMS), the insomnia severity scale(ISS), and The revised neo personality inventory.ResultTo analyze data, we used Pearson's correlation and logistic regression analysis. In the first step- Pearson's correlation analysis, a motivation, PHQ-9, ISI, perfectionism and neuroticism traits of NEO-PI significantly correlated with stress level (P < 0.001). In the second step, logistic regression analysis indicated that students who feel high level of academic stress (MSS score ≥ 28) are also related to a motivation, insomnia, perfectionism and neuroticism traits of NEO-PI (R2 = 0.389, P < 0.05).ConclusionStudent who tend to be perfect feel more academic stress. The high level of stress related to insomnia and a motivation in medical student. Moreover, personality trait also can influences their psychological stress level.Disclosure of interestThe authors have not supplied their declaration of competing interest.

Download Full-text

Understanding Student Experiences in a Near-Peer Resident Shadowing Program

Journal of Biomedical Education ◽

10.1155/2014/318598 ◽

2014 ◽

Vol 2014 ◽

pp. 1-5 ◽

Cited By ~ 1

Author(s):

Simon R. Turner ◽

Jonathan White ◽

Cheryl Poth ◽

W. Todd Rogers

Keyword(s):

Medical Student ◽

Medical Students ◽

Clinical Training ◽

Learning Experience ◽

Low Cost ◽

Student Experiences ◽

First Year ◽

Clinical Environment ◽

First Year Students ◽

Close Relationship

Introduction. The preparation of medical students for clerkship has been criticized, both in terms of students’ ability to understand their new role as clinical trainees and in their ability to carry out that role. To begin to address this gap, this paper reports the experiences of students in a shadowing program aimed at enhancing the preparedness of medical students for clinical training. The study examined a novel program, the Resident-Medical Student Shadowing Program, in which first-year medical students at the University of Alberta shadowed a first-year resident during clinical duties over the course of eight months. Methods. A study was conducted to assess the experiences of 83 first-year medical student participants who shadowed a first-year resident intermittently for one year. Student and resident participants’ experiences were explored using semistructured interviews. Results. Students and residents experiences indicate that participation increased students’ understanding of the clinical environment and their role within it and introduced them to skills and knowledge needed to perform that role. Students reported that a close relationship with their resident enhanced their learning experience. Conclusion. This study demonstrates that a low-cost program in which first-year students shadow residents may be a useful tool for helping prepare students for clerkship.

Download Full-text

Physical Activity (PA) of Pre-Clinical and Clinical Medical Students: Is Clinical Medical Student Less Physically Active than Pre-Clinical Medical Student?

Journal of the Medical Association of Thailand ◽

10.35755/jmedassocthai.2021.12.12584 ◽

2021 ◽

Vol 104 (12) ◽

pp. 1895-1901

Keyword(s):

Physical Activity ◽

Medical Student ◽

Medical Students ◽

Cross Sectional Study ◽

Online Questionnaire ◽

Cross Sectional ◽

Metabolic Equivalent ◽

Physically Active ◽

School Year ◽

Increased Risk

Background: An increase of sedentary behavior was observed in each year especially in clinical year. This accounts for a decrease in physical activities (PAs) of medical students, which could lead to reduced overall health and physical performance. Objective: To compare the levels of PA between pre-clinical and clinical medical students of Mae Fah Luang University (MFU) and to explored association between the levels of PA and school year. Materials and Methods: A cross-sectional study was conducted between May and August 2019. Online questionnaire, including personal profiles and global PA questionnaire, were distributed to all MFU medical students currently studying in the academic year of 2019. Results: One hundred fifty-six medical students participated in the present study. From the questionnaires, 67.74% of pre-clinical medical students (n=63) and 52.99% of clinical medical students (n=36) had moderate to high levels of PA. Pre-Clinical students had significantly higher levels of PA by median metabolic equivalent value (MET) at 1,908.73 versus 1,339.05 MET-minutes/week (p=0.03). The sixth-year medical students increased risk of lower PA than the first year about 8.34 times (p<0.01). Conclusion: One-third of the medical students reported as having low levels of PA. Clinical medical students had reduced levels of PA compared to pre-clinical medical students. Therefore, PA should be promoted to help increase the overall health of medical students. Keywords: Physical activity; Metabolic; Equivalent; Medical Student; Clinic; Pre-clinic

Download Full-text

Reliability of assessment of medical students’ non-technical skills using a behavioural marker system: does clinical experience matter?

BMJ Simulation and Technology Enhanced Learning ◽

10.1136/bmjstel-2020-000705 ◽

2020 ◽

pp. bmjstel-2020-000705

Author(s):

Benjamin Clarke ◽

Samantha E Smith ◽

Emma Claire Phillips ◽

Ailsa Hamilton ◽

Joanne Kerins ◽

...

Keyword(s):

Medical Students ◽

Clinical Experience ◽

Intraclass Correlation ◽

Correlation Coefficients ◽

Technical Skills ◽

Rater Reliability ◽

Single Measure ◽

Marker System ◽

Intraclass Correlation Coefficients ◽

Reliability Coefficients

IntroductionNon-technical skills are recognised to play an integral part in safe and effective patient care. Medi-StuNTS (Medical Students’ Non-Technical Skills) is a behavioural marker system developed to enable assessment of medical students’ non-technical skills. This study aimed to assess whether newly trained raters with high levels of clinical experience could achieve reliability coefficients of >0.7 and to compare differences in inter-rater reliability of raters with varying clinical experience.MethodsForty-four raters attended a workshop on Medi-StuNTS before independently rating three videos of medical students participating in immersive simulation scenarios. Data were grouped by raters’ levels of clinical experience. Inter-rater reliability was assessed by calculating intraclass correlation coefficients (ICC).ResultsEleven raters with more than 10 years of clinical experience achieved single-measure ICC of 0.37 and average-measures ICC of 0.87. Fourteen raters with more than or equal to 5 years and less than 10 years of clinical experience achieved single-measure ICC of 0.09 and average-measures ICC of 0.59. Nineteen raters with less than 5 years of clinical experience achieved single-measure ICC of 0.09 and average-measures ICC 0.65.ConclusionsUsing 11 newly trained raters with high levels of clinical experience produced highly reliable ratings that surpassed the prespecified inter-rater reliability standard; however, a single rater from this group would not achieve sufficiently reliable ratings. This is consistent with previous studies using other medical behavioural marker systems. This study demonstrated a decrease in inter-rater reliability of raters with lower levels of clinical experience, suggesting caution when using this population as raters for assessment of non-technical skills.

Download Full-text

A standardized low-cost peer role-playing training intervention improves medical student competency in communicating bad news to patients in Botswana

Palliative & Supportive Care ◽

10.1017/s1478951518000627 ◽

2018 ◽

Vol 17 (1) ◽

pp. 60-65 ◽

Cited By ~ 2

Author(s):

Katherine D. Westmoreland ◽

Francis M. Banda ◽

Andrew P. Steenhoff ◽

Elizabeth D. Lowenthal ◽

Erik Isaksson ◽

...

Keyword(s):

Medical Student ◽

Medical Students ◽

Low Cost ◽

Role Playing ◽

Bad News ◽

Breaking Bad News ◽

Multiple Choice Questions ◽

Educational Training ◽

Breaking Bad ◽

Delivering Bad News

AbstractObjectiveThe purpose of this study was to demonstrate effectiveness of an educational training workshop using role-playing to teach medical students in Botswana to deliver bad news.MethodA 3-hour small group workshop for University of Botswana medical students rotating at the Princess Marina Hospital in Gaborone was developed. The curriculum included an overview of communication basics and introduction of the validated (SPIKES) protocol for breaking bad news. Education strategies included didactic lecture, handouts, role-playing cases, and open forum discussion. Pre- and posttraining surveys assessed prior exposure and approach to breaking bad news using multiple-choice questions and perception of skill about breaking bad news using a 5-point Likert scale. An objective structured clinical examination (OSCE) with a standardized breaking bad news skills assessment was conducted; scores compared two medical student classes before and after the workshop was implemented.ResultForty-two medical students attended the workshop and 83% (35/42) completed the survey. Medical students reported exposure to delivering bad news on average 6.9 (SD = 13.7) times monthly, with 71% (25/35) having delivered bad news themselves without supervision. Self-perceived skill and confidence increased from 23% (8/35) to 86% (30/35) of those who reported feeling “good” or “very good” with their ability to break bad news after the workshop. Feedback after the workshop demonstrated that 100% found the SPIKES approach helpful and planned to use it in clinical practice, found role-playing helpful, and requested more sessions. Competency for delivering bad news increased from a mean score of 14/25 (56%, SD = 3.3) at baseline to 18/25 (72%, SD = 3.6) after the workshop (p = 0.0002).Significance of resultsThis workshop was effective in increasing medical student skill and confidence in delivering bad news. Standardized role-playing communication workshops integrated into medical school curricula could be a low-cost, effective, and easily implementable strategy to improve communication skills of doctors.

Download Full-text

High-Level Mobility Assessment Tool (HiMAT): Interrater Reliability, Retest Reliability, and Internal Consistency

Physical Therapy ◽

10.1093/ptj/86.3.395 ◽

2006 ◽

Vol 86 (3) ◽

pp. 395-400 ◽

Cited By ~ 51

Author(s):

Gavin P Williams ◽

Kenneth M Greenwood ◽

Val J Robertson ◽

Patricia A Goldie ◽

Meg E Morris

Keyword(s):

Internal Consistency ◽

Interrater Reliability ◽

Assessment Tool ◽

Physical Therapists ◽

Intraclass Correlation ◽

Systematic Change ◽

Retest Reliability ◽

Mobility Assessment ◽

High Level ◽

Very High

Abstract Background and Purpose. The High-Level Mobility Assessment Tool (HiMAT) assesses high-level mobility in people who have sustained a traumatic brain injury (TBI). The purpose of this study was to investigate the interrater reliability, retest reliability, and internal consistency of data obtained with the HiMAT. Subjects. Three physical therapists and 103 people with TBI were recruited from a rehabilitation hospital. Methods. Three physical therapists concurrently assessed a subset of 17 subjects with TBI to investigate interrater reliability. One physical therapist assessed a different subset of 20 subjects with TBI on 2 occasions, 2 days apart, to investigate retest reliability. Data from the entire sample of 103 subjects were used to investigate the internal consistency of this new scale. Results. Both the interrater reliability (intraclass correlation coefficient [ICC]=.99) and the retest reliability (ICC=.99) of the HiMAT data were very high. For retest reliability, a small systematic change was detected (t=3.82, df=19), indicating a marginal improvement of 1 point at retest. Internal consistency also was very high (Cronbach alpha=.97). Discussion and Conclusion. The HiMAT is a new tool specifically designed to measure high-level mobility, which currently is not a component of existing scales used in TBI. This study demonstrated that the HiMAT is a reliable tool for measuring high-level mobility. [Williams GP, Greenwood KM, Robertson VJ, et al. High-Level Mobility Assessment Tool (HiMAT): interrater reliability, retest reliability, and internal consistency. Phys Ther. 2006;86:395–400.]

Download Full-text

CODEM Instrument

GeroPsych ◽

10.1024/1662-9647/a000100 ◽

2014 ◽

Vol 27 (1) ◽

pp. 23-31 ◽

Cited By ~ 4

Author(s):

Anne Kuemmel (This author contributed eq ◽

Julia Haberstroh (This author contributed ◽

Johannes Pantel

Keyword(s):

Convergent Validity ◽

Interrater Reliability ◽

Discriminant Validity ◽

Assessment Tool ◽

Intraclass Correlation ◽

Well Being ◽

Communication Behavior ◽

People With Dementia ◽

Pearson's R ◽

Pearson’S R

Communication and communication behaviors in situational contexts are essential conditions for well-being and quality of life in people with dementia. Measuring methods, however, are limited. The CODEM instrument, a standardized observational communication behavior assessment tool, was developed and evaluated on the basis of the current state of research in dementia care and social-communicative behavior. Initially, interrater reliability was examined by means of videoratings (N = 10 people with dementia). Thereupon, six caregivers in six German nursing homes observed 69 residents suffering from dementia and used CODEM to rate their communication behavior. The interrater reliability of CODEM was excellent (mean κ = .79; intraclass correlation = .91). Statistical analysis indicated that CODEM had excellent internal consistency (Cronbach’s α = .95). CODEM also showed excellent convergent validity (Pearson’s R = .88) as well as discriminant validity (Pearson’s R = .63). Confirmatory factor analysis verified the two-factor solution of verbal/content aspects and nonverbal/relationship aspects. With regard to the severity of the disease, the content and relational aspects of communication exhibited different trends. CODEM proved to be a reliable, valid, and sensitive assessment tool for examining communication behavior in the field of dementia. CODEM also provides researchers a feasible examination tool for measuring effects of psychosocial intervention studies that strive to improve communication behavior and well-being in dementia.

Download Full-text

HiMAT: High-Level Mobility Assessment Tool

Critical Reviews in Physical and Rehabilitation Medicine ◽

10.1615/critrevphysrehabilmed.2018026190 ◽

2018 ◽

Vol 30 (2) ◽

pp. 97-99

Author(s):

Gretchen R. Pfost ◽

Brett Cook ◽

Chang Ha Doh ◽

Jerome Uthapan

Keyword(s):

Assessment Tool ◽

Mobility Assessment ◽

High Level

Download Full-text

Meditation buffers medical student compassion from the deleterious effects of depression.

10.31231/osf.io/jxdk8 ◽

2017 ◽

Author(s):

Jennifer S Mascaro ◽

Sean Kelley ◽

Alana Darcher ◽

Lobsang Negi ◽

Carol Worthman ◽

...

Keyword(s):

Medical Student ◽

Medical Students ◽

Medical School ◽

Well Being ◽

Daily Functioning ◽

Wait List ◽

School Students ◽

Social Emotions ◽

The Face ◽

Second Year

Increasing data suggest that for medical school students the stress of academic and psychologicaldemands can impair social emotions that are a core aspect of compassion and ultimately physiciancompetence. Few interventions have proven successful for enhancing physician compassion inways that persist in the face of suffering and that enable sustained caretaker well-being. To addressthis issue, the current study was designed to (1) investigate the feasibility of cognitively-basedcompassion training (CBCT) for second-year medical students, and (2) test whether CBCT decreasesdepression, enhances compassion, and improves daily functioning in medical students. Comparedto the wait-list group, students randomized to CBCT reported increased compassion, and decreasedloneliness and depression. Changes in compassion were most robust in individuals reporting highlevels of depression at baseline, suggesting that CBCT may benefit those most in need by breakingthe link between personal suffering and a concomitant drop in compassion

Download Full-text