Accuracy and usability of a diagnostic decision support system in the diagnosis of three representative rheumatic diseases: a randomized controlled trial among medical students

Abstract Background An increasing number of diagnostic decision support systems (DDSS) exist to support patients and physicians in establishing the correct diagnosis as early as possible. However, little evidence exists that supports the effectiveness of these DDSS. The objectives were to compare the diagnostic accuracy of medical students, with and without the use of a DDSS, and the diagnostic accuracy of the DDSS system itself, regarding the typical rheumatic diseases and to analyze the user experience. Methods A total of 102 medical students were openly recruited from a university hospital and randomized (unblinded) to a control group (CG) and an intervention group (IG) that used a DDSS (Ada – Your Health Guide) to create an ordered diagnostic hypotheses list for three rheumatic case vignettes. Diagnostic accuracy, measured as the presence of the correct diagnosis first or at all on the hypothesis list, was the main outcome measure and evaluated for CG, IG, and DDSS. Results The correct diagnosis was ranked first (or was present at all) in CG, IG, and DDSS in 37% (40%), 47% (55%), and 29% (43%) for the first case; 87% (94%), 84% (100%), and 51% (98%) in the second case; and 35% (59%), 20% (51%), and 4% (51%) in the third case, respectively. No significant benefit of using the DDDS could be observed. In a substantial number of situations, the mean probabilities reported by the DDSS for incorrect diagnoses were actually higher than for correct diagnoses, and students accepted false DDSS diagnostic suggestions. DDSS symptom entry greatly varied and was often incomplete or false. No significant correlation between the number of symptoms extracted and diagnostic accuracy was seen. It took on average 7 min longer to solve a case using the DDSS. In IG, 61% of students compared to 90% in CG stated that they could imagine using the DDSS in their future clinical work life. Conclusions The diagnostic accuracy of medical students was superior to the DDSS, and its usage did not significantly improve students’ diagnostic accuracy. DDSS usage was time-consuming and may be misleading due to prompting wrong diagnoses and probabilities. Trial registration DRKS.de, DRKS00024433. Retrospectively registered on February 5, 2021.

Download Full-text

The Effectiveness of Training on Auscultation of Heart with a Simulator of Cardiology in Medical Students

Open Access Macedonian Journal of Medical Sciences ◽

10.3889/oamjms.2021.7247 ◽

2021 ◽

Vol 9 (E) ◽

pp. 1055-1060

Author(s):

Gulbakit Koshmaganbetova ◽

Saulesh Kurmangalieva ◽

Yerlan Bazargaliyev ◽

Azhar Zhexenova ◽

Baktybergen Urekeshov ◽

...

Keyword(s):

Medical Students ◽

Diagnostic Accuracy ◽

Intervention Group ◽

Control Group ◽

Mitral Insufficiency ◽

Training Module ◽

Real Patient ◽

Fourth Year Medical Students ◽

Effectiveness Of Training ◽

Clinical Conditions

Abstract The purpose of this study was to determine whether the training module with a simulator of cardiology improves auscultation skills in medical students. Methods. Medical students of the third year after completing the module of the cardiovascular system of the discipline “Propaedeutics of internal diseases, passed a two-hour or four-hour training module in clinical auscultation with retesting immediately after the intervention and in the fourth year. The control group consisted of fourth-year medical students who had no intervention. Results. The diagnostic accuracy in two-hour training was 45.9% vs 35.3% in four-hour training p <.001. The use of a cardio simulator significantly increased the accurate detection of mitral regurgitation immediately after training on a simulator (more than 73%) p <.001. The next academic year, regression was observed in the diagnostic accuracy of mitral insufficiency in the intervention group after six months of observation by 4%. The auscultation skills of students at the bedside of real patients did not increase after training on a simulator: the accuracy of diagnosis of the auscultatory picture of the defect was equally low in the intervention group and the control group (35.0% vs 30.8%, p = 0.651). Conclusions. Two-hour training was more effective than four-hour training. After training on cardiac auscultation using a patient’s cardiological simulator, the accuracy rate was low in a situation close to the clinical conditions and a clinic on a real patient.

Download Full-text

The Flipped Classroom Is Effective for Medical Students to Improve Deep Tendon Reflex Examination Skills: A Mixed-Method Study

10.21203/rs.3.rs-139775/v1 ◽

2021 ◽

Author(s):

Shun Uchida ◽

Kiyoshi Shikino ◽

Kosuke Ishizuka ◽

Yosuke Yamauchi ◽

Yasutaka Yanagita ◽

...

Keyword(s):

Medical Students ◽

Flipped Classroom ◽

Deep Tendon Reflex ◽

Intervention Group ◽

General Medicine ◽

Likert Scale ◽

University Hospital ◽

Control Group ◽

Self Confidence ◽

Before And After

Abstract Background: Deep tendon reflexes (DTR) are a prerequisite skill in clinical clerkships. However, many medical students are not confident in their technique and need to be effectively trained. We evaluated the effectiveness of a flipped classroom for teaching DTR skills. Methods: We recruited 83 fifth-year medical students who participated in a clinical clerkship at the Department of General Medicine, Chiba University Hospital, from November 2018 to July 2019. They were allocated to the flipped classroom technique (intervention group, n=39) or the traditional technique instruction group (control group, n=44). Before procedural teaching, while the intervention group learned about DTR by e-learning, the control group did so face-to-face. A 5-point Likert scale was used to evaluate self-confidence in DTR examination before and after the procedural teaching (1=no confidence, 5=confidence). We evaluated the mastery of techniques after procedural teaching using the Direct Observation of Procedural Skills (DOPS). Unpaired t-test was used to analyze the results of the 5-point Likert scale and DOPS. We assessed self-confidence in DTR examination before and after procedural teaching using a free description questionnaire in the two groups. Additionally, in the intervention group, focus group interviews (FGI) (7 groups, n=39) were conducted to assess the effectiveness of the flipped classroom after procedural teaching.Results: Pre-test self-confidence in the DTR examination was significantly higher in the intervention group than in the control group (2.8 vs. 2.3, P=0.005). Post-test self-confidence in the DTR examination was not significantly different between the two groups (3.9 vs. 4.1, P=0.31), and so was mastery (4.3 vs. 4.1, P=0.68). The questionnaires before the procedural teaching revealed themes common to the two groups, including “lack of knowledge” and “lack of self-confidence.” Themes about prior learning, including “acquisition of knowledge” and “promoting understanding,” were specific in the intervention group. The FGI revealed themes including “application of knowledge,” “improvement in DTR technique,” and “increased self-confidence.” Conclusions: Teaching DTR skills to medical students in flipped classrooms improves readiness for learning and increases self-confidence in performing the procedure at a point before procedural teaching.

Download Full-text

Diagnostic accuracy in Family Medicine residents using a clinical decision support system (DXplain): a randomized-controlled trial

Diagnosis ◽

10.1515/dx-2017-0045 ◽

2018 ◽

Vol 5 (2) ◽

pp. 71-76 ◽

Cited By ~ 3

Author(s):

Adrian Israel Martinez-Franco ◽

Melchor Sanchez-Mendiola ◽

Juan Jose Mazon-Ramirez ◽

Isaias Hernandez-Torres ◽

Carlos Rivero-Lopez ◽

...

Keyword(s):

Decision Support ◽

Family Medicine ◽

Diagnostic Accuracy ◽

Clinical Decision Support ◽

Medical Practice ◽

Intervention Group ◽

Diagnostic Errors ◽

Clinical Decision ◽

Control Group ◽

Family Medicine Residents

Abstract Background: Clinical reasoning is an essential skill in physicians, required to address the challenges of accurate patient diagnoses. The goal of the study was to compare the diagnostic accuracy in Family Medicine residents, with and without the use of a clinical decision support tool (DXplain http://www.mghlcs.org/projects/dxplain). Methods: A total of 87 first-year Family Medicine residents, training at the National Autonomous University of Mexico (UNAM) Postgraduate Studies Division in Mexico City, participated voluntarily in the study. They were randomized to a control group and an intervention group that used DXplain. Both groups solved 30 clinical diagnosis cases (internal medicine, pediatrics, gynecology and emergency medicine) in a multiple-choice question test that had validity evidence. Results: The percent-correct score in the Diagnosis Test in the control group (44 residents) was 74.1±9.4 (mean±standard deviation) whereas the DXplain intervention group (43 residents) had a score of 82.4±8.5 (p<0.001). There were significant differences in the four knowledge content areas of the test. Conclusions: Family Medicine residents have appropriate diagnostic accuracy that can improve with the use of DXplain. This could help decrease diagnostic errors, improve patient safety and the quality of medical practice. The use of clinical decision support systems could be useful in educational interventions and medical practice.

Download Full-text

OP0147 RHEUMATIC? - A DIGITAL DIAGNOSTIC DECISION SUPPORT TOOL FOR INDIVIDUALS SUSPECTING RHEUMATIC DISEASES: A MULTICENTER VALIDATION STUDY

Annals of the Rheumatic Diseases ◽

10.1136/annrheumdis-2021-eular.1118 ◽

2021 ◽

Vol 80 (Suppl 1) ◽

pp. 87.1-88

Author(s):

R. Knevel ◽

J. Knitza ◽

A. Hensvold ◽

A. Circiumaru ◽

T. Bruce ◽

...

Keyword(s):

Health Care ◽

Decision Support ◽

Rheumatic Disease ◽

Rheumatic Diseases ◽

Receiver Operating Curve ◽

Musculoskeletal Complaints ◽

Immune Mediated ◽

Diagnostic Decision ◽

Ocean Observations ◽

Receiver Operating

Background:Digital diagnostic decision support tools promise to accelerate diagnosis and increase health care efficiency in rheumatology. Rheumatic? is an online tool developed by specialists in rheumatology and general medicine together with patients and patient organizations for individuals suspecting a rheumatic disease.1,2 The tool can be used by people suspicious for rheumatic diseases resulting in individual advise on eventually seeking further health care.Objectives:We tested Rheumatic? for its ability to differentiate symptoms from immune-mediated diseases from other rheumatic and musculoskeletal complaints and disorders in patients visiting rheumatology clinics.Methods:The performance of Rheumatic? was tested using data from 175 patients from three university rheumatology centers covering two different settings:A.Risk-RA phase setting. Here, we tested whether Rheumatic? could predict the development of arthritis in 50 at risk-individuals with musculoskeletal complaints and anti-citrullinated protein antibody positivity from the KI (Karolinska Institutet)B.Early arthritis setting. Here, we tested whether Rheumatic? could predict the development of an immune-mediated rheumatic disease in i) EUMC (Erlangen) n=52 patients and ii) LUMC (Leiden) n=73 patients.In each setting, we examined the discriminative power of the total score with the Wilcoxon rank test and the area-under-the-receiver-operating-characteristic curve (AUC-ROC).Results:In setting A, the total test score clearly differentiated between individuals developing arthritis or not, median 245 versus 163, P < 0.0001, AUC-ROC = 75.3 (Figure 1). Also within patients with arthritis the Rheumatic? total score was significantly higher in patients developing an immune-mediated arthritic disease versus those who did not: median score EUMC 191 versus 107, P < 0.0001, AUC-ROC = 79.0, and LUMC 262 versus 212, P < 0.0001, AUC-ROC = 53.6.Figure 1.(Area under) the receiver operating curve for the total Rheumatic? scoreConclusion:Rheumatic? is a web-based patient-centered multilingual diagnostic tool capable of differentiating immune-mediated rheumatic conditions from other musculoskeletal problems. A following subject of research is how the tool performs in a population-wide setting.References:[1]Knitza J. et al. Mobile Health in Rheumatology: A Patient Survey Study Exploring Usage, Preferences, Barriers and eHealth Literacy. JMIR mHealth and uHealth. 2020.[2]https://rheumatic.elsa.science/en/Acknowledgements:This project has received funding from EIT Health. EIT Health is supported by the European Institute of Innovation and Technology (EIT), a body of the European Union that receives support from the European Union’s Horizon 2020 Research and Innovation program.This project has received funding from the Innovative Medicines Initiative 2 Joint Undertaking under grant agreement No 777357, RTCure.Disclosure of Interests:Rachel Knevel: None declared, Johannes Knitza: None declared, Aase Hensvold: None declared, Alexandra Circiumaru: None declared, Tor Bruce Employee of: Ocean Observations, Sebastian Evans Employee of: Elsa Science, Tjardo Maarseveen: None declared, Marc Maurits: None declared, Liesbeth Beaart- van de Voorde: None declared, David Simon: None declared, Arnd Kleyer: None declared, Martina Johannesson: None declared, Georg Schett: None declared, Thomas Huizinga: None declared, Sofia Svanteson Employee of: Elsa Science, Alexandra Lindfors Employee of: Ocean Observations, Lars Klareskog: None declared, Anca Catrina: None declared

Download Full-text

Efficacy of implementing intermittent STOP THE BLEED® reviews on long term retention of hemorrhage control skills of first year medical students

Journal of Osteopathic Medicine ◽

10.1515/jom-2020-0231 ◽

2021 ◽

Vol 0 (0) ◽

Author(s):

Enkhtsogt Sainbayar ◽

Nathan Holt ◽

Amber Jacobson ◽

Shalini Bhatia ◽

Christina Weaver

Keyword(s):

Medical Students ◽

Intervention Group ◽

Training Session ◽

Likert Scale ◽

Control Group ◽

First Year ◽

Hemorrhage Control ◽

Term Retention ◽

Long Term Retention

Abstract Context Some medical schools integrate STOP THE BLEED® training into their curricula to teach students how to identify and stop life threatening bleeds; these classes that are taught as single day didactic and hands-on training sessions without posttraining reviews. To improve retention and confidence in hemorrhage control, additional review opportunities are necessary. Objectives To investigate whether intermittent STOP THE BLEED® reviews were effective for long term retention of hemorrhage control skills and improving perceived confidence. Methods First year osteopathic medical students were asked to complete an eight item survey (five Likert scale and three quiz format questions) before (pretraining) and after (posttraining) completing a STOP THE BLEED® training session. After the surveys were collected, students were randomly assigned to one of two study groups. Over a 12 week intervention period, each group watched a 4 min STOP THE BLEED® review video (intervention group) or a “distractor” video (control group) at 4 week intervals. After the 12 weeks, the students were asked to complete an 11 item survey. Results Scores on the posttraining survey were higher than the pretraining survey. The median score on the five Likert scale items was 23 points for the posttraining survey and 14 points for the pretraining survey. Two of the three knowledge based quiz format questions significantly improved from pretraining to posttraining (both p<0.001). On the 11 item postintervention survey, both groups performed similarly on the three quiz questions (all p>0.18), but the intervention group had much higher scores on the Likert scale items than the control group regarding their confidence in their ability to identify and control bleeding (intervention group median = 21.4 points vs. control group median = 16.8 points). Conclusions Intermittent review videos for STOP THE BLEED® training improved medical students’ confidence in their hemorrhage control skills, but the videos did not improve their ability to correctly answer quiz-format questions compared with the control group.

Download Full-text

Development and evaluation of an online medication safety module for medical students at a rural teaching hospital: the Winchester District Memorial Hospital

BMJ Open Quality ◽

10.1136/bmjoq-2021-001385 ◽

2021 ◽

Vol 10 (2) ◽

pp. e001385

Author(s):

Ali Elbeddini ◽

Yasamin Tayefehchamani

Keyword(s):

Online Learning ◽

Medical Students ◽

Medication Safety ◽

Controlled Trial ◽

Effective Means ◽

Intervention Group ◽

Control Group ◽

Open Label ◽

Online Module ◽

Learning Module

ObjectiveTo design, implement and assess an online learning module for third-year and fourth-year medical students addressing medication safety.DesignThis study was a prospective, parallel, open-label, randomised controlled trial with two arms: (1) a control arm in which students were given five articles to read about medication safety, and (2) an intervention arm in which students were given access to an interactive web-based learning module on medication safety. Pretesting and post-testing were done online to evaluate change in medication safety knowledge.ResultsTen students completed the study in the intervention group (online module) and six students completed the study in the control group. The increase in score obtained on the post-test, relative to the pretest, was 15.4% in the group who completed the online module and 2.0% in the control group (difference=13.4%, 95% CI 0.5% to 26.2%, p=0.04).ConclusionStudents who completed an online educational tool about medication safety demonstrated a significantly greater increase in knowledge than those who completed a few readings. Online learning modules can be a convenient and effective means of teaching safe prescribing concepts to medical trainees.

Download Full-text

Immunization competence retention in medical students: a comparation between conventional lectures and lectures with simulations method

Paediatrica Indonesiana ◽

10.14238/pi55.6.2015.339-44 ◽

2016 ◽

Vol 55 (6) ◽

pp. 339 ◽

Cited By ~ 1

Author(s):

Hartono Gunadi ◽

Rini Sekartini ◽

Retno Asti Werdhani ◽

Ardi Findyartini ◽

Muhammad Arvianda Kevin Kurnia

Keyword(s):

Experimental Study ◽

Medical Students ◽

Medical School ◽

Intervention Group ◽

Control Group ◽

Vaccine Preventable Diseases ◽

Before And After ◽

Quasi Experimental

Background Immunization is recognized as one of the strategiesto reduce vaccine preventable diseases. Competency related toimmunization are consequently important for medical students andthe medical school needs to assure the competence acquisition.Objective To assess competence related to immunization andits retention following lectures with simulations compared tolectures only.Methods A quasi-experimental study was conducted to the 5th yearstudents of University of Indonesia Medical School during the ChildAdolescent Health Module in 2012-2013. The intervention grouphad lectures with simulations and the control group had lecturesonly. Immunization knowledge was assessed with a 30 multiplechoice question (MCA) items performed before and after themodule. Competence retention was assessed by MCQ (knowledge)and OSCE (skills) 2-6 months afterwards.Results Sixty eight subjects for each group with similarcharacteristics were analyzed. There was significant differenceafter module MCQ score between two groups. Competenceretention in 2-6 months after module completion was betterin intervention group, both for the knowledge (median MCQscore of 70.00 (range 37-93) vs. mean score of 58.01 (SD 12.22),respectively; P<0.001) and skill (OSCE mean scores of 75.21 (SD10.74) vs. 62.62 (SD 11.89), respectively; P < 0.001). Proportionof subjects in the intervention group who passed both the MCQand OSCE were also significantly greater.Conclusion Lectures with simulations are proved to bemore effective in improving medical students’ immunizationcompetence as well as its retention compared to lectures onlyapproach.

Download Full-text

Assessment of the Effectiveness of a Computerised Decision-Support Tool for Health Professionals for the Prevention and Treatment of Childhood Obesity. Results from a Randomised Controlled Trial

Nutrients ◽

10.3390/nu11030706 ◽

2019 ◽

Vol 11 (3) ◽

pp. 706 ◽

Cited By ~ 4

Author(s):

George Moschonis ◽

Maria Michalopoulou ◽

Konstantina Tsoutsoulopoulou ◽

Elpis Vlachopapadopoulou ◽

Stefanos Michalacos ◽

...

Keyword(s):

Decision Support ◽

Childhood Obesity ◽

Randomised Controlled Trial ◽

Healthcare Professionals ◽

Controlled Trial ◽

Intervention Group ◽

Decision Support Tool ◽

Control Group ◽

Support Tool ◽

Randomised Controlled

We examined the effectiveness of a computerised decision-support tool (DST), designed for paediatric healthcare professionals, as a means to tackle childhood obesity. A randomised controlled trial was conducted with 65 families of 6–12-year old overweight or obese children. Paediatricians, paediatric endocrinologists and a dietitian in two children’s hospitals implemented the intervention. The intervention group (IG) received personalised meal plans and lifestyle optimisation recommendations via the DST, while families in the control group (CG) received general recommendations. After three months of intervention, the IG had a significant change in dietary fibre and sucrose intake by 4.1 and −4.6 g/day, respectively. In addition, the IG significantly reduced consumption of sweets (i.e., chocolates and cakes) and salty snacks (i.e., potato chips) by −0.1 and −0.3 portions/day, respectively. Furthermore, the CG had a significant increase of body weight and waist circumference by 1.4 kg and 2.1 cm, respectively, while Body Mass Index (BMI) decreased only in the IG by −0.4 kg/m2. However, the aforementioned findings did not differ significantly between study groups. In conclusion, these findings indicate the dynamics of the DST in supporting paediatric healthcare professionals to improve the effectiveness of care in modifying obesity-related behaviours. Further research is needed to confirm these findings.

Download Full-text

The effectiveness of training with an emergency department simulator on medical student performance in a simulated disaster

CJEM ◽

10.1017/s1481803500011982 ◽

2010 ◽

Vol 12 (01) ◽

pp. 27-32 ◽

Cited By ~ 23

Author(s):

Jeffrey Michael Franc-Law ◽

Pier Luigi Ingrassia ◽

Luca Ragazzoni ◽

Francesco Della Corte

Keyword(s):

Medical Students ◽

Student Performance ◽

Intervention Group ◽

Command And Control ◽

Disaster Medicine ◽

Simulated Patients ◽

Control Group ◽

Number Of Patients ◽

Simulation Based ◽

And Control

ABSTRACT Objective: Training in practical aspects of disaster medicine is often impossible, and simulation may offer an educational opportunity superior to traditional didactic methods. We sought to determine whether exposure to an electronic simulation tool would improve the ability of medical students to manage a simulated disaster. Methods: We stratified 22 students by year of education and randomly assigned 50% from each category to form the intervention group, with the remaining 50% forming the control group. Both groups received the same didactic training sessions. The intervention group received additional disaster medicine training on a patient simulator (disastermed.ca), and the control group spent equal time on the simulator in a nondisaster setting. We compared markers of patient flow during a simulated disaster, including mean differences in time and number of patients to reach triage, bed assignment, patient assessment and disposition. In addition, we compared triage accuracy and scores on a structured command-and-control instrument. We collected data on the students' evaluations of the course for secondary purposes. Results: Participants in the intervention group triaged their patients more quickly than participants in the control group (mean difference 43 s, 99.5% confidence interval [CI] 12 to 75 s). The score of performance indicators on a standardized scale was also significantly higher in the intervention group (18/18) when compared with the control group (8/18) (p < 0.001). All students indicated that they preferred the simulation-based curriculum to a lecture-based curriculum. When asked to rate the exercise overall, both groups gave a median score of 8 on a 10-point modified Likert scale. Conclusion: Participation in an electronic disaster simulation using the disastermed.ca software package appears to increase the speed at which medical students triage simulated patients and increase their score on a structured command-and-control performance indicator instrument. Participants indicated that the simulation-based curriculum in disaster medicine is preferable to a lecture-based curriculum. Overall student satisfaction with the simulation-based curriculum was high.

Download Full-text

The Effectiveness of Bioskills Training for Simulated Open Lumbar Laminectomy

Global Spine Journal ◽

10.1177/2192568217703337 ◽

2017 ◽

Vol 7 (8) ◽

pp. 794-800 ◽

Cited By ~ 8

Author(s):

Barrett S. Boody ◽

Brett D. Rosenthal ◽

Tyler J. Jenkins ◽

Alpesh A. Patel ◽

Jason W. Savage ◽

...

Keyword(s):

Medical Students ◽

Clinical Education ◽

Medical Center ◽

Academic Medical Center ◽

Intervention Group ◽

Training Session ◽

Senior Resident ◽

Control Group ◽

Junior Resident ◽

Training Module

Study Design: Randomized, prospective study within an orthopedic surgery resident program at a large urban academic medical center. Objectives: To develop an inexpensive, user-friendly, and reproducible lumbar laminectomy bioskills training module and evaluation protocol that can be readily implemented into residency training programs to augment the clinical education of orthopedic and neurosurgical physicians-in-training. Methods: Twenty participants comprising senior medical students and orthopedic surgical residents. Participants were randomized to control (n = 9) or intervention (n = 11) groups controlling for level of experience (medical students, junior resident, or senior resident). The intervention group underwent a 40-minute bioskills training module, while the control group spent the same time with self-directed study. Pre- and posttest performance was self-reported by each participant (Physician Performance Diagnostic Inventory Scale [PPDIS]). Objective outcome scores were obtained from a blinded fellowship-trained attending orthopedic spine surgeon using Objective Structured Assessment of Technical Skills (OSATS) and Objective Decompression Score metrics. Results: When compared with the control group, the intervention group yielded a significant mean improvement in OSATS ( P = .022) and PPDIS ( P = .0001) scores. The Objective Decompression Scores improved in the intervention group with a trend toward significance ( P = .058). Conclusions: We conclude that a concise lumbar laminectomy bioskills training session can be a useful educational tool for to augment clinical education. Although no direct clinical correlation can be concluded from this study, the improvement in trainee’s technical and procedural skills suggests that Sawbones training modules can be an efficient and effective tool for teaching fundamental spine surgical skills outside of the operating room.

Download Full-text