scholarly journals Improving medical residents’ self-assessment of their diagnostic accuracy: does feedback help?

Author(s):  
Josepha Kuhn ◽  
Pieter van den Berg ◽  
Silvia Mamede ◽  
Laura Zwaan ◽  
Patrick Bindels ◽  
...  

AbstractWhen physicians do not estimate their diagnostic accuracy correctly, i.e. show inaccurate diagnostic calibration, diagnostic errors or overtesting can occur. A previous study showed that physicians’ diagnostic calibration for easy cases improved, after they received feedback on their previous diagnoses. We investigated whether diagnostic calibration would also improve from this feedback when cases were more difficult. Sixty-nine general-practice residents were randomly assigned to one of two conditions. In the feedback condition, they diagnosed a case, rated their confidence in their diagnosis, their invested mental effort, and case complexity, and then were shown the correct diagnosis (feedback). This was repeated for 12 cases. Participants in the control condition did the same without receiving feedback. We analysed calibration in terms of (1) absolute accuracy (absolute difference between diagnostic accuracy and confidence), and (2) bias (confidence minus diagnostic calibration). There was no difference between the conditions in the measurements of calibration (absolute accuracy, p = .204; bias, p = .176). Post-hoc analyses showed that on correctly diagnosed cases (on which participants are either accurate or underconfident), calibration in the feedback condition was less accurate than in the control condition, p = .013. This study shows that feedback on diagnostic performance did not improve physicians’ calibration for more difficult cases. One explanation could be that participants were confronted with their mistakes and thereafter lowered their confidence ratings even if cases were diagnosed correctly. This shows how difficult it is to improve diagnostic calibration, which is important to prevent diagnostic errors or maltreatment.

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
J. Staal ◽  
J. Alsma ◽  
S. Mamede ◽  
A. P. J. Olson ◽  
G. Prins-van Gilst ◽  
...  

Abstract Background Diagnostic errors have been attributed to cognitive biases (reasoning shortcuts), which are thought to result from fast reasoning. Suggested solutions include slowing down the reasoning process. However, slower reasoning is not necessarily more accurate than faster reasoning. In this study, we studied the relationship between time to diagnose and diagnostic accuracy. Methods We conducted a multi-center within-subjects experiment where we prospectively induced availability bias (using Mamede et al.’s methodology) in 117 internal medicine residents. Subsequently, residents diagnosed cases that resembled those bias cases but had another correct diagnosis. We determined whether residents were correct, incorrect due to bias (i.e. they provided the diagnosis induced by availability bias) or due to other causes (i.e. they provided another incorrect diagnosis) and compared time to diagnose. Results We did not successfully induce bias: no significant effect of availability bias was found. Therefore, we compared correct diagnoses to all incorrect diagnoses. Residents reached correct diagnoses faster than incorrect diagnoses (115 s vs. 129 s, p < .001). Exploratory analyses of cases where bias was induced showed a trend of time to diagnose for bias diagnoses to be more similar to correct diagnoses (115 s vs 115 s, p = .971) than to other errors (115 s vs 136 s, p = .082). Conclusions We showed that correct diagnoses were made faster than incorrect diagnoses, even within subjects. Errors due to availability bias may be different: exploratory analyses suggest a trend that biased cases were diagnosed faster than incorrect diagnoses. The hypothesis that fast reasoning leads to diagnostic errors should be revisited, but more research into the characteristics of cognitive biases is important because they may be different from other causes of diagnostic errors.


2020 ◽  
Vol 3 (1) ◽  
pp. 21-27
Author(s):  
Pulikottil Wilson Vinny ◽  
Madakasira Vasantha Padma ◽  
P. N. Sylaja ◽  
Praveen Kesav ◽  
Vivek Lal ◽  
...  

Background: Diagnostic errors in neurological diagnosis are a source of preventable harm. Software tools like Differential Diagnosis (DDx) apps in neurology that hold the potential to mitigate this harm are conspicuously lacking. Materials and Methods: A multicenter cross-sectional observational study was designed to compare the diagnostic accuracy of a Neurology DDx App (Neurology Dx) with neurology residents by using vascular neurology clinical vignettes. The study was conducted at 7 leading neurology institutes in India. Study participants comprised of 100 neurology residents from the participating institutes. Measurements: Detecting diagnostic accuracy of residents and App measured as a proportion of correctly identified high likely gold standard DDx was prespecified as the main outcome. Proportions of correctly identified first high likely, first 3 high likely, first 5 high likely, and combined moderate plus high likely gold standard differentials by residents and App were secondary outcomes. Results: 1,000 vignettes were attempted by residents. Frequency of gold standard, high likely differentials correctly identified by residents was 27% compared to 72% by App (absolute difference 45%, 95% CI 35.7-52.8). When high and moderate likely differentials were combined, residents scored 17% compared to 57% by App (absolute difference 40%, 95% CI 33.8-50.0). Residents correctly identified first high likely gold standard differential as their first high likely differential in 34% compared to 18% by App (absolute difference 16%, 95% CI 1.2-25.4). Conclusion: App with predefined knowledge base can complement clinical reasoning of neurology residents. Portability and functionality of such Apps may further strengthen this symbiosis between humans and algorithms (CTRI/2017/06/008838).


2015 ◽  
Vol 36 (8) ◽  
pp. 949-956 ◽  
Author(s):  
Gregory A. Filice ◽  
Dimitri M. Drekonja ◽  
Joseph R. Thurn ◽  
Galen M. Hamann ◽  
Bobbie T. Masoud ◽  
...  

OBJECTIVEWe found previously that inappropriate inpatient antimicrobial use was often attributable to erroneous diagnoses. Here, we detail diagnostic errors and their relationship to inappropriate antimicrobial courses.DESIGNRetrospective cohort studySETTINGVeterans Affairs hospitalPATIENTSA cohort of 500 randomly selected inpatients with an antimicrobial courseMETHODSBlinded reviewers judged the accuracy of the initial provider diagnosis for the condition that led to an antimicrobial course and whether the course was appropriate.RESULTSThe diagnoses were correct in 291 cases (58%), incorrect in 156 cases (31%), and of indeterminate accuracy in 22 cases (4%). In the remaining 31 cases (6%), the diagnosis was a sign or symptom rather than a syndrome or disease. The odds ratio of a correct diagnosis was 4.3 (95% confidence interval [CI], 2.2–8.5) if the index condition was related to the reason for admission. When the diagnosis was correct, 181 of 292 courses (62%) were appropriate, compared with only 10 of 208 (5%) when the diagnosis was incorrect or indeterminate or when providers were treating a sign or symptom rather than a syndrome or disease (P<.001). Among the 309 cases in which antimicrobial courses were not appropriate, reasons varied by diagnostic accuracy; in 81 of 111 cases (73%) with a correct diagnosis, incorrect antimicrobial(s) were selected; in 166 of 198 other cases (84%), antimicrobial therapy was not indicated.CONCLUSIONSDiagnostic accuracy is important for optimal inpatient antimicrobial use. Antimicrobial stewardship strategies should help providers avoid diagnostic errors and know when antimicrobial therapy can be withheld safely.Infect Control Hosp Epidemiol 2015;36(8):949–956


2016 ◽  
Vol 12 (1) ◽  
pp. 13-24 ◽  
Author(s):  
Katie Ekberg ◽  
Markus Reuber

There are many areas in medicine in which the diagnosis poses significant difficulties and depends essentially on the clinician’s ability to take and interpret the patient’s history. The differential diagnosis of transient loss of consciousness (TLOC) is one such example, in particular the distinction between epilepsy and ‘psychogenic’ non-epileptic seizures (NES) is often difficult. A correct diagnosis is crucial because it determines the choice of treatment. Diagnosis is typically reliant on patients’ (and witnesses’) descriptions; however, conventional methods of history-taking focusing on the factual content of these descriptions are associated with relatively high rates of diagnostic errors. The use of linguistic methods (particularly conversation analysis) in research settings has demonstrated that these approaches can provide hints likely to be useful in the differentiation of epileptic and non-epileptic seizures. This paper explores to what extent (and under which conditions) the findings of these previous studies could be transposed from a research into a routine clinical setting.


2020 ◽  
Vol 79 (Suppl 1) ◽  
pp. 1961.1-1961
Author(s):  
J. Knitza ◽  
J. Mohn ◽  
C. Bergmann ◽  
E. Kampylafka ◽  
M. Hagen ◽  
...  

Background:Symptom checkers (SC) promise to reduce diagnostic delay, misdiagnosis and effectively guide patients through healthcare systems. They are increasingly used, however little evidence exists about their real-life effectiveness.Objectives:The aim of this study was to evaluate the diagnostic accuracy, usage time, usability and perceived usefulness of two promising SC, ADA (www.ada.com) and Rheport (www.rheport.de). Furthermore, symptom duration and previous symptom checking was recorded.Methods:Cross-sectional interim clinical data from the first of three recruiting centers from the prospective, real-world, multicenter bETTeR-study (DKRS DRKS00017642) was used. Patients newly presenting to a secondary rheumatology outpatient clinic between September and December 2019 completed the ADA and Rheport SC. The time and answers were recorded and compared to the patient’s actual diagnosis. ADA provides up to 5 disease suggestions, Rheport calculates a risk score for rheumatic musculoskeletal diseases (RMDs) (≥1=RMD). For both SC the sensitivity, specificity was calculated regarding RMDs. Furthermore, patients completed a survey evaluating the SC usability using the system usability scale (SUS), perceived usefulness, previous symptom checking and symptom duration.Results:Of the 129 consecutive patients approached, 97 agreed to participate. 38% (37/97) of the presenting patients presented with an RMD (Figure 1). Mean symptom duration was 146 weeks and a mean number of 10 physician contacts occurred previously, to evaluate current symptoms. 56% (54/96) had previously checked their symptoms on the internet using search engines, spending a mean of 6 hours. Rheport showed a sensitivity of 49% (18/37) and specificity of 58% (35/60) concerning RMDs. ADA’s top 1 and top 5 disease suggestions concerning RMD showed a sensitivity of 43% (16/37) and 54% (20/37) and a specificity of 58% (35/60) and 52% (31/60), respectively. ADA listed the correct diagnosis of the patients with RMDs first or within the first 5 disease suggestions in 19% (7/37) and 30% (11/37), respectively. The average perceived usefulness for checking symptoms using ADA, internet search engines and Rheport was 3.0, 3.5 and 3.1 on a visual analog scale from 1-5 (5=very useful). 61% (59/96) and 64% (61/96) would recommend using ADA and Rheport, respectively. The mean SUS score of ADA and Rheport was 72/100 and 73/100. The mean usage time for ADA and Rheport was 8 and 9 minutes, respectively.Conclusion:This is the first prospective, real-world, multicenter study evaluating the diagnostic accuracy and other features of two currently used SC in rheumatology. These interim results suggest that diagnostic accuracy is limited, however SC are well accepted among patients and in some cases, correct diagnosis can be provided out of the pocket within few minutes, saving valuable time.Figure:Acknowledgments:This study was supported by an unrestricted research grant from Novartis.Disclosure of Interests:Johannes Knitza Grant/research support from: Research Grant: Novartis, Jacob Mohn: None declared, Christina Bergmann: None declared, Eleni Kampylafka Speakers bureau: Novartis, BMS, Janssen, Melanie Hagen: None declared, Daniela Bohr: None declared, Elizabeth Araujo Speakers bureau: Novartis, Lilly, Abbott, Matthias Englbrecht Grant/research support from: Roche Pharma, Chugai Pharma Europe, Consultant of: AbbVie, Roche Pharma, RheumaDatenRhePort GbR, Speakers bureau: AbbVie, Celgene, Chugai Pharma Europe, Lilly, Mundipharma, Novartis, Pfizer, Roche Pharma, UCB, David Simon Grant/research support from: Else Kröner-Memorial Scholarship, Novartis, Consultant of: Novartis, Lilly, Arnd Kleyer Consultant of: Lilly, Gilead, Novartis,Abbvie, Speakers bureau: Novartis, Lilly, Timo Meinderink: None declared, Wolfgang Vorbrüggen: None declared, Cay-Benedict von der Decken: None declared, Stefan Kleinert Shareholder of: Morphosys, Grant/research support from: Novartis, Consultant of: Novartis, Speakers bureau: Abbvie, Novartis, Celgene, Roche, Chugai, Janssen, Andreas Ramming Grant/research support from: Pfizer, Novartis, Consultant of: Boehringer Ingelheim, Novartis, Gilead, Pfizer, Speakers bureau: Boehringer Ingelheim, Roche, Janssen, Jörg Distler Grant/research support from: Boehringer Ingelheim, Consultant of: Boehringer Ingelheim, Paid instructor for: Boehringer Ingelheim, Speakers bureau: Boehringer Ingelheim, Peter Bartz-Bazzanella: None declared, Georg Schett Speakers bureau: AbbVie, BMS, Celgene, Janssen, Eli Lilly, Novartis, Roche and UCB, Axel Hueber Grant/research support from: Novartis, Lilly, Pfizer, Consultant of: Abbvie, BMS, Celgene, Gilead, GSK, Lilly, Novartis, Speakers bureau: GSK, Lilly, Novartis, Martin Welcker Grant/research support from: Abbvie, Novartis, UCB, Hexal, BMS, Lilly, Roche, Celgene, Sanofi, Consultant of: Abbvie, Actelion, Aescu, Amgen, Celgene, Hexal, Janssen, Medac, Novartis, Pfizer, Sanofi, UCB, Speakers bureau: Abbvie, Aescu, Amgen, Biogen, Berlin Chemie, Celgene, GSK, Hexal, Mylan, Novartis, Pfizer, UCB


2019 ◽  
pp. jramc-2018-001132
Author(s):  
Pierre Perrier ◽  
J Leyral ◽  
O Thabouillot ◽  
D Papeix ◽  
G Comat ◽  
...  

IntroductionTo evaluate the usefulness of point-of-care ultrasound (POCUS) performed by young military medicine residents after short training in the diagnosis of medical emergencies.MethodsA prospective study was performed in the emergency department of a French army teaching hospital. Two young military medicine residents received ultrasound training focused on gall bladder, kidneys and lower limb veins. After clinical examination, they assigned a ‘clinicaldiagnostic probability’ (CP) on a visual analogue scale from 0 (definitely not diagnosis) to 10 (definitive diagnosis). The same student performed ultrasound examination and assigned an ‘ultrasounddiagnostic probability’ (UP) in the same way. The absolute difference between CP and UP was calculated. This result corresponded to the Ultrasound Diagnostic Index (UDI), which was positive if UP was closer to the final diagnosis than CP (POCUS improved the diagnostic accuracy), and negative conversely (POCUS decreased the diagnostic accuracy).ResultsForty-eight patients were included and 48 ultrasound examinations were performed. The present pathologies were found in 14 patients (29%). The mean UDI value was +3 (0–5). UDI was positive in 35 exams (73%), zero in 12 exams (25%) and negative in only one exam (2%).ConclusionPOCUS performed after clinical examination increases the diagnostic accuracy of young military medicine residents.


2018 ◽  
Vol 88 (3) ◽  
Author(s):  
Sergio C. Conte ◽  
Giulia Spagnol ◽  
Marco Confalonieri ◽  
Beatrice Brizi

The sedation plays an important role in the endobronchial ultrasound transbronchial needle aspiration (EBUS-TBNA) procedure. The sedation can be Minimal (anxiolysis), Moderate (conscious sedation) or Deep. The ACCP guidelines suggest that moderate or deep sedation (DS) is an acceptable approach. In fact, several studies compare moderate versus deep sedation, but no study has been carried out to compare deep sedation versus minimal. We carried out a retrospective study to compare the Deep versus Minimal sedation (MiS) in patients undergoing EBUS-TBNA.  The primary end point was the diagnostic accuracy. The secondary end points were adequacy and sensitivity. We evaluated the LN size sampling, procedural time, complications and patient tolerance. Thirty-six patients underwent EBUS-TBNA, 16 under DS and 20 under MiS. The overall diagnostic accuracy for correct diagnosis was 92.9% in DS group and 94.1% in MiS group (p=0.554). Sample adequacy, defined as the percentage of patients with a specific diagnosis by EBUS-TBNA, was 87.5% (14 of 16) and 85% (17 of 20) for the DS group and MiS group, respectively, (p=0.788); the sensitivity was 92.9% in the DS group (95% CI, 73-100%) and 92.9% in the MiS group (95% CI, 77-100%) (p=0.463). There were no major complications in either group. Minor complications were 4 in MiS and 1 in DS (p=0.355).  The patients in the MiS group recalled the procedure more often compared to the other group (p=0.041). The majority of the patients would agree to undergo the same procedure again in the future in both groups (p=0.766).  In our experience EBUS-TBNA performed under MiS has comparable accuracy, adequacy, sensitivity, complications and patient satisfaction to DS, even if the sample was small.  Future prospective multicenter studies are needed to confirm our results.


2021 ◽  
Vol 15 (8) ◽  
pp. 2057-2059
Author(s):  
Maham Ashraf ◽  
Aysha Anjum ◽  
Eisha Tahir ◽  
Amber Goraya ◽  
Rabia Aqeel

Background & Objective: Renal tumors are a common finding in diagnostic imaging; these lesions usually are solid or cystic, benign or malignant, and the correct diagnosis may be difficult. The current study aims at to determine the diagnostic accuracy of magnetic resonance imaging for the diagnosis of Wilms tumor taking histopathology as gold standard. Methodology: This cross-sectional study was carried at the Department of Radiology, The Children’s Hospital & Institute of Child Health Lahore over 6 months from March 2019 to September 2019. The study involved 125 children of both genders aged between 2 years to 14 years presenting with a neoplastic mass in the retroperitoneum on ultrasound abdomen during initial workup. These children were assessed on MRI for the diagnosis of Wilms tumor. Later the diagnosis was confirmed on histopathology which was taken as gold standard and the diagnosis of MRI was judged accordingly as true/false and positive/negative. A written informed consent was obtained from every patient. Results: The mean age of the children was 5.8±3.9 years having a male predominance with male to female ratio of 1.8:1. Wilms tumor was suspected in 13 (10.4%) children on MRI. The diagnosis of Wilms tumor was confirmed in 13 (10.4%) children on histopathology. MRI was found to be 76.9% sensitive, 97.3% specific and 95.2% accurate with positive and negative predictive values of 76.9% and 97.3% respectively. Conclusion: In the present study, MRI was found to be 95.2% accurate in the differential diagnosis of Wilms tumor in children presenting with retroperitoneal mass which along with its non-invasive and radiation free nature advocates the preferred use of MRI in the diagnostic evaluation of such children in future oncologic practice. Keywords: Retroperitoneal Tumor, Wilms Tumor, MRI, Diagnostic Accuracy


10.2196/16047 ◽  
2019 ◽  
Vol 8 (11) ◽  
pp. e16047 ◽  
Author(s):  
Don Roosan ◽  
Anandi V Law ◽  
Mazharul Karim ◽  
Moom Roosan

Background According to the September 2015 Institute of Medicine report, Improving Diagnosis in Health Care, each of us is likely to experience one diagnostic error in our lifetime, often with devastating consequences. Traditionally, diagnostic decision making has been the sole responsibility of an individual clinician. However, diagnosis involves an interaction among interprofessional team members with different training, skills, cultures, knowledge, and backgrounds. Moreover, diagnostic error is prevalent in the interruption-prone environment, such as the emergency department, where the loss of information may hinder a correct diagnosis. Objective The overall purpose of this protocol is to improve team-based diagnostic decision making by focusing on data analytics and informatics tools that improve collective information management. Methods To achieve this goal, we will identify the factors contributing to failures in team-based diagnostic decision making (aim 1), understand the barriers of using current health information technology tools for team collaboration (aim 2), and develop and evaluate a collaborative decision-making prototype that can improve team-based diagnostic decision making (aim 3). Results Between 2019 to 2020, we are collecting data for this study. The results are anticipated to be published between 2020 and 2021. Conclusions The results from this study can shed light on improving diagnostic decision making by incorporating diagnostics rationale from team members. We believe a positive direction to move forward in solving diagnostic errors is by incorporating all team members, and using informatics. International Registered Report Identifier (IRRID) DERR1-10.2196/16047


Folia Medica ◽  
2019 ◽  
Vol 61 (1) ◽  
pp. 158-162 ◽  
Author(s):  
Radka T. Komitova ◽  
Vasko A Grklanov ◽  
Konstantinos K. Lindas ◽  
Petya V. Ganeva

Abstract Low-dose once weekly methotrexate (MTX) is the first-line disease-modifying antirheumatic drug (DMARD) to treat rheumatoid arthritis and psoriasis. Although methotrexate is generally considered to have a good safety profile it can occasionally induce severe side effects such as pancytopenia, mucositis, disorders of kidney and liver. Oral mucositis should alert physicians to MTX toxicity. We report a 64-year-old woman with a severe drug reaction including disseminated shingles, oral mucositis and pancytopenia only three days after starting a therapeutic low-dose of MTX. Initially, mucositis and myelosuppression couldn’t be accounted for by the underlying disease. In taking the patient’s history, MTX intake 10 days ago was missed, the patient reporting only current medications. However, there was more to the skin rash than meets the eye. Only after further inquiry did the patient reveal the intake of 2 doses of MTX and the subsequent withdrawal of medication. Arriving at the correct diagnosis in difficult cases, as in the case presented, requires further evaluation, including repeat history taking and eliciting more details if diagnosis remains elusive.


Sign in / Sign up

Export Citation Format

Share Document