BACKGROUND
Automated medical history-taking systems that generate differential diagnosis lists have been suggested to contribute to improved diagnostic accuracy. However, the effect of this system on diagnostic errors in clinical practice remains unknown.
OBJECTIVE
This study aimed to assess the incidence of diagnostic errors in an outpatient department, where an artificial intelligence (AI)-driven automated medical history-taking system that generates differential diagnosis lists was implemented in clinical practice.
METHODS
We conducted a retrospective observational study using data from a community hospital in Japan. We included patients aged 20 and older who used an AI-driven automated medical history-taking system that generates differential diagnosis lists in the outpatient department of internal medicine for whom the index visit was between July 1, 2019, and June 30, 2020, followed by unplanned hospitalization within 14 days. The primary endpoint was the incidence of diagnostic errors, which were detected using the Revised Safer Dx instrument by at least two independent reviewers. To evaluate the differential diagnosis list of AI on the incidence of diagnostic errors, we compared the incidence of diagnostic errors between the groups in which AI generated the final diagnosis in the differential diagnosis list and in which AI did not generate the final diagnosis in the differential diagnosis list; Fisher’s exact test was used for comparison between these groups. For cases with confirmed diagnostic errors, further review was conducted to identify the contributing factors of diagnostic errors via discussion among the three reviewers, using the Safer Dx Process Breakdown Supplement as a reference.
RESULTS
A total of 146 patients were analyzed. The final diagnosis was confirmed in 138 patients and the final diagnosis was observed in the differential diagnosis list of the AI in 69 patients. Diagnostic errors occurred in 16 of 146 patients (11.0%; 95% confidence interval, 6.4-17.2%). Although statistically insignificant, the incidence of diagnostic errors was lower in cases in which the final diagnosis was included in the differential diagnosis list of AI than in cases in which the final diagnosis was not included (7.2% vs. 15.9%, P=.18). Regarding the quality of clinical history taken by AI, the final diagnosis was easily assumed by reading only the clinical history taken by the system in 11 of 16 cases (68.8%).
CONCLUSIONS
The incidence of diagnostic errors in the internal medicine outpatients used an automated medical history-taking system that generates differential diagnosis lists seemed to be lower than the previously reported incidence of diagnostic errors. This result suggests that the implementation of an automated medical history-taking system that generates differential diagnosis lists could be beneficial for diagnostic safety in the outpatient department of internal medicine.
Incidence of diagnostic errors in unplanned hospitalized patients using an automated medical history-taking system with differential diagnosis generator: retrospective observational study (Preprint)