A quality assessment tool for artificial intelligence-centered diagnostic test accuracy studies: QUADAS-AI

Delirium is a reversible impairment of metabolism in the human brain. Early detection is important, and an effective screening tool for nurses is crucial. The Delirium Observation Screening (DOS) scale is one such screening tool; however, its diagnostic test accuracy has not yet been thoroughly examined. This study, therefore, aimed to evaluate the accuracy of the scale through a systematic review and meta-analysis. In July 2019, a search was conducted in the MEDLINE, CINAHL, Embase, and PsycARTICLES databases, and following a review against pre-defined eligibility criteria, eight studies were finally included. The quality assessment tool of diagnostic accuracy studies was applied to each study and a hierarchical regression model was used to calculate the pooled estimates of sensitivity (90%; 76%–97%, CI 95%) and specificity (92%; 88%–94%, CI 95%). The findings indicated a high diagnostic test accuracy for the DOS scale.

Download Full-text

Artificial Intelligence for the Prediction of Helicobacter Pylori Infection in Endoscopic Images: Systematic Review and Meta-Analysis Of Diagnostic Test Accuracy (Preprint)

10.2196/preprints.21983 ◽

2020 ◽

Author(s):

Chang Seok Bang ◽

Jae Jun Lee ◽

Gwang Ho Baik

Keyword(s):

Artificial Intelligence ◽

Systematic Review ◽

Helicobacter Pylori ◽

Diagnostic Test ◽

Meta Analysis ◽

Diagnostic Test Accuracy ◽

Test Accuracy ◽

Endoscopic Images ◽

Number Of Patients ◽

H Pylori

BACKGROUND Helicobacter pylori plays a central role in the development of gastric cancer, and prediction of H pylori infection by visual inspection of the gastric mucosa is an important function of endoscopy. However, there are currently no established methods of optical diagnosis of H pylori infection using endoscopic images. Definitive diagnosis requires endoscopic biopsy. Artificial intelligence (AI) has been increasingly adopted in clinical practice, especially for image recognition and classification. OBJECTIVE This study aimed to evaluate the diagnostic test accuracy of AI for the prediction of H pylori infection using endoscopic images. METHODS Two independent evaluators searched core databases. The inclusion criteria included studies with endoscopic images of H pylori infection and with application of AI for the prediction of H pylori infection presenting diagnostic performance. Systematic review and diagnostic test accuracy meta-analysis were performed. RESULTS Ultimately, 8 studies were identified. Pooled sensitivity, specificity, diagnostic odds ratio, and area under the curve of AI for the prediction of H pylori infection were 0.87 (95% CI 0.72-0.94), 0.86 (95% CI 0.77-0.92), 40 (95% CI 15-112), and 0.92 (95% CI 0.90-0.94), respectively, in the 1719 patients (385 patients with H pylori infection vs 1334 controls). Meta-regression showed methodological quality and included the number of patients in each study for the purpose of heterogeneity. There was no evidence of publication bias. The accuracy of the AI algorithm reached 82% for discrimination between noninfected images and posteradication images. CONCLUSIONS An AI algorithm is a reliable tool for endoscopic diagnosis of H pylori infection. The limitations of lacking external validation performance and being conducted only in Asia should be overcome. CLINICALTRIAL PROSPERO CRD42020175957; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=175957

Download Full-text

33 Diagnostic Test Accuracy of the 4AT for Delirium Detection: Systematic Review and Meta-Analysis

Age and Ageing ◽

10.1093/ageing/afab029.12 ◽

2021 ◽

Vol 50 (Supplement_1) ◽

pp. i7-i11

Author(s):

Z Tieges ◽

A M J MacLullich ◽

A Anand ◽

M Cassaroni ◽

M O'Connor ◽

...

Keyword(s):

Systematic Review ◽

Older Adults ◽

Diagnostic Test ◽

Sensitivity And Specificity ◽

Assessment Tool ◽

Meta Analysis ◽

Assessment Method ◽

Diagnostic Test Accuracy ◽

Test Accuracy ◽

Cochrane Central Register

Abstract Introduction Detection of delirium in hospitalised older adults is recommended in national and international guidelines. The 4 ‘A’s Test (4AT; www.the4AT.com) is a short (<2 min) instrument for delirium detection that is used internationally as a standard tool in clinical practice. We performed a systematic review and meta-analysis of diagnostic test accuracy of the 4AT for delirium detection. Methods We searched the following electronic databases through Ovid: MEDLINE, Embase, and PsycINFO. Additional databases were searched: CINAHL (EBSCOhost), clinicaltrials.gov and Cochrane Central Register of Controlled Trials from 2011 (4AT publication) until 21 December 2019. Inclusion criteria: older adults (≥65) across any setting of care except critical care; validation study of the 4AT against a delirium reference standard (standard diagnostic criteria or validated tool). Two reviewers independently screened abstracts and papers and performed the data extraction. Pooled estimates of sensitivity and specificity were generated from a bivariate random effects model. Results 17 studies (n = 3,701 observations) were included. Various settings including acute medicine, surgery, stroke wards and the emergency department were represented. The overall prevalence of delirium was 24.2% (95% CI 17.8–32.1%; range 10.5–61.9%). The pooled sensitivity was 0.88 (95% CI 0.80–0.93) and the pooled specificity was 0.88 (95% CI 0.82–0.92). The methodological quality of studies was mostly good. Conclusions The 4AT is now supported by a substantial evidence base comparable to other well-studied tools such as the Confusion Assessment Method (CAM). The strong pooled sensitivity and specificity findings for the 4AT in this meta-analysis along with its brevity and lack of need for specific training provide support for its use as an effective assessment tool for delirium.

Download Full-text

Developing a reporting guideline for artificial intelligence-centred diagnostic test accuracy studies: the STARD-AI protocol

BMJ Open ◽

10.1136/bmjopen-2020-047709 ◽

2021 ◽

Vol 11 (6) ◽

pp. e047709

Author(s):

Viknesh Sounderajah ◽

Hutan Ashrafian ◽

Robert M Golub ◽

Shravya Shetty ◽

Jeffrey De Fauw ◽

...

Keyword(s):

Artificial Intelligence ◽

Diagnostic Test ◽

Public Involvement ◽

Project Team ◽

Diagnostic Test Accuracy ◽

Dissemination And Implementation ◽

Test Accuracy ◽

Generation Process ◽

Dissemination Strategy ◽

Current Form

IntroductionStandards for Reporting of Diagnostic Accuracy Study (STARD) was developed to improve the completeness and transparency of reporting in studies investigating diagnostic test accuracy. However, its current form, STARD 2015 does not address the issues and challenges raised by artificial intelligence (AI)-centred interventions. As such, we propose an AI-specific version of the STARD checklist (STARD-AI), which focuses on the reporting of AI diagnostic test accuracy studies. This paper describes the methods that will be used to develop STARD-AI.Methods and analysisThe development of the STARD-AI checklist can be distilled into six stages. (1) A project organisation phase has been undertaken, during which a Project Team and a Steering Committee were established; (2) An item generation process has been completed following a literature review, a patient and public involvement and engagement exercise and an online scoping survey of international experts; (3) A three-round modified Delphi consensus methodology is underway, which will culminate in a teleconference consensus meeting of experts; (4) Thereafter, the Project Team will draft the initial STARD-AI checklist and the accompanying documents; (5) A piloting phase among expert users will be undertaken to identify items which are either unclear or missing. This process, consisting of surveys and semistructured interviews, will contribute towards the explanation and elaboration document and (6) On finalisation of the manuscripts, the group’s efforts turn towards an organised dissemination and implementation strategy to maximise end-user adoption.Ethics and disseminationEthical approval has been granted by the Joint Research Compliance Office at Imperial College London (reference number: 19IC5679). A dissemination strategy will be aimed towards five groups of stakeholders: (1) academia, (2) policy, (3) guidelines and regulation, (4) industry and (5) public and non-specific stakeholders. We anticipate that dissemination will take place in Q3 of 2021.

Download Full-text

Artificial Intelligence for the Prediction of Helicobacter Pylori Infection in Endoscopic Images: Systematic Review and Meta-Analysis Of Diagnostic Test Accuracy

Journal of Medical Internet Research ◽

10.2196/21983 ◽

2020 ◽

Vol 22 (9) ◽

pp. e21983

Author(s):

Chang Seok Bang ◽

Jae Jun Lee ◽

Gwang Ho Baik

Keyword(s):

Artificial Intelligence ◽

Systematic Review ◽

Helicobacter Pylori ◽

Diagnostic Test ◽

Meta Analysis ◽

Diagnostic Test Accuracy ◽

Test Accuracy ◽

Endoscopic Images ◽

Number Of Patients ◽

H Pylori

Background Helicobacter pylori plays a central role in the development of gastric cancer, and prediction of H pylori infection by visual inspection of the gastric mucosa is an important function of endoscopy. However, there are currently no established methods of optical diagnosis of H pylori infection using endoscopic images. Definitive diagnosis requires endoscopic biopsy. Artificial intelligence (AI) has been increasingly adopted in clinical practice, especially for image recognition and classification. Objective This study aimed to evaluate the diagnostic test accuracy of AI for the prediction of H pylori infection using endoscopic images. Methods Two independent evaluators searched core databases. The inclusion criteria included studies with endoscopic images of H pylori infection and with application of AI for the prediction of H pylori infection presenting diagnostic performance. Systematic review and diagnostic test accuracy meta-analysis were performed. Results Ultimately, 8 studies were identified. Pooled sensitivity, specificity, diagnostic odds ratio, and area under the curve of AI for the prediction of H pylori infection were 0.87 (95% CI 0.72-0.94), 0.86 (95% CI 0.77-0.92), 40 (95% CI 15-112), and 0.92 (95% CI 0.90-0.94), respectively, in the 1719 patients (385 patients with H pylori infection vs 1334 controls). Meta-regression showed methodological quality and included the number of patients in each study for the purpose of heterogeneity. There was no evidence of publication bias. The accuracy of the AI algorithm reached 82% for discrimination between noninfected images and posteradication images. Conclusions An AI algorithm is a reliable tool for endoscopic diagnosis of H pylori infection. The limitations of lacking external validation performance and being conducted only in Asia should be overcome. Trial Registration PROSPERO CRD42020175957; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=175957

Download Full-text