A tutorial for meta-analysis of diagnostic tests for low-prevalence diseases: Bayesian models and software

Although measures such as sensitivity and specificity are used in the study of diagnostic test accuracy, these are not appropriate for integrating heterogeneous studies. Therefore, it is essential to assess in detail all related aspects prior to integrating a set of studies so that the correct model can then be selected. This work describes the scheme employed for making decisions regarding the use of the R, STATA and SAS statistical programs. We used the R Program Meta-Analysis of Diagnostic Accuracy package for determining the correlation between sensitivity and specificity. This package considers fixed, random and mixed effects models and provides excellent summaries and assesses heterogeneity. For selecting various cutoff points in the meta-analysis, we used the STATA module for meta-analytical integration of diagnostic test accuracy studies, which produces bivariate outputs for heterogeneity.

Download Full-text

Metadta: A Stata Command for Meta-analysis and Meta-regression of Diagnostic Test Accuracy Data – A Tutorial.

10.21203/rs.3.rs-721114/v1 ◽

2021 ◽

Author(s):

Victoria Nyawira Nyaga ◽

Marc Arbyn

Keyword(s):

Bladder Cancer ◽

Diagnostic Test ◽

Sensitivity And Specificity ◽

Fixed Effects ◽

Meta Analysis ◽

Diagnostic Test Accuracy ◽

Forest Plot ◽

Test Accuracy ◽

Statistical Procedures ◽

Meta Regression

Abstract BackgroundAlthough statistical procedures for pooling of several epidemiological metrics are generally available in statistical packages, those for meta-analysis of diagnostic test accuracy studies including options for multivariate regression are lacking. Fitting regression models and the processing of the estimates often entails lengthy and tedious calculations. Therefore, packaging appropriate statistical procedures in a robust and user-friendly program is of great interest to the scientific community. Methodsmetadta is a statistical program for pooling of diagnostic accuracy test data in Stata. It implements both the bivariate random-effects and fixed-effects model, allows for meta-regression, and presents the results in tables, a forest plot and/or summary receiver operating characteristic (SROC) plot. For a model without covariates, it also quantifies heterogeneity using an I2 statistic that accounts for the mean-variance relationship, and correlation between sensitivity and specificity, a typical characteristic of diagnostic data. To demonstrate metadta, we applied the program on two published meta-analyses on: 1) the sensitivity and specificity of cytology and other markers including telomerase for primary diagnosis of bladder cancer; and 2) the accuracy of human papillomavirus testing on self-collected versus clinician-collected samples to detect cervical precancer.ResultsWithout requiring a continuity correction, metadta generated a pooled sensitivity and specificity of 0.77 [95% CI: 0.70, 0.82] and 0.91 [95% CI: 0.75, 0.97] respectively of telomerase for the diagnosis of primary bladder cancer. metadta allowed to assess the relative accuracy of human Papilloma virus (HPV) testing on self- versus clinician-taken specimens in matched studies taking into account two covariates. Under the condition of using assays based on target-amplification, HPV tests were similarly sensitive to detect cervical pre-cancer, irrespective of clinical setting. ConclusionThe metadta program implements state of art statistical procedures in an attempt to close the gap between methodological statisticians and systematic reviewers. With metadta, we hope to popularize even further, the use of appropriate statistical methods for diagnostic meta-analysis.

Download Full-text

Use of Smartphones to Detect Diabetic Retinopathy: Scoping Review and Meta-Analysis of Diagnostic Test Accuracy Studies (Preprint)

10.2196/preprints.16658 ◽

2019 ◽

Author(s):

Choon Han Tan ◽

Bhone Myint Kyaw ◽

Helen Smith ◽

Colin S Tan ◽

Lorainne Tudor Car

Keyword(s):

Diabetic Retinopathy ◽

Diagnostic Accuracy ◽

Diagnostic Test ◽

Sensitivity And Specificity ◽

Meta Analysis ◽

Diagnostic Test Accuracy ◽

Fundus Photography ◽

Diabetic Patients ◽

Test Accuracy ◽

Reference Standard

BACKGROUND Diabetic retinopathy (DR), a common complication of diabetes mellitus, is the leading cause of impaired vision in adults worldwide. Smartphone ophthalmoscopy involves using a smartphone camera for digital retinal imaging. Utilizing smartphones to detect DR is potentially more affordable, accessible, and easier to use than conventional methods. OBJECTIVE This study aimed to determine the diagnostic accuracy of various smartphone ophthalmoscopy approaches for detecting DR in diabetic patients. METHODS We performed an electronic search on the Medical Literature Analysis and Retrieval System Online (MEDLINE), EMBASE, and Cochrane Library for literature published from January 2000 to November 2018. We included studies involving diabetic patients, which compared the diagnostic accuracy of smartphone ophthalmoscopy for detecting DR to an accurate or commonly employed reference standard, such as indirect ophthalmoscopy, slit-lamp biomicroscopy, and tabletop fundus photography. Two reviewers independently screened studies against the inclusion criteria, extracted data, and assessed the quality of included studies using the Quality Assessment of Diagnostic Accuracy Studies–2 tool, with disagreements resolved via consensus. Sensitivity and specificity were pooled using the random effects model. A summary receiver operating characteristic (SROC) curve was constructed. This review is reported in line with the Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies guidelines. RESULTS In all, nine studies involving 1430 participants were included. Most studies were of high quality, except one study with limited applicability because of its reference standard. The pooled sensitivity and specificity for detecting any DR was 87% (95% CI 74%-94%) and 94% (95% CI 81%-98%); mild nonproliferative DR (NPDR) was 39% (95% CI 10%-79%) and 95% (95% CI 91%-98%); moderate NPDR was 71% (95% CI 57%-81%) and 95% (95% CI 88%-98%); severe NPDR was 80% (95% CI 49%-94%) and 97% (95% CI 88%-99%); proliferative DR (PDR) was 92% (95% CI 79%-97%) and 99% (95% CI 96%-99%); diabetic macular edema was 79% (95% CI 63%-89%) and 93% (95% CI 82%-97%); and referral-warranted DR was 91% (95% CI 86%-94%) and 89% (95% CI 56%-98%). The area under SROC curve ranged from 0.879 to 0.979. The diagnostic odds ratio ranged from 11.3 to 1225. CONCLUSIONS We found heterogeneous evidence showing that smartphone ophthalmoscopy performs well in detecting DR. The diagnostic accuracy for PDR was highest. Future studies should standardize reference criteria and classification criteria and evaluate other available forms of smartphone ophthalmoscopy in primary care settings.

Download Full-text

33 Diagnostic Test Accuracy of the 4AT for Delirium Detection: Systematic Review and Meta-Analysis

Age and Ageing ◽

10.1093/ageing/afab029.12 ◽

2021 ◽

Vol 50 (Supplement_1) ◽

pp. i7-i11

Author(s):

Z Tieges ◽

A M J MacLullich ◽

A Anand ◽

M Cassaroni ◽

M O'Connor ◽

...

Keyword(s):

Systematic Review ◽

Older Adults ◽

Diagnostic Test ◽

Sensitivity And Specificity ◽

Assessment Tool ◽

Meta Analysis ◽

Assessment Method ◽

Diagnostic Test Accuracy ◽

Test Accuracy ◽

Cochrane Central Register

Abstract Introduction Detection of delirium in hospitalised older adults is recommended in national and international guidelines. The 4 ‘A’s Test (4AT; www.the4AT.com) is a short (<2 min) instrument for delirium detection that is used internationally as a standard tool in clinical practice. We performed a systematic review and meta-analysis of diagnostic test accuracy of the 4AT for delirium detection. Methods We searched the following electronic databases through Ovid: MEDLINE, Embase, and PsycINFO. Additional databases were searched: CINAHL (EBSCOhost), clinicaltrials.gov and Cochrane Central Register of Controlled Trials from 2011 (4AT publication) until 21 December 2019. Inclusion criteria: older adults (≥65) across any setting of care except critical care; validation study of the 4AT against a delirium reference standard (standard diagnostic criteria or validated tool). Two reviewers independently screened abstracts and papers and performed the data extraction. Pooled estimates of sensitivity and specificity were generated from a bivariate random effects model. Results 17 studies (n = 3,701 observations) were included. Various settings including acute medicine, surgery, stroke wards and the emergency department were represented. The overall prevalence of delirium was 24.2% (95% CI 17.8–32.1%; range 10.5–61.9%). The pooled sensitivity was 0.88 (95% CI 0.80–0.93) and the pooled specificity was 0.88 (95% CI 0.82–0.92). The methodological quality of studies was mostly good. Conclusions The 4AT is now supported by a substantial evidence base comparable to other well-studied tools such as the Confusion Assessment Method (CAM). The strong pooled sensitivity and specificity findings for the 4AT in this meta-analysis along with its brevity and lack of need for specific training provide support for its use as an effective assessment tool for delirium.

Download Full-text

Ultrasonography for the diagnosis of extra-cranial carotid occlusion – diagnostic test accuracy meta-analysis

VASA ◽

10.1024/0301-1526/a000850 ◽

2020 ◽

Vol 49 (3) ◽

pp. 195-204

Author(s):

Djamila M. Rojoa ◽

Ahmad Q. D. Lodhi ◽

Nikos Kontopodis ◽

Christos V. Ioannou ◽

Nicos Labropoulos ◽

...

Keyword(s):

Systematic Review ◽

Diagnostic Test ◽

Sensitivity And Specificity ◽

Meta Analysis ◽

Diagnostic Test Accuracy ◽

Carotid Occlusion ◽

Test Accuracy ◽

Free Text ◽

Correct Identification ◽

Reference Standard

Summary: Background: The correct diagnosis of internal carotid artery (ICA) occlusion is crucial as it limits unnecessary intervention, whereas correct identification of patients with severe ICA stenosis is paramount in decision making and selecting patients who would benefit from intervention. We aimed to evaluate the accuracy of ultrasonography (US) in the diagnosis of ICA occlusion. Methods: We conducted a systematic review in compliance with the Preferred Reporting Items for a Systematic Review and Meta-analysis (PRISMA) of diagnostic test accuracy studies. We interrogated electronic bibliographic sources using a combination of free text and thesaurus terms to identify studies assessing the diagnostic accuracy of US in ICA occlusion. We used a mixed-effects logistic regression bivariate model to estimate summary sensitivity and specificity. We developed hierarchical summary receiver operating characteristic (HSROC) curves. Results: We identified 23 studies reporting a total of 5,675 arteries of which 722 were proven to be occluded by the reference standard. The reference standard was digital subtraction or cerebral angiography in all but two studies, which used surgery to ascertain a carotid occlusion. The pooled estimates for sensitivity and specificity were 0.97 (95% confidence interval (CI) 0.94 to 0.99) and 0.99 (95% CI 0.98 to 1.00), respectively. The diagnostic odds ratio was 3,846.15 (95% CI 1,375.74 to 10,752.65). The positive and negative likelihood ratio were 114.71 (95% CI 58.84 to 223.63) and 0.03 (95% CI 0.01 to 0.06), respectively. Conclusions: US is a reliable and accurate method in diagnosing ICA occlusion. US can be used as a screening tool with cross-sectional imaging being reserved for ambiguous cases.

Download Full-text

Diagnostic Test Accuracy of Commercial Tests for Detection of Shiga Toxin–Producing Escherichia coli: A Systematic Review and Meta-Analysis

Clinical Chemistry ◽

10.1093/clinchem/hvz006 ◽

2020 ◽

Vol 66 (2) ◽

pp. 302-315 ◽

Cited By ~ 2

Author(s):

Gillian A M Tarr ◽

Chu Yang Lin ◽

Ben Vandermeer ◽

Diane L Lorenzetti ◽

Phillip I Tarr ◽

...

Keyword(s):

Escherichia Coli ◽

Diagnostic Test ◽

Sensitivity And Specificity ◽

Shiga Toxin ◽

Meta Analysis ◽

Diagnostic Test Accuracy ◽

Nucleic Acid Amplification ◽

Test Accuracy ◽

Random Effects Models ◽

Clinical Sensitivity

Abstract Background Rapid detection of Shiga toxin–producing Escherichia coli (STEC) enables appropriate monitoring and treatment. We synthesized available evidence to compare the performance of enzyme immunoassay (EIA) and PCR tests for the detection of STEC. Methods We searched published and gray literature for studies of STEC EIA and/or PCR diagnostic test accuracy relative to reference standards including at least one nucleic acid amplification test. Two reviewers independently screened studies, extracted data, and assessed quality with the second version of the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool. Bivariate random effects models were used to meta-analyze the clinical sensitivity and specificity of commercial EIA and PCR STEC diagnostic tests, and summary receiver operator characteristic curves were constructed. We evaluated the certainty of evidence with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. Results We identified 43 articles reflecting 25 260 specimens. Meta-analysis of EIA and PCR accuracy included 25 and 22 articles, respectively. STEC EIA pooled sensitivity and specificity were 0.681 (95% CI, 0.571–0.773; very low certainty of evidence) and 1.00 (95% CI, 0.998–1.00; moderate certainty of evidence), respectively. STEC PCR pooled sensitivity and specificity were 1.00 (95% CI, 0.904–1.00; low certainty of evidence) and 0.999 (95% CI, 0.997–0.999; low certainty of evidence), respectively. Certainty of evidence was downgraded because of high risk of bias. Conclusions PCR tests to identify the presence of STEC are more sensitive than EIA tests, with no meaningful loss of specificity. However, given the low certainty of evidence, our results may overestimate the difference in performance.

Download Full-text

Use of Smartphones to Detect Diabetic Retinopathy: Scoping Review and Meta-Analysis of Diagnostic Test Accuracy Studies

Journal of Medical Internet Research ◽

10.2196/16658 ◽

2020 ◽

Vol 22 (5) ◽

pp. e16658

Author(s):

Choon Han Tan ◽

Bhone Myint Kyaw ◽

Helen Smith ◽

Colin S Tan ◽

Lorainne Tudor Car

Keyword(s):

Diabetic Retinopathy ◽

Diagnostic Accuracy ◽

Diagnostic Test ◽

Sensitivity And Specificity ◽

Meta Analysis ◽

Diagnostic Test Accuracy ◽

Fundus Photography ◽

Diabetic Patients ◽

Test Accuracy ◽

Reference Standard

Background Diabetic retinopathy (DR), a common complication of diabetes mellitus, is the leading cause of impaired vision in adults worldwide. Smartphone ophthalmoscopy involves using a smartphone camera for digital retinal imaging. Utilizing smartphones to detect DR is potentially more affordable, accessible, and easier to use than conventional methods. Objective This study aimed to determine the diagnostic accuracy of various smartphone ophthalmoscopy approaches for detecting DR in diabetic patients. Methods We performed an electronic search on the Medical Literature Analysis and Retrieval System Online (MEDLINE), EMBASE, and Cochrane Library for literature published from January 2000 to November 2018. We included studies involving diabetic patients, which compared the diagnostic accuracy of smartphone ophthalmoscopy for detecting DR to an accurate or commonly employed reference standard, such as indirect ophthalmoscopy, slit-lamp biomicroscopy, and tabletop fundus photography. Two reviewers independently screened studies against the inclusion criteria, extracted data, and assessed the quality of included studies using the Quality Assessment of Diagnostic Accuracy Studies–2 tool, with disagreements resolved via consensus. Sensitivity and specificity were pooled using the random effects model. A summary receiver operating characteristic (SROC) curve was constructed. This review is reported in line with the Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies guidelines. Results In all, nine studies involving 1430 participants were included. Most studies were of high quality, except one study with limited applicability because of its reference standard. The pooled sensitivity and specificity for detecting any DR was 87% (95% CI 74%-94%) and 94% (95% CI 81%-98%); mild nonproliferative DR (NPDR) was 39% (95% CI 10%-79%) and 95% (95% CI 91%-98%); moderate NPDR was 71% (95% CI 57%-81%) and 95% (95% CI 88%-98%); severe NPDR was 80% (95% CI 49%-94%) and 97% (95% CI 88%-99%); proliferative DR (PDR) was 92% (95% CI 79%-97%) and 99% (95% CI 96%-99%); diabetic macular edema was 79% (95% CI 63%-89%) and 93% (95% CI 82%-97%); and referral-warranted DR was 91% (95% CI 86%-94%) and 89% (95% CI 56%-98%). The area under SROC curve ranged from 0.879 to 0.979. The diagnostic odds ratio ranged from 11.3 to 1225. Conclusions We found heterogeneous evidence showing that smartphone ophthalmoscopy performs well in detecting DR. The diagnostic accuracy for PDR was highest. Future studies should standardize reference criteria and classification criteria and evaluate other available forms of smartphone ophthalmoscopy in primary care settings.

Download Full-text