Implications of covariate induced test dependence on the diagnostic accuracy of latent class analysis in pulmonary tuberculosis

Abstract Background Lack of a perfect reference standard for pulmonary tuberculosis (PTB) diagnosis complicates assessment of accuracy of new diagnostic tests. Alternative strategies such as discrepant resolution and use of composite reference standards may lead to incorrect inferences on disease prevalence and diagnostic test sensitivity and specificity. Latent class analysis (LCA), a statistical method for analyzing diagnostic test results in the absence of a gold standard, allows correct estimation under strict assumptions. The model assumes that the diagnostic tests are independent conditional on the true disease status and that the diagnostic test sensitivity and specificity remain constant across subpopulations. These assumptions are violated when a factor such as severe comorbidity affects the prevalence and/or alters the diagnostic test performance. We aim to provide guidance on correct estimation of the prevalence and diagnostic test accuracy based on LCA when a known factor induces dependence among the diagnostic tests. If unaccounted for, this dependence may lead to misleading inferences. Methods Through likelihood evaluation and simulation we examined implications of likely model violations on estimation of prevalence, sensitivity and specificity among passive case-finding presumptive PTB patients with or without HIV. We generated independent results for five diagnostic tests conditional on PTB and HIV. We performed Bayesian LCA, separately for five and three diagnostic tests using four working models with or without constant PTB prevalence and diagnostic test accuracy across HIV subpopulations. Results In evaluating three diagnostic tests, the models accounting for heterogeneity in diagnostic accuracy produced consistent estimates while the models ignoring it produced biased estimates. The model ignoring heterogeneity in PTB prevalence is less problematic. When evaluating five diagnostic tests, the models were robust to violation of the assumptions. Conclusions Well-chosen covariate-specific adaptations of the model can avoid bias implied by recognized heterogeneity in PTB patient populations generating otherwise dependent test results in LCA.

Download Full-text

Introduction to diagnostic test accuracy studies

Acta Endocrinologica ◽

10.1530/eje-20-1239 ◽

2021 ◽

Vol 184 (2) ◽

pp. E5-E9

Author(s):

Alice J Sitch ◽

Olaf M Dekkers ◽

Barnaby R Scholefield ◽

Yemisi Takwoingi

Keyword(s):

Diagnostic Accuracy ◽

Diagnostic Test ◽

Sensitivity And Specificity ◽

Diagnostic Tests ◽

Clinical Situation ◽

Diagnostic Test Accuracy ◽

Test Accuracy ◽

Trade Off ◽

Specificity Test ◽

Uncertainty Of Results

Diagnostic accuracy studies are fundamental for the assessment of diagnostic tests. Researchers need to understand the implications of their chosen design, opting for comparative designs where possible. Researchers should analyse test accuracy studies using the appropriate methods, acknowledging the uncertainty of results and avoiding overstating conclusions and ignoring the clinical situation which should inform the trade-off between sensitivity and specificity. Test accuracy studies should be reported with transparency using the STAndards for the Reporting of Diagnostic accuracy studies (STARD) checklist.

Download Full-text

Observed Differences in Diagnostic Test Accuracy between Patient Subgroups: Is It Real or Due to Reference Standard Misclassification?

Clinical Chemistry ◽

10.1373/clinchem.2007.087403 ◽

2007 ◽

Vol 53 (10) ◽

pp. 1725-1729 ◽

Cited By ~ 15

Author(s):

Corné Biesheuvel ◽

Les Irwig ◽

Patrick Bossuyt

Keyword(s):

Clinical Practice ◽

Diagnostic Test ◽

Sensitivity And Specificity ◽

Diagnostic Test Accuracy ◽

Test Accuracy ◽

Reference Standard ◽

Target Condition ◽

Test Sensitivity ◽

The Past ◽

Patient Subgroups

Abstract Before a new test is introduced in clinical practice, its accuracy should be assessed. In the past decade, researchers have put an increased emphasis on exploring differences in test sensitivity and specificity between patient subgroups. If the reference standard is imperfect and the prevalence of the target condition differs among subgroups, apparent differences in test sensitivity and specificity between subgroups may be caused by reference standard misclassification. We provide guidance on how to determine whether observed differences may be explained by reference standard misclassification. Such misclassification may be ascertained by examining how the apparent sensitivity and specificity change with the prevalence of the target condition in the subgroups.

Download Full-text

Use of Smartphones to Detect Diabetic Retinopathy: Scoping Review and Meta-Analysis of Diagnostic Test Accuracy Studies (Preprint)

10.2196/preprints.16658 ◽

2019 ◽

Author(s):

Choon Han Tan ◽

Bhone Myint Kyaw ◽

Helen Smith ◽

Colin S Tan ◽

Lorainne Tudor Car

Keyword(s):

Diabetic Retinopathy ◽

Diagnostic Accuracy ◽

Diagnostic Test ◽

Sensitivity And Specificity ◽

Meta Analysis ◽

Diagnostic Test Accuracy ◽

Fundus Photography ◽

Diabetic Patients ◽

Test Accuracy ◽

Reference Standard

BACKGROUND Diabetic retinopathy (DR), a common complication of diabetes mellitus, is the leading cause of impaired vision in adults worldwide. Smartphone ophthalmoscopy involves using a smartphone camera for digital retinal imaging. Utilizing smartphones to detect DR is potentially more affordable, accessible, and easier to use than conventional methods. OBJECTIVE This study aimed to determine the diagnostic accuracy of various smartphone ophthalmoscopy approaches for detecting DR in diabetic patients. METHODS We performed an electronic search on the Medical Literature Analysis and Retrieval System Online (MEDLINE), EMBASE, and Cochrane Library for literature published from January 2000 to November 2018. We included studies involving diabetic patients, which compared the diagnostic accuracy of smartphone ophthalmoscopy for detecting DR to an accurate or commonly employed reference standard, such as indirect ophthalmoscopy, slit-lamp biomicroscopy, and tabletop fundus photography. Two reviewers independently screened studies against the inclusion criteria, extracted data, and assessed the quality of included studies using the Quality Assessment of Diagnostic Accuracy Studies–2 tool, with disagreements resolved via consensus. Sensitivity and specificity were pooled using the random effects model. A summary receiver operating characteristic (SROC) curve was constructed. This review is reported in line with the Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies guidelines. RESULTS In all, nine studies involving 1430 participants were included. Most studies were of high quality, except one study with limited applicability because of its reference standard. The pooled sensitivity and specificity for detecting any DR was 87% (95% CI 74%-94%) and 94% (95% CI 81%-98%); mild nonproliferative DR (NPDR) was 39% (95% CI 10%-79%) and 95% (95% CI 91%-98%); moderate NPDR was 71% (95% CI 57%-81%) and 95% (95% CI 88%-98%); severe NPDR was 80% (95% CI 49%-94%) and 97% (95% CI 88%-99%); proliferative DR (PDR) was 92% (95% CI 79%-97%) and 99% (95% CI 96%-99%); diabetic macular edema was 79% (95% CI 63%-89%) and 93% (95% CI 82%-97%); and referral-warranted DR was 91% (95% CI 86%-94%) and 89% (95% CI 56%-98%). The area under SROC curve ranged from 0.879 to 0.979. The diagnostic odds ratio ranged from 11.3 to 1225. CONCLUSIONS We found heterogeneous evidence showing that smartphone ophthalmoscopy performs well in detecting DR. The diagnostic accuracy for PDR was highest. Future studies should standardize reference criteria and classification criteria and evaluate other available forms of smartphone ophthalmoscopy in primary care settings.

Download Full-text

Clinical Epidemiology

Mayo Clinic Internal Medicine Board Review ◽

10.1093/med/9780190464868.003.0024 ◽

2016 ◽

pp. 285-290

Author(s):

Scott C. Litin ◽

John B. Bundrick

Keyword(s):

Diagnostic Test ◽

Diagnostic Tests ◽

Clinical Decision Making ◽

Clinical Epidemiology ◽

Clinical Decision ◽

Test Results ◽

Specific Patient ◽

Predictive Values ◽

Test Sensitivity ◽

Sensitivity Specificity

Diagnostic tests are tools that either increase or decrease the likelihood of disease. The sensitivity, specificity, and predictive values of normal and abnormal test results can be calculated with even a limited amount of information. Some physicians prefer interpreting diagnostic test results by using the likelihood ratio. This ratio takes properties of a diagnostic test (sensitivity and specificity) and makes them more helpful in clinical decision making. It helps the clinician determine the probability of disease in a specific patient after a diagnostic test has been performed.

Download Full-text

Combining Results of Performance-Based and Informant Test Accuracy Studies: Bayes or Boole?

Dementia and Geriatric Cognitive Disorders ◽

10.1159/000514063 ◽

2021 ◽

pp. 1-7

Author(s):

Andrew J. Larner

Keyword(s):

Sensitivity And Specificity ◽

Test Accuracy ◽

Test Results ◽

Test Sensitivity ◽

Individual Test ◽

Bayes Formula ◽

Post Test ◽

Combined Test ◽

Accuracy Study ◽

Test Probability

Background/Aims: Since screening and diagnostic tests for dementia do not have perfect accuracy, >1 test is often administered when assessing patients with cognitive complaints. Use of both patient performance tests and informant questionnaires has been recommended. Combination of individual test results may be based on methods originally defined by Thomas Bayes (revision or updating of pretest probabilities to post-test probabilities given the test results) and by George Boole (application of associative “AND” or “OR” operator). This study sought to apply these methods in clinical practice. Methods: Using the dataset of a pragmatic test accuracy study of the Six-Item Cognitive Impairment Test (6CIT) and informant Ascertain Dementia 8 (AD8), post-test probabilities for the combination were calculated using Bayes’ formula and compared to Boolean “AND” combination. Combined test sensitivity and specificity was calculated using either Boolean “AND” or “OR” operator and compared to results using equations based on individual test sensitivity and specificity. Results: Both Bayesian and Boolean methods produced similar improvements from pretest probability (0.288) to combined post-test probability for dementia (≈0.5). Likewise, the 2 different methods for calculating combined sensitivities and specificities gave similar results, with, as anticipated, the “AND” combination improving overall specificity (to ≈0.65) whereas the “OR” combination improved sensitivity (to ≈1.00). Conclusion: Combination of individual screening test results using Bayesian and Boolean methods is relatively straightforward and may add to clinicians’ intuitive judgements when combining test results.

Download Full-text

Concordance analysis between HER2 immunohistochemistry and in situ hybridization in non-small cell lung cancer

The International Journal of Biological Markers ◽

10.5301/ijbm.5000271 ◽

2017 ◽

Vol 33 (1) ◽

pp. 49-54 ◽

Cited By ~ 3

Author(s):

Young San Ko ◽

Nae Yu Kim ◽

Jung-Soo Pyo

Keyword(s):

Lung Cancer ◽

In Situ Hybridization ◽

Diagnostic Accuracy ◽

Small Cell Lung Cancer ◽

Diagnostic Test ◽

Sensitivity And Specificity ◽

Small Cell ◽

Test Accuracy ◽

Small Cell Lung

Purpose: This study aimed to elucidate the concordance between human epidermal growth factor receptor 2 (HER2) immunohistochemistry (IHC) and in situ hybridization (ISH) and the diagnostic accuracy of HER2 IHC in non-small cell lung cancer (NSCLC) through a meta-analysis and diagnostic test accuracy review. Methods: Seven eligible studies and 1,217 patients with NSCLC were included in the review. The concordance between HER2 IHC and ISH was analyzed. To confirm the diagnostic accuracy of HER2 IHC, the sensitivity and specificity were analyzed and the area under the curve (AUC) in the summary receiver operating characteristic (SROC) curve was calculated. Results: The concordance rate between HER2 IHC and ISH was 0.795 (95% confidence interval [CI] 0.534-0.929). In the HER2 IHC-negative (score 0/1+) subgroup, the concordance rate was 0.975 (95% CI 0.854-0.996). The concordance rates of the HER2 IHC score 2+ and 3+ subgroups were 0.091 (95% CI 0.039-0.197) and 0.665 (95% CI 0.446-0.830), respectively. In diagnostic test accuracy review, the pooled sensitivity and specificity were 0.67 (95% CI 0.54-0.78) and 0.89 (95% CI 0.87-0.91), respectively. The AUC in the SROC curve was 0.891 and the diagnostic odds ratio was 16.99 (95% CI 5.08-56.76). Conclusions: HER2 IHC was largely in agreement with ISH in cases of HER2 IHC score 0/1+. Because the concordance rates of HER2 IHC score 2/3+ cases were lower than that of HER2 IHC score 0/1+ cases, further studies for detailed analysis criteria for HER2 IHC score 2+ or 3+ are required.

Download Full-text

Systematic Reviews of Studies Quantifying the Accuracy of Diagnostic Tests and Markers

Clinical Chemistry ◽

10.1373/clinchem.2012.182568 ◽

2012 ◽

Vol 58 (11) ◽

pp. 1534-1545 ◽

Cited By ~ 28

Author(s):

Johannes B Reitsma ◽

Karel GM Moons ◽

Patrick MM Bossuyt ◽

Kristian Linnet

Keyword(s):

Diagnostic Accuracy ◽

Diagnostic Test ◽

Systematic Reviews ◽

Diagnostic Tests ◽

Diagnostic Test Accuracy ◽

Test Accuracy ◽

Recent Developments ◽

Potential Benefits ◽

Made In

Abstract Systematic reviews of diagnostic accuracy studies allow calculation of pooled estimates of accuracy with increased precision and examination of differences in accuracy between tests or subgroups of studies. Recently, several advances have been made in the methods used in performing systematic reviews of diagnostic test accuracy studies, most notably in how to assess the methodological quality of primary diagnostic test accuracy studies by use of QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies 2) instrument and how to develop sound statistical models for metaanalysis of the paired measures of test accuracy (bivariate metaregression model of sensitivity and specificity). This article provides an overview of the different steps within a diagnostic systematic review and highlights these advances, illustrated with empirical data. The potential benefits of some recent developments in the areas of network metaanalysis and individual patient data metaanalysis for diagnostic tests are also discussed.

Download Full-text

Hierarchical Modeling for Diagnostic Test Accuracy Using Multivariate Probability Distribution Functions

Mathematics ◽

10.3390/math9111310 ◽

2021 ◽

Vol 9 (11) ◽

pp. 1310

Author(s):

Johny Pambabay-Calero ◽

Sergio Bauz-Olvera ◽

Ana Nieto-Librero ◽

Ana Sánchez-García ◽

Puri Galindo-Villardón

Keyword(s):

Probability Distribution ◽

Diagnostic Test ◽

Sensitivity And Specificity ◽

Goodness Of Fit ◽

Meta Analysis ◽

Test Accuracy ◽

Test Results ◽

Goodness Of Fit Test ◽

Bivariate Model ◽

Copula Models

Models implemented in statistical software for the precision analysis of diagnostic tests include random-effects modeling (bivariate model) and hierarchical regression (hierarchical summary receiver operating characteristic). However, these models do not provide an overall mean, but calculate the mean of a central study when the random effect is equal to zero; hence, it is difficult to calculate the covariance between sensitivity and specificity when the number of studies in the meta-analysis is small. Furthermore, the estimation of the correlation between specificity and sensitivity is affected by the number of studies included in the meta-analysis, or the variability among the analyzed studies. To model the relationship of diagnostic test results, a binary covariance matrix is assumed. Here we used copulas as an alternative to capture the dependence between sensitivity and specificity. The posterior values were estimated using methods that consider sampling algorithms from a probability distribution (Markov chain Monte Carlo), and estimates were compared with the results of the bivariate model, which assumes statistical independence in the test results. To illustrate the applicability of the models and their respective comparisons, data from 14 published studies reporting estimates of the accuracy of the Alcohol Use Disorder Identification Test were used. Using simulations, we investigated the performance of four copula models that incorporate scenarios designed to replicate realistic situations for meta-analyses of diagnostic accuracy of the tests. The models’ performances were evaluated based on p-values using the Cramér–von Mises goodness-of-fit test. Our results indicated that copula models are valid when the assumptions of the bivariate model are not fulfilled.

Download Full-text

Use of Smartphones to Detect Diabetic Retinopathy: Scoping Review and Meta-Analysis of Diagnostic Test Accuracy Studies

Journal of Medical Internet Research ◽

10.2196/16658 ◽

2020 ◽

Vol 22 (5) ◽

pp. e16658

Author(s):

Choon Han Tan ◽

Bhone Myint Kyaw ◽

Helen Smith ◽

Colin S Tan ◽

Lorainne Tudor Car

Keyword(s):

Diabetic Retinopathy ◽

Diagnostic Accuracy ◽

Diagnostic Test ◽

Sensitivity And Specificity ◽

Meta Analysis ◽

Diagnostic Test Accuracy ◽

Fundus Photography ◽

Diabetic Patients ◽

Test Accuracy ◽

Reference Standard

Background Diabetic retinopathy (DR), a common complication of diabetes mellitus, is the leading cause of impaired vision in adults worldwide. Smartphone ophthalmoscopy involves using a smartphone camera for digital retinal imaging. Utilizing smartphones to detect DR is potentially more affordable, accessible, and easier to use than conventional methods. Objective This study aimed to determine the diagnostic accuracy of various smartphone ophthalmoscopy approaches for detecting DR in diabetic patients. Methods We performed an electronic search on the Medical Literature Analysis and Retrieval System Online (MEDLINE), EMBASE, and Cochrane Library for literature published from January 2000 to November 2018. We included studies involving diabetic patients, which compared the diagnostic accuracy of smartphone ophthalmoscopy for detecting DR to an accurate or commonly employed reference standard, such as indirect ophthalmoscopy, slit-lamp biomicroscopy, and tabletop fundus photography. Two reviewers independently screened studies against the inclusion criteria, extracted data, and assessed the quality of included studies using the Quality Assessment of Diagnostic Accuracy Studies–2 tool, with disagreements resolved via consensus. Sensitivity and specificity were pooled using the random effects model. A summary receiver operating characteristic (SROC) curve was constructed. This review is reported in line with the Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies guidelines. Results In all, nine studies involving 1430 participants were included. Most studies were of high quality, except one study with limited applicability because of its reference standard. The pooled sensitivity and specificity for detecting any DR was 87% (95% CI 74%-94%) and 94% (95% CI 81%-98%); mild nonproliferative DR (NPDR) was 39% (95% CI 10%-79%) and 95% (95% CI 91%-98%); moderate NPDR was 71% (95% CI 57%-81%) and 95% (95% CI 88%-98%); severe NPDR was 80% (95% CI 49%-94%) and 97% (95% CI 88%-99%); proliferative DR (PDR) was 92% (95% CI 79%-97%) and 99% (95% CI 96%-99%); diabetic macular edema was 79% (95% CI 63%-89%) and 93% (95% CI 82%-97%); and referral-warranted DR was 91% (95% CI 86%-94%) and 89% (95% CI 56%-98%). The area under SROC curve ranged from 0.879 to 0.979. The diagnostic odds ratio ranged from 11.3 to 1225. Conclusions We found heterogeneous evidence showing that smartphone ophthalmoscopy performs well in detecting DR. The diagnostic accuracy for PDR was highest. Future studies should standardize reference criteria and classification criteria and evaluate other available forms of smartphone ophthalmoscopy in primary care settings.

Download Full-text