Bayes Lines Tool (BLT): a SQL-script for analyzing diagnostic test results with an application to SARS-CoV-2-testing

The performance of diagnostic tests crucially depends on the disease prevalence, test sensitivity, and test specificity. However, these quantities are often not well known when tests are performed outside defined routine lab procedures which make the rating of the test results somewhat problematic. A current example is the mass testing taking place within the context of the world-wide SARS-CoV-2 crisis. Here, for the first time in history, laboratory test results have a dramatic impact on political decisions. Therefore, transparent, comprehensible, and reliable data is mandatory. It is in the nature of wet lab tests that their quality and outcome are influenced by multiple factors reducing their performance by handling procedures, underlying test protocols, and analytical reagents. These limitations in sensitivity and specificity have to be taken into account when calculating the real test results. As a resolution method, we have developed a Bayesian calculator, the Bayes Lines Tool (BLT), for analyzing disease prevalence, test sensitivity, test specificity, and, therefore, true positive, false positive, true negative, and false negative numbers from official test outcome reports. The calculator performs a simple SQL (Structured Query Language) query and can easily be implemented on any system supporting SQL. We provide an example of influenza test results from California, USA, as well as two examples of SARS-CoV-2 test results from official government reports from The Netherlands and Germany-Bavaria, to illustrate the possible parameter space of prevalence, sensitivity, and specificity consistent with the observed data. Finally, we discuss this tool’s multiple applications, including its putative importance for informing policy decisions.

Download Full-text

Bayes Lines Tool (BLT): a SQL-script for analyzing diagnostic test results with an application to SARS-CoV-2-testing

F1000Research ◽

10.12688/f1000research.51061.1 ◽

2021 ◽

Vol 10 ◽

pp. 369

Author(s):

Wouter Aukema ◽

Bobby Rajesh Malhotra ◽

Simon Goddek ◽

Ulrike Kämmerer ◽

Peter Borger ◽

...

Keyword(s):

Sensitivity And Specificity ◽

Query Language ◽

False Negative ◽

Disease Prevalence ◽

Test Results ◽

True Negative ◽

Test Sensitivity ◽

Laboratory Test Results ◽

Test Specificity ◽

Analytical Reagents

Download Full-text

Usefulness of Nasal Endoscopy for Diagnosing Patients With Chronic Rhinosinusitis: A Meta-Analysis

American Journal of Rhinology and Allergy ◽

10.1177/1945892419892157 ◽

2019 ◽

Vol 34 (2) ◽

pp. 306-314

Author(s):

Do Hyun Kim ◽

Youngjun Seo ◽

Kyung Min Kim ◽

Seoungmin Lee ◽

Se Hwan Hwang

Keyword(s):

Diagnostic Accuracy ◽

Chronic Rhinosinusitis ◽

Sensitivity And Specificity ◽

Meta Analysis ◽

Characteristic Curve ◽

False Negative ◽

Nasal Endoscopy ◽

Summary Receiver Operating Characteristic ◽

True Negative ◽

Endoscopic Score

Background We evaluated the accuracy of nasal endoscopy in diagnosing chronic rhinosinusitis (CRS) compared with paranasal sinus computed tomography (CT). Methods Two authors independently searched the 5 databases (PubMed, SCOPUS, Embase, the Web of Science, and the Cochrane database) up to March 2019. For all included studies, we calculated correlation coefficients between the endoscopic and CT scores. We extracted data on true-positive and false-positive and true-negative and false-negative results. Methodological quality was assessed using the Quality Assessment of Diagnostic Accuracy Studies tool (version 2). Results We included 16 observational or retrospective studies. A high correlation ( r = .8543; 95% confidence interval [CI] [0.7685–0.9401], P < .0001, I2 = 76.58%) between endoscopy and CT in terms of the diagnostic accuracy for CRS was apparent. The odds ratio (Lund–Kennedy endoscopic score ≥1) was 7.915 (95% CI [4.435–14.124]; I2 = 28.361%). The area under the summary receiver operating characteristic curve was 0.765. The sensitivity and specificity were 0.726 (95% CI [0.584–0.834]) and 0.767 (95% CI [0.685–0.849]), respectively. However, high interstudy heterogeneity was evident given the different endoscopic score thresholds used (Lund–Kennedy endoscopic score ≥1 vs 2). In a subgroup analysis of studies using a Lund–Kennedy endoscopic score threshold ≥2, the area under the summary curve was 0.881, and the sensitivity and specificity were 0.874 (95% CI [0.783–0.930]) and 0.793 (95% CI [0.366–0.962]), respectively. Conclusion Nasal endoscopy is a useful diagnostic tool; the Lund–Kennedy score was comparable with that of CT.

Download Full-text

Parotid Tumors: Fine-Needle Aspiration and/or Frozen Section

Otolaryngology ◽

10.1016/j.otohns.2008.05.139 ◽

2008 ◽

Vol 139 (2_suppl) ◽

pp. P42-P43

Author(s):

Peter Zbaren ◽

Heinz Loosli ◽

Edouard Stauffer

Keyword(s):

Sensitivity And Specificity ◽

Fine Needle Aspiration ◽

False Positive ◽

Frozen Section ◽

False Negative ◽

Needle Aspiration ◽

True Positive ◽

True Negative ◽

Fine Needle ◽

Parotid Neoplasms

Objective Assess the difficulties of preoperative and intraoperative tumor typing of parotid neoplasms. Know the advantages and pitfalls of fine-needle-aspiration cytology (FNAC) and frozen section (FS) analysis in primary parotid neoplasms. Methods In 113 parotid neoplasms (70 malignancies and 43 benign tumors) preoperative FNAC as well as intraoperative FS analysis were performed. FNAC and FS findings were analyzed and compared with the final histopathologic diagnosis. Results The FNAC smear was non-diagnostic in 6 tumors. In 2 FS specimens, it was not possible to determine the tumor dignity. FNAC findings and FS findings were both available in 105 neoplasMS The FNAC findings were true positive for malignancy in 54, true negative in 36, false positive in 4, and false negative in 11 tumors. The accuracy, sensitivity, and specificity were 86%, 83%, and 90% respectively. The FS findings were true positive in 60, true negative in 38, false positive in 2, and false negative in 5 tumors. The accuracy, sensitivity, and specificity were 93%, 92% and 95% respectively. The exact histologic tumor typing by FNAC was correct, false or not mentioned in 58%, 20% and 22% true positive or true negative evaluated tumors, and by FS in 83%, 5% and 12% true positive or true negative evaluated tumors. Conclusions The current analysis showed a superiority of FS compared with FNAC regarding the diagnosis of malignancy and especially of tumor typing. FNAC alone is not prone in many cases to determine the surgical management of primary parotid carcinomas.

Download Full-text

Testing the Sensitivity and Specificity of ICU Patients and Diagnose Statistics Hypothetical Errors

International Journal of Innovative Technology and Exploring Engineering - Special Issue ◽

10.35940/ijitee.j9179.0881019 ◽

2019 ◽

Vol 8 (10) ◽

pp. 1042-1045

Keyword(s):

Experimental Data ◽

Clinical Trial ◽

Sensitivity And Specificity ◽

False Negative ◽

Theoretical Concept ◽

Statistical Hypothesis ◽

True Positive ◽

True Negative ◽

Icu Patients ◽

Trial Test

Statistical performance especially for certain information based on data analyse and incorporate clinical trial incomplete observation. The handling statistical hypothesis measure to regulate, type one error and type two errors is related to the assessment of sensitivity and specificity in clinical trial test and experimental data. A theoretical concept is considered two types of errors has been made and measure to find out of False positive, False Negative, True Positive and True Negative. The study presumed to analyse the ICU patient’s condition based on who have admitted in elective or emergency. We are conclude that there is association between types of admission and patient’s status

Download Full-text

Diagnostic Accuracy of PCR Alone and Compared to Urinary Antigen Testing for Detection of Legionella spp.: a Systematic Review

Journal of Clinical Microbiology ◽

10.1128/jcm.02675-15 ◽

2015 ◽

Vol 54 (2) ◽

pp. 401-411 ◽

Cited By ~ 34

Author(s):

Tomer Avni ◽

Amir Bieber ◽

Hefziba Green ◽

Tali Steinmetz ◽

Leonard Leibovici ◽

...

Keyword(s):

Systematic Review ◽

Diagnostic Accuracy ◽

Sensitivity And Specificity ◽

Diagnostic Performance ◽

False Negative ◽

Clinical Samples ◽

Direct Immunofluorescence ◽

Urinary Antigen ◽

True Negative ◽

Consensus Definition

The diagnosis of Legionnaires' disease (LD) is based on the isolation ofLegionellaspp., a 4-fold rise in antibodies, a positive urinary antigen (UA), or direct immunofluorescence tests. PCR is not accepted as a diagnostic tool for LD. This systematic review assesses the diagnostic accuracy of PCR in various clinical samples with a direct comparison versus UA. We included prospective or retrospective cohort and case-control studies. Studies were included if they used the Centers for Disease Control and Prevention consensus definition criteria of LD or a similar one, assessed only patients with clinical pneumonia, and reported data for all true-positive, false-positive, true-negative, and false-negative results. Two reviewers abstracted data independently. Risk of bias was assessed using Quadas-2. Summary sensitivity and specificity values were estimated using a bivariate model and reported with a 95% confidence interval (CI). Thirty-eight studies were included. A total of 653 patients had confirmed LD, and 3,593 patients had pneumonia due to other pathogens. The methodological quality of the studies as assessed by the Quadas-2 tool was poor to fair. The summary sensitivity and specificity values for diagnosis of LD in respiratory samples were 97.4% (95% CI, 91.1% to 99.2%) and 98.6% (95% CI, 97.4% to 99.3%), respectively. These results were mainly unchanged by any covariates tested and subgroup analysis. The diagnostic performance of PCR in respiratory samples was much better than that of UA. Compared to UA, PCR in respiratory samples (especially in sputum samples or swabs) revealed a significant advantage in sensitivity and an additional diagnosis of 18% to 30% of LD cases. The diagnostic performance of PCR in respiratory samples was excellent and preferable to that of the UA. Results were independent on the covariate tested. PCR in respiratory samples should be regarded as a valid tool for the diagnosis of LD.

Download Full-text

Somatostatin Receptor PET/CT Imaging for the Detection and Staging of Pancreatic NET: A Systematic Review and Meta-Analysis

Diagnostics ◽

10.3390/diagnostics10080598 ◽

2020 ◽

Vol 10 (8) ◽

pp. 598 ◽

Cited By ~ 2

Author(s):

Matteo Bauckneht ◽

Domenico Albano ◽

Salvatore Annunziata ◽

Giulia Santo ◽

Priscilla Guglielmo ◽

...

Keyword(s):

Sensitivity And Specificity ◽

Meta Analysis ◽

Somatostatin Receptor ◽

False Negative ◽

Primary Lesion ◽

Index Test ◽

True Negative ◽

Positron Emission ◽

Pet Ct ◽

Initial Staging

We investigated the diagnostic performance of Somatostatin Receptor Positron Emission Tomography/Computed Tomography (SSR-PET/CT) for the detection of primary lesion and initial staging of pancreatic neuroendocrine tumors (pNETs). A comprehensive literature search up to January 2020 was performed selecting studies in presence of: sample size ≥10 patients; index test (i.e., 68Ga-DOTATOC or 68Ga-DOTANOC or 68Ga-DOTATATE PET/CT); and outcomes (i.e., detection rate (DR), true positive, true negative, false positive, and false-negative). The methodological quality was evaluated with QUADAS-2. Pooled DR and pooled sensitivity and specificity for the identification of the primary tumor were assessed by a patient-based and a lesion-based analysis. Thirty-eight studies were selected for the qualitative analysis, while 18 papers were included in the meta-analysis. The number of pNET patients ranged from 10 to 142, for a total of 1143 subjects. At patient-based analysis, the pooled sensitivity and specificity for the assessment of primary pNET were 79.6% (95% confidence interval (95%CI): 71–87%) and 95% (95%CI: 75–100%) with a heterogeneity of 59.6% and 51.5%, respectively. Pooled DR for the primary lesion was 81% (95%CI: 65–90%) and 92% (95%CI: 80–97%), respectively, at patient-based and lesion-based analysis. In conclusion, SSR-PET/CT has high DR and diagnostic performances for primary lesion and initial staging of pNETs.

Download Full-text

Occupancy modeling and resampling overcomes low test sensitivity to produce accurate SARS-CoV-2 prevalence estimates

BMC Public Health ◽

10.1186/s12889-021-10609-y ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Jamie S. Sanderlin ◽

Jessie D. Golding ◽

Taylor Wilcox ◽

Daniel H. Mason ◽

Kevin S. McKelvey ◽

...

Keyword(s):

Low Cost ◽

False Negative ◽

Emerging Infectious Diseases ◽

Disease Prevalence ◽

Occupancy Modeling ◽

Test Sensitivity ◽

Prevalence Estimates ◽

Repeated Sampling ◽

Rapid Tests ◽

Low Sensitivity

Abstract Background We evaluated whether occupancy modeling, an approach developed for detecting rare wildlife species, could overcome inherent accuracy limitations associated with rapid disease tests to generate fast, accurate, and affordable SARS-CoV-2 prevalence estimates. Occupancy modeling uses repeated sampling to estimate probability of false negative results, like those linked to rapid tests, for generating unbiased prevalence estimates. Methods We developed a simulation study to estimate SARS-CoV-2 prevalence using rapid, low-sensitivity, low-cost tests and slower, high-sensitivity, higher cost tests across a range of disease prevalence and sampling strategies. Results Occupancy modeling overcame the low sensitivity of rapid tests to generate prevalence estimates comparable to more accurate, slower tests. Moreover, minimal repeated sampling was required to offset low test sensitivity at low disease prevalence (0.1%), when rapid testing is most critical for informing disease management. Conclusions Occupancy modeling enables the use of rapid tests to provide accurate, affordable, real-time estimates of the prevalence of emerging infectious diseases like SARS-CoV-2.

Download Full-text

Evaluation of Bias in Diagnostic-Test Sensitivity and Specificity Estimates Computed by Discrepant Analysis

Journal of Clinical Microbiology ◽

10.1128/jcm.36.2.375-381.1998 ◽

1998 ◽

Vol 36 (2) ◽

pp. 375-381 ◽

Cited By ~ 23

Author(s):

Timothy A. Green ◽

Carolyn M. Black ◽

Robert E. Johnson

Keyword(s):

Cell Culture ◽

Nucleic Acid ◽

Diagnostic Test ◽

Sensitivity And Specificity ◽

Nucleic Acid Amplification ◽

Reference Test ◽

Test Sensitivity ◽

Test Specificity ◽

Positive Results

When a new diagnostic test is potentially more sensitive than the reference test used to classify persons as infected or uninfected, a substantial number of specimens from infected persons may be reference-test negative but new-test positive. Discrepant analysis involves the performance of one or more additional tests with these specimens, reclassification as infected those persons for whom the new-test-positive results are confirmed, and recalculation of the estimates of new-test sensitivity and specificity by using the revised classification. This approach has been criticized because of the bias introduced by the selective use of confirmation testing. Under conditions appropriate for evaluating a nucleic acid amplification (NAA) test for Chlamydia trachomatis infection with cell culture as the reference test, we compared the bias in estimates based on the discrepant-analysis classification of persons as infected or uninfected with that in estimates based on the culture classification. We concluded that the bias in estimates of NAA-test specificity based on discrepant analysis is small and generally less than that in estimates based on culture. However, the accuracy of discrepant-analysis-based estimates of NAA-test sensitivity depends critically on whether culture specificity is equal to or is slightly less than 100%, and it is affected by competing biases that are not fully taken into account by discrepant analysis.

Download Full-text

Disease Prevalence Matters: Challenge for SARS-CoV-2 Testing

Antibodies ◽

10.3390/antib10040050 ◽

2021 ◽

Vol 10 (4) ◽

pp. 50

Author(s):

Chin-Shern Lau ◽

Tar-Choon Aw

Keyword(s):

Diagnostic Test ◽

Sensitivity And Specificity ◽

False Positive ◽

Disease Prevalence ◽

Test Results ◽

Community Screening ◽

Predictive Values ◽

Low Prevalence

While sensitivity and specificity are important characteristics for any diagnostic test, the influence of prevalence is equally, if not more, important when such tests are used in community screening. We review the concepts of positive/negative predictive values (PPV/NPV) and how disease prevalence affects false positive/negative rates. In low-prevalence situations, the PPV decreases drastically. We demonstrate how using two tests in an orthogonal fashion can be especially beneficial in low-prevalence settings and greatly improve the PPV of the diagnostic test results.

Download Full-text

Implications of covariate induced test dependence on the diagnostic accuracy of latent class analysis in pulmonary tuberculosis

10.21203/rs.3.rs-900353/v1 ◽

2021 ◽

Author(s):

Alfred Kipyegon Keter ◽

Lutgarde Lynen ◽

Alastair van Heerden ◽

Els Goetghebeur ◽

Bart K.M. Jacobs

Keyword(s):

Pulmonary Tuberculosis ◽

Diagnostic Accuracy ◽

Diagnostic Test ◽

Sensitivity And Specificity ◽

Diagnostic Tests ◽

Latent Class ◽

Test Accuracy ◽

Test Results ◽

Test Sensitivity ◽

Correct Estimation

Abstract Background Lack of a perfect reference standard for pulmonary tuberculosis (PTB) diagnosis complicates assessment of accuracy of new diagnostic tests. Alternative strategies such as discrepant resolution and use of composite reference standards may lead to incorrect inferences on disease prevalence and diagnostic test sensitivity and specificity. Latent class analysis (LCA), a statistical method for analyzing diagnostic test results in the absence of a gold standard, allows correct estimation under strict assumptions. The model assumes that the diagnostic tests are independent conditional on the true disease status and that the diagnostic test sensitivity and specificity remain constant across subpopulations. These assumptions are violated when a factor such as severe comorbidity affects the prevalence and/or alters the diagnostic test performance. We aim to provide guidance on correct estimation of the prevalence and diagnostic test accuracy based on LCA when a known factor induces dependence among the diagnostic tests. If unaccounted for, this dependence may lead to misleading inferences. Methods Through likelihood evaluation and simulation we examined implications of likely model violations on estimation of prevalence, sensitivity and specificity among passive case-finding presumptive PTB patients with or without HIV. We generated independent results for five diagnostic tests conditional on PTB and HIV. We performed Bayesian LCA, separately for five and three diagnostic tests using four working models with or without constant PTB prevalence and diagnostic test accuracy across HIV subpopulations. Results In evaluating three diagnostic tests, the models accounting for heterogeneity in diagnostic accuracy produced consistent estimates while the models ignoring it produced biased estimates. The model ignoring heterogeneity in PTB prevalence is less problematic. When evaluating five diagnostic tests, the models were robust to violation of the assumptions. Conclusions Well-chosen covariate-specific adaptations of the model can avoid bias implied by recognized heterogeneity in PTB patient populations generating otherwise dependent test results in LCA.

Download Full-text