Interpretation of dichotomous outcomes: sensitivity, specificity, likelihood ratios, and pre-test and post-test probability

Abstract Data on predictors of intraoperative cardiac arrest (ICA) outcomes are scarce in the literature. This study analysed predictors of poor outcome and their prognostic value after an ICA. Clinical and laboratory data before and 24 hours (h) after ICA were analysed as predictors for no return of spontaneous circulation (ROSC) and 24 h and 1-year mortality. Receiver operating characteristic curves for each predictor and sensitivity, specificity, positive and negative likelihood ratios, and post-test probability were calculated. A total of 167,574 anaesthetic procedures were performed, including 158 cases of ICAs. Based on the predictors for no ROSC, a threshold of 13 minutes of ICA yielded the highest area under curve (AUC) (0.867[0.80–0.93]), with a sensitivity and specificity of 78.4% [69.6–86.3%] and 89.3% [80.4–96.4%], respectively. For the 1-year mortality, the GCS without the verbal component 24 h after an ICA had the highest AUC (0.616 [0.792–0.956]), with a sensitivity of 79.3% [65.5–93.1%] and specificity of 86.1 [74.4–95.4]. ICA duration and GCS 24 h after the event had the best prognostic value for no ROSC and 1-year mortality. For 24 h mortality, no predictors had prognostic value.

Download Full-text

A “Clinician’s Probability Calculator” to convert pre-test to post-test probability of COVID-19 , based on method validation from each laboratory

10.20944/preprints202012.0094.v1 ◽

2020 ◽

Author(s):

Zoe Brooks ◽

Saswati Das ◽

Tom Pliura

Keyword(s):

Health Staff ◽

Test Results ◽

Likelihood Ratios ◽

Percent Agreement ◽

Public Health Staff ◽

Post Test ◽

Online Calculator ◽

Test Result ◽

Post Test Probability ◽

Test Probability

During coronavirus pandemic testing and identifying the virus has been a unique and constant challenge for the scientific community. In this paper, we discuss a practical solution to help guide clinicians and public health staff with the interpretation of the probability that a positive, or negative, COVID-19 test result indicates an infected person, based on their clinical estimate of pre-test probability of infection. The LinkedIn survey confirmed that the pre-test probability of COVID-19 increases with patient age, known contact, and severity of symptoms, as well as prevalence of disease in the local population. PPA (Positive Percent Agreement, PPA) and NPA (Negative Percent Agreement, specificity), differ between individual methods. Results vary between laboratories and the manufacturer for the same method. The confidence intervals of results vary with the number of samples tested, often adding a large range of possibilities to the reported test result. The online calculator met the objective.The authors postulated that the clinical pre-test probability of COVID-19 increases relative to local prevalence of disease plus patient age, known contact, and severity of symptoms. We conducted a small survey on LinkedIn to confirm that hypothesis. We examined results of PPA (Positive Percent Agreement, sensitivity) and NPA (Negative Percent Agreement, specificity) from 73 individual laboratory experiments for molecular tests for SARS-CoV-2as reported to the FIND database,(1) and for selected methods in FDA EUA submissions (2,3). We calculated likelihood ratios to convert pre-test to post-test probability of disease, then further calculated the number of true and false results expected in every ten positive or negative test results, plus an estimate that one in ‘x’ test results is true. We designed an online calculator to create graphics and text to fulfill the objective. A positive or negative test result from one laboratory conveys a higher probability for the presence or absence of COVID-19 than the same result from another laboratory, depending on clinical pre-test probability of disease plus proven method PPA and NPA in each laboratory. Likelihood ratios and confidence intervals provide valuable information but are seldom used in clinical settings. We recommend that testing laboratories verify PPA and NPA, and utilize a tool such as the “Clinician’s Probability Calculator” to verify acceptable test performance and create reports to help guide clinicians and public health staff with estimation of post-test probability of COVID-19 .

Download Full-text

A Bayesian decision support sequential model for severity of illness predictors and intensive care admissions in pneumonia

BMC Medical Informatics and Decision Making ◽

10.1186/s12911-019-1015-5 ◽

2019 ◽

Vol 19 (1) ◽

Cited By ~ 1

Author(s):

Amado Alejandro Baez ◽

Laila Cochon ◽

Jose Maria Nicolas

Keyword(s):

Predictive Value ◽

Severity Of Illness ◽

Positive Test ◽

Bayesian Decision ◽

Likelihood Ratios ◽

Negative Test ◽

Significant Difference ◽

Post Test ◽

Post Test Probability ◽

Test Probability

Abstract Background Community-acquired pneumonia (CAP) is one of the leading causes of morbidity and mortality in the USA. Our objective was to assess the predictive value on critical illness and disposition of a sequential Bayesian Model that integrates Lactate and procalcitonin (PCT) for pneumonia. Methods Sensitivity and specificity of lactate and PCT attained from pooled meta-analysis data. Likelihood ratios calculated and inserted in Bayesian/ Fagan nomogram to calculate posttest probabilities. Bayesian Diagnostic Gains (BDG) were analyzed comparing pre and post-test probability. To assess the value of integrating both PCT and Lactate in Severity of Illness Prediction we built a model that combined CURB65 with PCT as the Pre-Test markers and later integrated the Lactate Likelihood Ratio Values to generate a combined CURB 65 + Procalcitonin + Lactate Sequential value. Results The BDG model integrated a CUBR65 Scores combined with Procalcitonin (LR+ and LR-) for Pre-Test Probability Intermediate and High with Lactate Positive Likelihood Ratios. This generated for the PCT LR+ Post-test Probability (POSITIVE TEST) Posterior probability: 93% (95% CI [91,96%]) and Post Test Probability (NEGATIVE TEST) of: 17% (95% CI [15–20%]) for the Intermediate subgroup and 97% for the high risk sub-group POSITIVE TEST: Post-Test probability:97% (95% CI [95,98%]) NEGATIVE TEST: Post-test probability: 33% (95% CI [31,36%]) . ANOVA analysis for CURB 65 (alone) vs CURB 65 and PCT (LR+) vs CURB 65 and PCT (LR+) and Lactate showed a statistically significant difference (P value = 0.013). Conclusions The sequential combination of CURB 65 plus PCT with Lactate yielded statistically significant results, demonstrating a greater predictive value for severity of illness thus ICU level care.

Download Full-text

The diagnostic accuracy of wrist cineradiography in diagnosing scapholunate dissociation

Journal of Hand Surgery (European Volume) ◽

10.1177/1753193413489056 ◽

2013 ◽

Vol 39 (3) ◽

pp. 263-271 ◽

Cited By ~ 20

Author(s):

G. S. I. Sulkers ◽

N. W. L. Schep ◽

M. Maas ◽

C. M. A. M. van der Horst ◽

J. C. Goslings ◽

...

Keyword(s):

Diagnostic Accuracy ◽

Predictive Value ◽

Diagnostic Value ◽

Inclusion Criteria ◽

Scapholunate Ligament ◽

Scapholunate Dissociation ◽

Post Test ◽

Sensitivity Specificity ◽

Post Test Probability ◽

Test Probability

Ruptures of the scapholunate ligament (SLL) may cause carpal instability, also known as scapholunate dissociation (SLD). SLD may lead to osteoarthritis of the radiocarpal and midcarpal joints. The aim of this retrospective study was to determine the diagnostic value of wrist cineradiography in detecting SLD. All cineradiographic studies made during a 24 year period were retrieved. All patients who underwent the confirmation method (arthroscopy and/or arthrotomy) and cineradiography were included. In total, 84 patients met the inclusion criteria. Sensitivity, specificity, likelihood ratio, positive predictive value, negative predictive value, and diagnostic accuracy for detecting SLD were calculated for radiography and cineradiography. Cineradiography had a sensitivity of 90%, a specificity of 97%, and a diagnostic accuracy of 0.93 in detecting SLD. Radiography had a sensitivity of 81%, a specificity of 80%, and a diagnostic accuracy of 0.81. Cineradiography has a high diagnostic value for diagnosing SLDs. A positive cineradiography markedly increases the post-test probability of SLD.

Download Full-text

Diagnostic accuracy of three clinical dehydration scales: a systematic review

Archives of Disease in Childhood ◽

10.1136/archdischild-2017-313762 ◽

2017 ◽

Vol 103 (4) ◽

pp. 383-388 ◽

Cited By ~ 6

Author(s):

Anna Falszewska ◽

Hania Szajewska ◽

Piotr Dziechciarz

Keyword(s):

Diagnostic Accuracy ◽

Low Income ◽

Positive Likelihood Ratio ◽

High Income ◽

Low Income Countries ◽

Severe Dehydration ◽

Post Test ◽

Sensitivity Specificity ◽

Post Test Probability ◽

Test Probability

ObjectiveTo systematically assess the diagnostic accuracy of the Clinical Dehydration Scale (CDS), the WHO Scale and the Gorelick Scale in identifying dehydration in children with acute gastroenteritis (AGE).DesignThree databases, two registers of clinical trials and the reference lists from identified articles were searched for diagnostic accuracy studies in children with AGE. The index tests were the CDS, WHO Scale and Gorelick Scale, and reference standard was the percentage loss of body weight. The main analysed outcomes were the sensitivity, specificity, positive likelihood ratio (LR) and negative LR.ResultsTen studies were included. In high-income countries, the CDS provided a moderate-to-large increase in the post-test probability of predicting moderate to severe (≥6%) dehydration (positive LR 3.9–11.79), but it was of limited value for ruling it out (negative LR 0.55–0.71). In low-income countries, the CDS showed limited value both for ruling in and ruling out moderate-to-severe dehydration. In both settings, the CDS showed poor diagnostic accuracy for ruling in or out no dehydration (<3%) or some dehydration (3%–6%). The WHO Scale showed no or limited value in assessing dehydration in children with diarrhoea. With one exception, the included studies did not confirm the diagnostic accuracy of the Gorelick Scale.ConclusionLimited evidence suggests that the CDS can help in ruling in moderate-to-severe dehydration (≥6%) in high-income settings only. The WHO and Gorelick Scales are not helpful for assessing dehydration in children with AGE.

Download Full-text

Hip aspiration culture: analysing data from a single operator series investigating periprosthetic joint infection

Journal of Bone and Joint Infection ◽

10.5194/jbji-6-165-2021 ◽

2021 ◽

Vol 6 (6) ◽

pp. 165-170

Author(s):

Connor J. Barker ◽

Alan Marriot ◽

Munir Khan ◽

Tamsin Oswald ◽

Samuel J. Tingle ◽

...

Keyword(s):

Periprosthetic Joint Infection ◽

Joint Infection ◽

Surgical Care ◽

Comparison Results ◽

Significant Difference ◽

Post Test ◽

Single Operator ◽

Sensitivity Specificity ◽

Post Test Probability ◽

Test Probability

Abstract. Introduction: We undertook this study to know the sensitivity, specificity and post-test probabilities of hip aspiration when diagnosing periprosthetic hip infections. We also examined “dry tap” (injection with saline and aspiration) results and aspiration volumes. Methods: This is a retrospective cohort study of patients aspirated for suspected periprosthetic joint infection between July 2012 and October 2016. All aspirations were carried out by one trained surgical care practitioner (SCP). All aspirations followed an aseptic technique and fluoroscopic guidance. Aspiration was compared to tissue biopsy taken at revision. Aspiration volumes were analysed for comparison. Results: Between January 2012 and September 2016, 461 hip aspirations were performed by our SCP. Of these 125 progressed to revision. We calculated sensitivity 59 % (confidence interval (CI) 35 %–82 %) and specificity 94 % (CI 89 %–98 %). Pre-test probability for our cohort was 0.14. Positive post-test probability was 0.59 and negative post-test probability 0.06. Aspiration volume for infected (n=17) and non-infected (n=108) joints was compared and showed no significant difference. Dry taps were experienced five times; in each instance the dry tap agreed with the biopsy result. Conclusions: Our data show that hip aspiration culture is a highly specific investigation for diagnosing infection but that it is not sensitive. Aspiration volume showed no significant difference between infected and non-infected groups. Each time a joint was infiltrated with saline to achieve a result, the result matched tissue sampling.

Download Full-text

“Clinician’s Probability Calculator” to Convert Pre-Test to Post-Test Probability of COVID-19, Based on Method Validation from Each Laboratory

10.20944/preprints202012.0094.v3 ◽

2021 ◽

Author(s):

Zoe Brooks ◽

Saswati Das ◽

Tom Pliura

Keyword(s):

Test Performance ◽

False Negative ◽

Likelihood Ratios ◽

Unique Challenge ◽

Molecular Tests ◽

Percent Agreement ◽

Post Test ◽

Test Result ◽

Post Test Probability ◽

Test Probability

Identifying the SARS-CoV-2 virus has been a unique challenge for the scientific community. In this paper, we discuss a practical solution to help guide clinicians with interpretation of the probability that a positive, or negative, COVID-19 test result indicates an infected person, based on their clinical estimate of pre-test probability of infection.The authors conducted a small survey on LinkedIn to confirm that hypothesis that that the clinical pre-test probability of COVID-19 increases relative to local prevalence of disease plus patient age, known contact, and severity of symptoms. We examined results of PPA (Positive Percent Agreement, sensitivity) and NPA (Negative Percent Agreement, specificity) from 73 individual laboratory experiments for molecular tests for SARS-CoV-2 as reported to the FIND database 1, and for selected methods in FDA EUA submissions2,3. Authors calculated likelihood ratios to convert pre-test to post-test probability of disease and designed an online calculator to create graphics and text to report results. Despite best efforts, false positive and false negative Covid-19 test results are unavoidable4,5. A positive or negative test result from one laboratory has a different probability for the presence of disease than the same result from another laboratory. Likelihood ratios and confidence intervals can convert the physician or other healthcare professional’s clinical estimate of pre-test probability to post-test probability of disease. Ranges of probabilities differ depending on proven method PPA and NPA in each laboratory. We recommend that laboratories verify PPA and NPA and utilize a the “Clinician’s Probability Calculator” to verify acceptable test performance and create reports to help guide clinicians with estimation of post-test probability of COVID-19.

Download Full-text

Stratum-specific likelihood ratios of two versions of the General Health Questionnaire

Psychological Medicine ◽

10.1017/s0033291701003713 ◽

2001 ◽

Vol 31 (3) ◽

pp. 519-529 ◽

Cited By ~ 36

Author(s):

T. A. FURUKAWA ◽

D. P. GOLDBERG, ◽

S. RABE-HESKETH ◽

T. B. ÜSTÜN

Keyword(s):

General Health ◽

Meta Analysis ◽

General Health Questionnaire ◽

Likelihood Ratios ◽

Random Samples ◽

Post Test ◽

Meta Regression ◽

Post Test Probability ◽

Threshold Approach ◽

Test Probability

Background. In other branches of epidemiology, stratum specific likelihood ratios (SSLRs) have been found to be preferable to fixed best threshold approaches to screening instruments. This paper presents SSLRs of GHQ-12 and GHQ-28 and compares the SSLR method with the traditional optimal threshold approach.Methods. Random effects meta-analysis and meta-regression were used to obtain pooled estimates of SSLRs of the two questionnaires for the 15 centres participating in the WHO study of Psychological Problems in General Health Care. We illustrated the use of SSLRs by applying them to random samples of patients from centres with different backgrounds.Results. For developed and urban centres, the estimates of SSLRs were homogeneous for 10 out of 12 strata of the GHQ-12 and GHQ-28. For other centres, the overall results, which were heterogeneous for six out of 12 strata, were deemed the currently available best estimates. When we applied these results to centres with different prevalences of mental disorders and backgrounds, the estimates matched the actually observed closely. These examples showed how the SSLR approach is more informative than the traditional threshold approach.Conclusions. Those working in developed urban settings can use the corresponding SSLRs with reasonable confidence. Those working in non-urban or developing areas may wish to use the overall results, while acknowledging that they must remain less certain until further research can explicate heterogeneity. These SSLRs have been incorporated into nomograms and spreadsheet programmes so that future researchers can swiftly derive the post-test probability for a patient or a group of patients from a pre-test probability and GHQ score.

Download Full-text

Diagnostic accuracy of a fully automated multiplex celiac disease antibody panel for serum and plasma

Clinical Chemistry and Laboratory Medicine (CCLM) ◽

10.1515/cclm-2019-0088 ◽

2019 ◽

Vol 57 (8) ◽

pp. 1207-1217

Author(s):

Jeff Terryberry ◽

Jani Tuomi ◽

Subo Perampalam ◽

Russ Peloquin ◽

Eric Brouwer ◽

...

Keyword(s):

Celiac Disease ◽

Diagnostic Accuracy ◽

Response To Treatment ◽

Likelihood Ratios ◽

Post Test ◽

Biopsy Confirmation ◽

Automated Platform ◽

Disease Antibody ◽

Post Test Probability ◽

Test Probability

Abstract Background An automated multiplex platform using capillary blood can promote greater throughput and more comprehensive studies in celiac disease (CD). Diagnostic accuracy should be improved using likelihood ratios for the post-test probability of ruling-in disease. Methods The Ig_plex™ Celiac Disease Panel on the sqidlite™ automated platform measured IgA and IgG antibodies to tTG and DGP in n = 224 CD serum or plasma samples. Diagnostic accuracy metrics were applied to the combined multiplex test results for several CD populations and compared to conventional single antibody ELISA tests. Results With multiple positive antibody results, the post-test probability for ruling-in untreated and treated CD increased to over 90%. The number of samples positive for more than one antibody also increased in untreated CD to ≥90%. Measurement of all four CD antibodies generate cut-off dependent accuracy profiles that can monitor response to treatment with the gluten-free diet (GFD). Higher positive tTG and DGP antibodies are seen more frequently in confirmed CD without (81%–94%) than with GFD treatment (44%–64%). In CD lacking biopsy confirmation, overall agreement of plasma to serum was ≥98% for all antibodies, and 100% for venous to capillary plasma. Conclusions The Ig_plex Celiac Disease Panel increases the likelihood of confirming CD based on the post-test probability of disease results for multi-reactive markers. Specific positivity profiles and cut-off intervals can be used to monitor GFD treatment and likely disease progression. Using serum, venous and capillary plasma yield comparable and accurate results.

Download Full-text

Atypical Endometrial Hyperplasia and Unexpected Cancers at Final Histology: A Study on Endometrial Sampling Methods and Risk Factors

Diagnostics ◽

10.3390/diagnostics10070474 ◽

2020 ◽

Vol 10 (7) ◽

pp. 474 ◽

Cited By ~ 2

Author(s):

Luca Giannella ◽

Giovanni Delli Carpini ◽

Francesco Sopracordevole ◽

Maria Papiccio ◽

Matteo Serri ◽

...

Keyword(s):

Risk Factors ◽

Endometrial Hyperplasia ◽

Probability Analysis ◽

Secondary Outcome ◽

Endometrial Sampling ◽

Final Histology ◽

Atypical Endometrial Hyperplasia ◽

Post Test ◽

Post Test Probability ◽

Test Probability

Background: Up to 40% of women with atypical endometrial hyperplasia (AEH) can reveal endometrial cancer (EC) at hysterectomy. The pre-operative endometrial sampling method (ESM) and some independent cancer predictors may affect this outcome. The present study aimed to compare the rate of EC at hysterectomy in women with AEH undergoing dilation and curettage (D&C), hysteroscopically-guided biopsy (HSC-bio), or hysteroscopic endometrial resection (HSC-res). The secondary outcome was to compare the reliability of ESMs in women showing independent variables associated with EC. Methods: Two-hundred-and-eight consecutive women with AEH and undergoing hysterectomy between January 2000 and December 2017 were analyzed retrospectively. Based on pre- and post-test probability analysis for EC, three ESMs were compared: D&C, HSC-bio, and HSC-res. Univariate and multivariate analyses were performed to assess risk factors predicting cancer on final histology. Finally, the patient’s characteristics were compared between the three ESM groups. Results: D&C and HSC-bio included 75 women in each group, while HSC-res included 58 women. Forty-nine women (23.6%) revealed cancer at hysterectomy (pre-test probability). Post-test probability analysis showed that HSC-res had the lowest percentage of EC underestimation: HSC-res = 11.6%; HSC-bio = 19.5%; D&C = 35.3%. Patient characteristics showed no significant differences between the three ESMs. Multivariate analysis showed that body mass index ≥40 (Odds Ratio (OR) = 19.75; Confidence Intervals (CI) 2.193–177.829), and age (criterion > 60 years) (OR = 1.055, CI 1.002–1.111) associated significantly with EC. In women with one or both risk factors, post-test probability analysis showed that HSC-res was the only method with a lower EC rate at hysterectomy compared to a pre-test probability of 44.2%: HSC-res = 19.96%; HSC-bio = 53.81%; D&C = 63.12%. Conclusions: HSC-res provided the lowest rate of EC underestimation in AEH, also in women showing EC predictors. These data may be considered for better diagnostic and therapeutic planning of AEH.

Download Full-text