An investigation of new toxicity test method performance in validation studies: 2. comparison of three measures of toxicity test performance

2002 ◽  
Vol 21 (6) ◽  
pp. 313-323 ◽  
Author(s):  
L H Bruner ◽  
G J Carr ◽  
J W Harbell ◽  
R D Curren
Keyword(s):  
Test Performance ◽  
Predictive Value ◽  
Validation Studies ◽  
Toxicity Test ◽  
Prediction Interval ◽  
Test Method ◽  
Data Sets ◽  
Method Performance ◽  
Quantitative Measurements ◽  
Sensitivity Specificity

An area that requires further research is how best to measure test method performance in validation studies and how to set criteria that should be used to judge the adequacy of this performance. The studies reported here were designed to begin an investigation of these questions. Computer simulations were used to generate data sets similar to those that might be obtained from a large validation study. These data were then analysed using three procedures including determination of the 95% prediction interval (PI), calculation of Pearson's correlation coefficient and calculation of the contingent probability statistics (CPS), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). The results of this work suggest that of the three approaches examined, quantitative measurements with calculation of the 95% PI provide the most information to allow discrimination between the performance of several different NTMs. The results also suggest that dividing data sets into positive and negative toxicity classifications followed by the calculation of CPS leads to considerable information loss. This loss of information may be so significant that it is not possible in certain circumstances to distinguish between NTMs that are adequate and those that are not.

An investigation of new toxicity test method performance in validation studies: 3. sensitivity and specificity are not independent of prevalence or distribution of toxicity

2002 ◽  
Vol 21 (6) ◽  
pp. 325-334 ◽  
Author(s):  
L H Bruner ◽  
G J Carr ◽  
J W Harbell ◽  
R D Curren
Keyword(s):  
Positive Predictive Value ◽  
Sensitivity And Specificity ◽  
Test Performance ◽  
Predictive Value ◽  
Validation Studies ◽  
Toxicity Test ◽  
Test Method ◽  
Toxic Substances ◽  
Method Performance ◽  
Sensitivity Specificity

Often, the only measures of toxicity test performance provided in validation studies are the contingent probability statistics (CPS) sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Sensitivity and specificity are generally used in preference to NPV and PPV since NPV and PPV are assumed to vary with changes in prevalence while sensitivity and specificity are assumed to be independent of changes in prevalence. The purpose of the studies reported here was to test whether or not sensitivity and specificity are actually independent of changes in prevalence. Results derived from these studies indicate that sensitivity and specificity vary significantly depending on the prevalence of toxic substances in the set of chemicals being tested. This means sensitivity and specificity should not always be considered constant indicators of toxicity test performance.

An investigation of new toxicity test method performance in validation studies: 1. toxicity test methods that have predictive capacity no greater than chance

2002 ◽  
Vol 21 (6) ◽  
pp. 305-312 ◽  
Author(s):  
L H Bruner ◽  
G J Carr ◽  
J W Harbell ◽  
R D Curren
Keyword(s):  
Predictive Value ◽  
Validation Studies ◽  
Toxicity Test ◽  
High Sensitivity ◽  
Test Methods ◽  
Test Method ◽  
Predictive Capacity ◽  
Worst Case ◽  
Method Performance ◽  
Sensitivity Specificity

An approach commonly used to measure new toxicity test method (NTM) performance in validation studies is to divide toxicity results into positive and negative classifications, and then identify true positive (TP), true negative (TN), false positive (FP) and false negative (FN) results. After this step is completed, the contingent probability statistics (CPS), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) are calculated. Although these statistics are widely used and often the only statistics used to assess the performance of toxicity test methods, there is little specific guidance in the validation literature on what values for these statistics indicate adequate performance. The purpose of this study was to begin developing data-based answers to this question by characterizing the CPS obtained from an NTM whose data have a completely random association with a reference test method (RTM). Determining the CPS of this worst-case scenario is useful because it provides a lower baseline from which the performance of an NTM can be judged in future validation studies. It also provides an indication of relationships in the CPS that help identify random or near-random relationships in the data. The results from this study of randomly associated tests show that the values obtained for the statistics vary significantly depending on the cut-offs chosen, that high values can be obtained for individual statistics, and that the different measures cannot be considered independently when evaluating the performance of an NTM. When the association between results of an NTM and RTM is random the sum of the complementary pairs of statistics (sensitivity+ specificity, NPV+PPV) is approximately 1, and the prevalence (i.e., the proportion of toxic chemicals in the population of chemicals) and PPV are equal. Given that combinations of high sensitivity–low specificity or low specificity–high sensitivity (i.e., the sum of the sensitivity and specificity equal to approximately 1) indicate lack of predictive capacity, an NTM having these performance characteristics should be considered no better for predicting toxicity than by chance alone.

scholarly journals Diagnostic Performance of Plasma Gastrin Concentration for the Diagnosis of Type 1 Abomasal Ulcer in Water Buffalo: A Preliminary Case Control Study

2021 ◽  
Author(s):  
Syed Ashaq Hussain ◽  
S K Uppal ◽  
N K Sood
Keyword(s):  
Test Performance ◽  
Predictive Value ◽  
Histopathological Examination ◽  
Clinical Signs ◽  
Plasma Gastrin ◽  
Roc Curve Analysis ◽  
Optimal Value ◽  
Sensitivity Specificity ◽  
Control Study

Abstract The type 1 abomasal ulcer (AU1) does not have specific clinical signs, and there is a need to identify some early biochemical markers for diagnosis of AU1 in cattle and buffaloes. Plasma gastrin is reported to reflect the gastric mucosa damage but its utility for diagnosis of AU1 in buffaloes has not been evaluated. The objective of this study was to study the test performance of plasma gastrin to distinguish healthy buffaloes and buffaloes with AU1. Twenty-three buffaloes with AU1 and six buffaloes without abomasal ulcer were used. Blood samples were collected from the buffaloes, slaughtered in a buffalo specific slaughter house, for estimation of plasma gastrin. After slaughter abomasa were examined for presence of AU1 and were confirmed by histology. The mean plasma gastrin concentration of ulcer positive buffaloes was significantly (p<0.05) higher than the ulcer negative buffaloes. The ROC curve analysis suggested optimal value of plasma gastrin for diagnosis of AU1 was 106.2 pg/ml. Since the abomasal ulcer negative animals were established to be ulcer negative on histopathological examination we consider the values of gastrin valid for detection of abomasal ulceration in buffaloes. The sensitivity, specificity, positive predictive value and negative predictive value of plasma gastrin to diagnose AU1 in buffalo were 78.3, 100, 100 and 69.9, respectively.

scholarly journals Assessment of Oropharyngeal Specimens for Discontinuation of Transmission-Based COVID-19 Precautions

2020 ◽  
Vol 7 (9) ◽  
Author(s):  
Joshua A Barocas ◽  
Miriam Komaromy ◽  
Deeanna Haidar ◽  
Tamar F Barlam ◽  
Beverley L Orr ◽  
...  
Keyword(s):  
Negative Predictive Value ◽  
Retrospective Review ◽  
Test Performance ◽  
Predictive Value ◽  
Reference Standard ◽  
Paired Samples ◽  
Oropharyngeal Swab ◽  
Sensitivity Specificity

Abstract We compared oropharyngeal swab test performance with nasopharyngeal testing for discontinuation of transmission-based COVID-19 precautions. We performed a retrospective review of confirmed COVID-19-positive patients who received paired nasopharyngeal and oropharyngeal SARS-CoV-2 tests for clearance from isolation from May 4, 2020, to May 26, 2020. Using nasopharyngeal swabs as the reference standard, we calculated the sensitivity, specificity, and negative predictive value of oropharyngeal swabs. We also calculated the kappa between the 2 tests. A total of 189 paired samples were collected from 74 patients. Oropharyngeal swab sensitivity was 38%, specificity was 87%, and negative predictive value was 70%. The kappa was 0.25. Our study suggests that oropharyngeal swabs are inferior to nasopharyngeal swabs for test-based clearance from COVID-19 isolation.

Gastric cancer detection using the serum pepsinogen test method

2021 ◽  
pp. 030089162110149
Author(s):  
Dragan Trivanovic ◽  
Stjepko Plestina ◽  
Lorena Honovic ◽  
Renata Dobrila-Dintinjana ◽  
Jelena Vlasic Tanaskovic ◽  
...  
Keyword(s):  
Gastric Cancer ◽  
Early Detection ◽  
Predictive Value ◽  
Area Under The Curve ◽  
Test Method ◽  
Operating Characteristics ◽  
Roc Curve Analysis ◽  
Laboratory Assessment ◽  
Pepsinogen I ◽  
Sensitivity Specificity

Background: Gastric cancer (GC) is the eighth most common cause of cancer deaths in Croatia and one of the most common causes of cancer deaths worldwide. A reliable diagnostic tool for the early detection of GC is essential. Objective: We previously suggested a pepsinogen test method to reduce the mortality from GC by allowing early detection. Here, we report an updated analysis from a prospective single-center clinical study to evaluate the sensitivity and specificity of the pepsinogen test method and to determine whether this test can be used as a part of routine laboratory assessment of high-risk patients. Methods: We present mature data of the pepsinogen test method in the Croatian population after a median follow-up of 36 months. Statistical analyses were performed using a Mann-Whitney U test, multiple logistic regression, and receiver operating characteristics (ROC) to evaluate the predictive power of the assayed biomarkers. Results: Of the 116 patients, 25 patients had GC and 91 demonstrated a nonmalignant pathology based on tissue biopsy. Cutoff values were pepsinogen I ⩽70 and pepsinogen I/II ratio ⩽3.0. Using ROC curve analysis, the accuracy, sensitivity, specificity, positive predictive value, and negative predictive value were determined to be 87.22%, 78.12%, 90.10%, 71.43%, and 92.86%, respectively, for the diagnosis of GC. The area under the curve was 0.700 (95% confidence interval 0.57–0.83). Conclusion: Pepsinogen tests are valuable for screening a population in need of further diagnosis and could help to avoid unnecessary invasive endoscopic procedures.

scholarly journals MPT64 antigen detection test improves diagnosis of pediatric extrapulmonary tuberculosis in Mbeya, Tanzania

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Erlend Grønningen ◽  
Marywinnie Nanyaro ◽  
Lisbet Sviland ◽  
Esther Ngadaya ◽  
William Muller ◽  
...  
Keyword(s):  
Test Performance ◽  
Predictive Value ◽  
Extrapulmonary Tuberculosis ◽  
Final Diagnosis ◽  
Antigen Detection ◽  
Diagnostic Challenge ◽  
Routine Diagnostics ◽  
Resource Limited ◽  
Resource Setting ◽  
Sensitivity Specificity

AbstractPediatric extrapulmonary tuberculosis (EPTB) is a diagnostic challenge. A new immunochemistry based MPT64 antigen detection test has shown improved sensitivity compared to current laboratory tests. The aim of this study was to implement and validate the test performance in a resource limited African setting. Presumptive pediatric (0–18 y) EPTB patients were prospectively enrolled at Mbeya Zonal Referral Hospital, and followed to the end of treatment or until a final diagnosis was reached. Specimens from suspected sites of infection were subject to routine diagnostics, GeneXpert MTB/RIF assay and the MPT64 test. The performance of the tests was assessed using mycobacterial culture as well as a composite reference standard. 30 patients were categorized as TB cases, 31 as non-TB cases and 2 were uncategorized. In the TB group, the three most common infections were adenitis (30%), peritonitis (30%) and meningitis (20%). The sensitivity, specificity, positive predictive value, negative predictive value and accuracy of the MPT64 test was 92%, 88%, 87%, 92% and 90%, respectively. Mortality was equally high among TB/non-TB cases (23% vs 21%), and malnutrition was the main comorbidity among TB cases. The MPT64 test was implementable in the routine diagnostics in a low-resource setting and improved the diagnosis of pediatric EPTB.

scholarly journals Highly Accurate and Reproducible Diagnosis of Peanut Allergy Using Epitope Mapping

2020 ◽  
Author(s):  
Paul Kearney ◽  
Robert Getts ◽  
Clive Hayward ◽  
David Luta ◽  
Alex Porter ◽  
...  
Keyword(s):  
Likelihood Ratio ◽  
Test Performance ◽  
Predictive Value ◽  
Peanut Allergy ◽  
Skin Prick Testing ◽  
Positive Likelihood Ratio ◽  
Negative Likelihood Ratio ◽  
Component Testing ◽  
The Subject ◽  
Sensitivity Specificity

AbstractBackgroundMisdiagnosis of peanut allergy is a significant clinical challenge. Here, a novel diagnostic blood-based test using a Bead-Based Epitope Assay (“peanut BBEA”) has been developed on the LEAP cohort and then independently validated on the CoFAR2 and POISED cohorts.MethodsDevelopment of the peanut BBEA followed the National Academy of Medicine’s established guidelines with discovery performed on 133 subjects from the non-interventional arm of the LEAP trial and an independent validation performed on 81 subjects from the CoFAR2 study and 84 subjects from the POISED study. All subject samples were analyzed using the BBEA methodology. The peanut BBEA test measures levels of two Ara h 2 epitopes and compares their combination to a pre=specified threshold. If the combination of the two epitope levels is at or below the threshold, then the subject is ruled “Not Allergic”, otherwise the subject is ruled “Allergic”.All allergic diagnoses were OFC confirmed and subjects’ ages were 7-55 years.ResultsIn validation on the CoFAR2 and POISED cohorts, the peanut BBEA test had a combined sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, negative likelihood ratio and accuracy of 91%, 95%, 95%, 91%, 18.2, 0.09 and 93%, respectively.ConclusionThe peanut BBEA test performance in validation demonstrated overall high accuracy and compared very favorably with existing diagnostic tests for peanut allergy including skin prick testing, peanut sIgE and peanut component testing.

Proteinuria: accuracy and precision of laboratory diagnosis by dip-stick analysis.

1978 ◽  
Vol 24 (11) ◽  
pp. 1934-1939 ◽  
Author(s):  
G P James ◽  
D E Bee ◽  
J B Fuller
Keyword(s):  
Total Protein ◽  
Predictive Value ◽  
Laboratory Diagnosis ◽  
Urinary Protein ◽  
Quantitative Measurements ◽  
Accuracy And Precision ◽  
Sensitivity Specificity

Abstract We studied and compared results of two urine dip-stick procedures for protein with those of quantitative measurements of total protein and albumin in urine. Sensitivity, specificity, and predictive value of "normal" and "abnormal" dip-stick results, assay efficiency, technologist precision, and method association were determined. Precision was poor, attributable largely to variation within each technologist. Relative to a defined measure of precision, technologist performance appears to be uniform within each product. Patients' urinary protein concentrations of less than 200 mg/liter can be distinguished from concentrations greater than 3000 mg/liter with confidence by dip-stick procedures. Urin samples with low albumin/total protein ratios were frequently falsely negative by dip-stick assays.

scholarly journals Evaluation of different platforms for the detection of anti-SARS coronavirus-2 antibodies, Thailand

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Hatairat Lerdsamran ◽  
Anek Mungaomklang ◽  
Sopon Iamsirithaworn ◽  
Jarunee Prasertsopon ◽  
Kriengkrai Prasert ◽  
...  
Keyword(s):  
At Risk ◽  
Test Performance ◽  
Predictive Value ◽  
Disease Diagnosis ◽  
Immunofluorescence Assay ◽  
Antibody Detection ◽  
Immunological Methods ◽  
Serum Samples ◽  
Previous Infection ◽  
Sensitivity Specificity

Abstract Background Antibodies against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) help determine previous infection in individuals, regardless of whether they are asymptomatic or symptomatic. The detection of antibodies serves several purposes, including supporting other assays for disease diagnosis, conducting seroepidemiological studies, and evaluating vaccines. Many platforms of immunological methods for anti-SARS-CoV-2 antibody detection and their performance require validation. Methods This study evaluated the test performance of three autoanalyzer-based assays (Architect IgG, Vitros IgG, and Vitros total Ig) and one manual ELISA (Wantai total Ig) against a microneutralization (microNT) assay on the detection of SARS-CoV-2 antibodies. Furthermore, an indirect immunofluorescence assay verified the discordant results between the microNT and commercial assays. The test sensitivity, specificity, positive predictive value, and negative predictive value were determined based on four groups of 1005 serum samples: 102 COVID-19 prepandemic sera, 45 anti-SARS-CoV-2 positive sera, 366 sera of people at risk, and 492 sera of citizens returning from countries with a high prevalence of infection. Results The analyses as a whole showed that the performance of these commercial assays was comparable. Each group was also analysed separately to gain further insight into test performance. The Architect did not detect two positive sera of people at risk (prevalence of infection 0.55%). The other methods correctly identified these two positive sera but yielded varying false-positive results. The group of returning travellers with an infection rate of 28.3% (139 of 492) better differentiated the test performance of individual assays. Conclusions High-throughput Architect and Vitros autoanalyzers appear appropriate for working on large sample sizes in countries that can afford the cost. The Wantai ELISA, while requiring more individual time and technical skill, may provide reliable results at a lower cost. The selection of assays will depend on the laboratory facilities and feasibility.

scholarly journals Prospective evaluation of mannan and anti-mannan antibodies for diagnosis of invasive Candida infections in patients with neutropenic fever

2009 ◽  
Vol 58 (5) ◽  
pp. 606-615 ◽  
Author(s):  
Michael Ellis ◽  
Basel Al-Ramadi ◽  
Roos Bernsen ◽  
Jorgen Kristensen ◽  
Hussain Alizadeh ◽  
...  
Keyword(s):  
Febrile Neutropenia ◽  
Test Performance ◽  
Predictive Value ◽  
Antibody Test ◽  
Diagnostic Strategy ◽  
European Organization ◽  
Mycological Diagnosis ◽  
Patients At Risk ◽  
High Level ◽  
Sensitivity Specificity

The diagnostic performance and usefulness of the Platelia antigen and antibody test (Bio-Rad) was investigated in a prospective study of haematological patients at risk for invasive Candida infections. Among 100 patients, 86 were eligible, of whom invasive candidiasis (IC) occurred in 12 (14 %), according to the criteria of the European Organization for Research and Treatment of Cancer/Mycoses Study Group. These included candidaemia due to Candida albicans (one patient) or Candida tropicalis (four patients), and hepatosplenic candidiasis (seven patients). The comparator group of 74 patients included 50 with febrile neutropenia alone and 24 with mould infections. A strategy was developed to determine diagnostic cut-offs from receiver operating characteristic curves with maximal sensitivity and, given this sensitivity, maximal specificity, both being greater than 0. In this patient population, these values were 0.25 ng ml−1 for mannan (M) and 2.6 arbitrary units ml−1 for anti-mannan (AM), which are lower than those recommended by the manufacturer. All patients developed at least one positive diagnostic M or AM result during the 10 days of persistent febrile neutropenia (PFN). The optimal overall performance was found when two consecutive positive tests for both M and AM were used [sensitivity, specificity, positive predictive value and negative predictive value (NPV) (95 % confidence intervals) of 0.73 (0.39–0.94), 0.80 (0.69–0.89), 0.36 (0.17–0.59) and 0.95 (0.86–0.99), respectively]. There was a positive correlation of M with β-d-glucan (r=0.28, P=0.01). The first positive M test was found up to a mean±sd of 8.8±8.5 (range 2–23) days prior to a clinical/mycological diagnosis of IC. Day-to-day variation in quantitative M levels was high. High-level AM responses were delayed until leucopenia resolved. The low specificities of the test performance may have been due to some of the comparator patients having subclinical Candida infections as evidenced by the high incidence of colonization among them (60 % had a colonization index of ≥0.5). The high NPVs suggest that the tests may be particularly useful in excluding IC. It is feasible to explore the use of serial measurements of M and AM as part of a broader diagnostic strategy for selecting PFN patients to receive antifungal drug therapy.

Sign in / Sign up

Export Citation Format

Share Document