An investigation of new toxicity test method performance in validation studies: 1. toxicity test methods that have predictive capacity no greater than chance

2002 ◽  
Vol 21 (6) ◽  
pp. 305-312 ◽  
Author(s):  
L H Bruner ◽  
G J Carr ◽  
J W Harbell ◽  
R D Curren
Keyword(s):  
Predictive Value ◽  
Validation Studies ◽  
Toxicity Test ◽  
High Sensitivity ◽  
Test Methods ◽  
Test Method ◽  
Predictive Capacity ◽  
Worst Case ◽  
Method Performance ◽  
Sensitivity Specificity

An approach commonly used to measure new toxicity test method (NTM) performance in validation studies is to divide toxicity results into positive and negative classifications, and then identify true positive (TP), true negative (TN), false positive (FP) and false negative (FN) results. After this step is completed, the contingent probability statistics (CPS), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) are calculated. Although these statistics are widely used and often the only statistics used to assess the performance of toxicity test methods, there is little specific guidance in the validation literature on what values for these statistics indicate adequate performance. The purpose of this study was to begin developing data-based answers to this question by characterizing the CPS obtained from an NTM whose data have a completely random association with a reference test method (RTM). Determining the CPS of this worst-case scenario is useful because it provides a lower baseline from which the performance of an NTM can be judged in future validation studies. It also provides an indication of relationships in the CPS that help identify random or near-random relationships in the data. The results from this study of randomly associated tests show that the values obtained for the statistics vary significantly depending on the cut-offs chosen, that high values can be obtained for individual statistics, and that the different measures cannot be considered independently when evaluating the performance of an NTM. When the association between results of an NTM and RTM is random the sum of the complementary pairs of statistics (sensitivity+ specificity, NPV+PPV) is approximately 1, and the prevalence (i.e., the proportion of toxic chemicals in the population of chemicals) and PPV are equal. Given that combinations of high sensitivity–low specificity or low specificity–high sensitivity (i.e., the sum of the sensitivity and specificity equal to approximately 1) indicate lack of predictive capacity, an NTM having these performance characteristics should be considered no better for predicting toxicity than by chance alone.

An investigation of new toxicity test method performance in validation studies: 2. comparison of three measures of toxicity test performance

2002 ◽  
Vol 21 (6) ◽  
pp. 313-323 ◽  
Author(s):  
L H Bruner ◽  
G J Carr ◽  
J W Harbell ◽  
R D Curren
Keyword(s):  
Test Performance ◽  
Predictive Value ◽  
Validation Studies ◽  
Toxicity Test ◽  
Prediction Interval ◽  
Test Method ◽  
Data Sets ◽  
Method Performance ◽  
Quantitative Measurements ◽  
Sensitivity Specificity

An area that requires further research is how best to measure test method performance in validation studies and how to set criteria that should be used to judge the adequacy of this performance. The studies reported here were designed to begin an investigation of these questions. Computer simulations were used to generate data sets similar to those that might be obtained from a large validation study. These data were then analysed using three procedures including determination of the 95% prediction interval (PI), calculation of Pearson's correlation coefficient and calculation of the contingent probability statistics (CPS), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). The results of this work suggest that of the three approaches examined, quantitative measurements with calculation of the 95% PI provide the most information to allow discrimination between the performance of several different NTMs. The results also suggest that dividing data sets into positive and negative toxicity classifications followed by the calculation of CPS leads to considerable information loss. This loss of information may be so significant that it is not possible in certain circumstances to distinguish between NTMs that are adequate and those that are not.

An investigation of new toxicity test method performance in validation studies: 3. sensitivity and specificity are not independent of prevalence or distribution of toxicity

2002 ◽  
Vol 21 (6) ◽  
pp. 325-334 ◽  
Author(s):  
L H Bruner ◽  
G J Carr ◽  
J W Harbell ◽  
R D Curren
Keyword(s):  
Positive Predictive Value ◽  
Sensitivity And Specificity ◽  
Test Performance ◽  
Predictive Value ◽  
Validation Studies ◽  
Toxicity Test ◽  
Test Method ◽  
Toxic Substances ◽  
Method Performance ◽  
Sensitivity Specificity

Often, the only measures of toxicity test performance provided in validation studies are the contingent probability statistics (CPS) sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Sensitivity and specificity are generally used in preference to NPV and PPV since NPV and PPV are assumed to vary with changes in prevalence while sensitivity and specificity are assumed to be independent of changes in prevalence. The purpose of the studies reported here was to test whether or not sensitivity and specificity are actually independent of changes in prevalence. Results derived from these studies indicate that sensitivity and specificity vary significantly depending on the prevalence of toxic substances in the set of chemicals being tested. This means sensitivity and specificity should not always be considered constant indicators of toxicity test performance.

scholarly journals Use of β-D-glucan in diagnosis of suspected Pneumocystis jirovecii pneumonia in adults with HIV infection

2021 ◽  
pp. 095646242110222
Author(s):  
Thomas Juniper ◽  
Chris P Eades ◽  
Eliza Gil ◽  
Harriet Fodder ◽  
Killian Quinn ◽  
...  
Keyword(s):  
Predictive Value ◽  
Diagnostic Tests ◽  
Elevated Serum ◽  
Clinical Information ◽  
High Sensitivity ◽  
Pneumocystis Pneumonia ◽  
Pneumocystis Jirovecii Pneumonia ◽  
Alternative Diagnosis ◽  
Positive Threshold ◽  
Sensitivity Specificity

Objectives: An elevated serum (1-3)-β-D-glucan (BDG) concentration has high sensitivity for a diagnosis of Pneumocystis pneumonia (PCP) in people with HIV (PWH). At the current manufacturer-recommended positive threshold of 80 pg/mL (Fungitell), specificity for PCP is variable and other diagnostic tests are required. We evaluated the utility of serum BDG for diagnosis of suspected PCP in PWH at three inner-London hospitals to determine BDG concentrations for diagnosis and exclusion of PCP. Methods: From clinical case records, we abstracted demographic and clinical information and categorised patients as having confirmed or probable PCP, or an alternative diagnosis. We calculated sensitivity, specificity and positive predictive value (PPV) of serum BDG concentrations >400 pg/mL and negative predictive value (NPV) of BDG <80 pg/mL. Results: 76 patients were included; 29 had laboratory-confirmed PCP, 17 had probable PCP and 30 had an alternative diagnosis. Serum BDG >400 pg/mL had a sensitivity of 83%, specificity of 97% and PPV 97% for diagnosis of PCP; BDG <80 pg/mL had 100% NPV for exclusion of PCP. Conclusions: In PWH with suspected PCP, BDG <80 pg/mL excludes a diagnosis of PCP, whereas BDG concentrations >400 pg/mL effectively confirm the diagnosis. Values 80–400 pg/mL should prompt additional diagnostic tests.

A Negative Antinuclear Antibody Does Not Indicate Autoantibody Negativity in Myositis: Role of Anticytoplasmic Antibody as a Screening Test for Antisynthetase Syndrome

2016 ◽  
Vol 44 (2) ◽  
pp. 223-229 ◽  
Author(s):  
Rohit Aggarwal ◽  
Namrata Dhillon ◽  
Noreen Fertig ◽  
Diane Koontz ◽  
Zengbiao Qi ◽  
...  
Keyword(s):  
Lupus Erythematosus ◽  
Predictive Value ◽  
Antinuclear Antibody ◽  
High Sensitivity ◽  
Epithelial Cell Line ◽  
Antisynthetase Syndrome ◽  
Control Groups ◽  
Serum Samples ◽  
Cytoplasmic Staining ◽  
Sensitivity Specificity

Objective.To evaluate the utility of anticytoplasmic autoantibody (anti-CytAb) in antisynthetase antibody–positive (anti-SynAb+) patients.Methods.Anti-SynAb+ patients were evaluated for antinuclear antibody (ANA) and anti-CytAb [cytoplasmic staining on indirect immunofluorescence (IIF)] positivity. Anti-SynAb+ patients included those possessing anti-Jo1 and other antisynthetase autoantibodies. Control groups included scleroderma, systemic lupus erythematosus, Sjögren syndrome, rheumatoid arthritis, and healthy subjects. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy of anti-CytAb, and ANA were assessed. Anti-CytAb and ANA testing was done by IIF on human epithelial cell line 2, both reported on each serum sample without knowledge of the clinical diagnosis or final anti-SynAb results.Results.Anti-SynAb+ patients (n = 202; Jo1, n = 122; non-Jo1, n = 80) between 1985–2013 with available serum samples were assessed. Anti-CytAb showed high sensitivity (72%), specificity (89%), NPV (95%), and accuracy (86%), but only modest PPV (54%) for anti-SynAb positivity. In contrast, ANA showed only modest sensitivity (50%) and poor specificity (6%), PPV (9%), NPV (41%), and accuracy (12%). Positive anti-CytAb was significantly greater in the anti-SynAb+ patients than ANA positivity (72% vs 50%, p < 0.001), and 81/99 (82%) ANA-negative patients in the anti-SynAb+ cohort had positive anti-CytAb. In contrast, the control groups showed high rates for ANA positivity (93.5%), but very low rates for anti-CytAb positivity (11.5%). Combining anti-CytAb or Jo1 positivity showed high sensitivity (92%) and specificity (89%) for identification of anti-SynAb+ patients.Conclusion.Assessing patients for anti-CytAb serves as an excellent screen for anti-SynAb+ patients using simple IIF. Cytoplasmic staining should be assessed and reported for patients suspected of having antisynthetase syndrome and a negative ANA should not be used to exclude this diagnosis.

scholarly journals Adaptation of the Systematic Review Framework to the Assessment of Toxicological Test Methods: Challenges and Lessons Learned With the Zebrafish Embryotoxicity Test

2019 ◽  
Vol 171 (1) ◽  
pp. 56-68
Author(s):  
Martin L Stephens ◽  
Sevcan Gül Akgün-Ölmez ◽  
Sebastian Hoffmann ◽  
Rob de Vries ◽  
Burkhard Flick ◽  
...  
Keyword(s):  
Systematic Review ◽  
Developmental Toxicity ◽  
Test Methods ◽  
Lessons Learned ◽  
Hazard Identification ◽  
Test Method ◽  
Main Study ◽  
Systematic Review Methodology ◽  
Method Performance ◽  
Toxicological Test

AbstractSystematic review methodology is a means of addressing specific questions through structured, consistent, and transparent examinations of the relevant scientific evidence. This methodology has been used to advantage in clinical medicine, and is being adapted for use in other disciplines. Although some applications to toxicology have been explored, especially for hazard identification, the present preparatory study is, to our knowledge, the first attempt to adapt it to the assessment of toxicological test methods. As our test case, we chose the zebrafish embryotoxicity test (ZET) for developmental toxicity and its mammalian counterpart, the standard mammalian prenatal development toxicity study, focusing the review on how well the ZET predicts the presence or absence of chemical-induced prenatal developmental toxicity observed in mammalian studies. An interdisciplinary team prepared a systematic review protocol and adjusted it throughout this piloting phase, where needed. The final protocol was registered and will guide the main study (systematic review), which will execute the protocol to comprehensively answer the review question. The goal of this preparatory study was to translate systematic review methodology to the assessment of toxicological test method performance. Consequently, it focused on the methodological issues encountered, whereas the main study will report substantive findings. These relate to numerous systematic review steps, but primarily to searching and selecting the evidence. Applying the lessons learned to these challenges can improve not only our main study, but may also be helpful to others seeking to use systematic review methodology to compare toxicological test methods. We conclude with a series of recommendations that, if adopted, would help improve the quality of the published literature, and make conducting systematic reviews of toxicological studies faster and easier over time.

scholarly journals XPERT MYCOBACTERIUM TUBERCULOSIS/RIFAMPICIN ASSAY: A BOON IN TUBERCULOSIS DIAGNOSTICS

2016 ◽  
Vol 9 (5) ◽  
pp. 225 ◽  
Author(s):  
Sevitha Bhat ◽  
Kavya Ramamurthy ◽  
Shalini Shenoy ◽  
Aseem Rangnekar
Keyword(s):  
Mycobacterium Tuberculosis ◽  
Predictive Value ◽  
High Sensitivity ◽  
Mortality And Morbidity ◽  
Indicator Tube ◽  
Promising Tool ◽  
Causes Of Mortality ◽  
Conventional Microscopy ◽  
Tb Diagnostics ◽  
Sensitivity Specificity

ABSTRACTObjectives: Mycobacterium tuberculosis (MTB) remains one of the most significant causes of mortality and morbidity in developing countriesespecially India. India has the highest burden of TB, with an estimated incidence figure of 2.1 million cases out of the 9 million cases of TB globally.Diagnosis of TB relies on conventional microscopy and culture with drawbacks related to sensitivity, specificity, turn around time (TAT). The aim ofthis study was to evaluate the performance of Xpert MTB/rifampicin (RIF) assay (GX) for MTB detection in pulmonary and extrapulmonary clinicalsamples.Methods: A total of 209 clinical specimens (182: pulmonary and 27: extrapulmonary) were processed using auramine smear, culture by mycobacteriagrowth indicator tube and GenXpert.Results: The sensitivity of GenXpert was 62.63% for pulmonary and 55% for extrapulmonary samples. The sensitivity and specificity of GX were100% for the smear positive cases. The sensitivity, specificity, positive predictive value, and negative predictive value of the GX for smear negativecases were 67.8%, 97.5%, 90.4%, and 89.6%, respectively. RIF resistance was detected in 3.8% the samples.Conclusion: GenXpert, with short TAT, high sensitivity, specificity and less technical expertise required is a promising tool in TB diagnostics for thefuture.Keywords: GenXpert, Tuberculosis diagnosis, Molecular method, Rifampicin resistance.

scholarly journals Role of MRI in the Diagnosis of Injury to the Lisfranc Ligament Complex

2018 ◽  
Vol 3 (3) ◽  
pp. 2473011418S0029
Author(s):  
Christopher Kreulen ◽  
Eric Giza ◽  
Eva Escobedo ◽  
Cyrus Bateni ◽  
Michael Doherty
Keyword(s):  
Magnetic Resonance ◽  
Mr Imaging ◽  
Negative Predictive Value ◽  
Predictive Value ◽  
High Sensitivity ◽  
Lisfranc Injury ◽  
Radiology Reports ◽  
Lisfranc Ligament ◽  
Sensitivity Specificity

Category: Sports Introduction/Purpose: Subtle Lisfranc ligamentous injuries are difficult to diagnose and magnetic resonance is becoming a useful tool. The purpose of this study is to evaluate the efficacy of magnetic resonance (MR) imaging for the diagnosis of injuries of the Lisfranc ligament complex. Methods: The radiology database was searched between Jan 1, 2010 and Mar 10, 2015 to identify patients over the age of 18 years who had MR imaging of the foot for suspected injury of the Lisfranc ligament complex. MR images were reviewed by 2 fellowship trained musculoskeletal radiologists, whom were blinded to the original radiology reports. Findings were categorized as: no injury or injury present. Injury was deemed to be present if 2 of the 3 components of C1-M2 ligament showed disruption or signal alterations on T1 and T2 weighted images. Disagreements were resolved by consensus. Correlation was made with surgical findings whenever performed. In patients not undergoing surgery, the presence or absence of injury was determined by clinical examination performed by an orthopedic surgeon and follow-up. Sensitivity, specificity, positive predictive value(PPV), and negative predictive value (NPV) of MR for diagnosis of Lisfranc ligament complex injury was determined. Results: Of 60 patients, 9 were excluded due to a lack of follow-up. Lisfranc injury was determined to be present on MR in 26 patients and 18 underwent surgery. Injury was confirmed in 16, and 2 were intact. 2 patients underwent closed reduction and were clinically determined to be injured. 6 of the injured 26 patients were sprained and not injured/torn on clinical evaluation. Of the 25 patients determined to have no injury on MR, 24 were intact clinically. 1 patient had a Lisfranc injury on follow-up. Sensitivity, specificity, PPV and NPV of MR for detection of significant Lisfranc injury were 94.7% (CI: 73.9% to 99.9%), 75% (CI: 56.6% to 88.5%), 69.2% (CI: 55% to 80.5%) and 96% (CI: 77.9% to 99.4%) respectively. Conclusion: MR has a high sensitivity and negative predictive value for diagnosis of injury to the Lisfranc ligament complex. MR of the foot should be considered in patients with clinical suspicion of injury to the Lisfranc ligament complex, and it is highly accurate in excluding such injuries.

Isolated demyelinating syndromes: comparison of CSF oligoclonal bands and different MR imaging criteria to predict conversion to CDMS

2001 ◽  
Vol 7 (6) ◽  
pp. 359-363 ◽  
Author(s):  
M Tintoré ◽  
A Rovira ◽  
L Brieva ◽  
E Grivé ◽  
R Jardí ◽  
...  
Keyword(s):  
Mr Imaging ◽  
Positive Predictive Value ◽  
Negative Predictive Value ◽  
Predictive Value ◽  
High Sensitivity ◽  
Oligoclonal Bands ◽  
High Negative Predictive Value ◽  
Csf Oligoclonal Bands ◽  
Sensitivity Specificity

Aim of the study: To evaluate and compare the capacity of oligoclonal bands (OB) and three sets of MR imaging criteria to predict the conversion of clinically isolated syndromes (CIS) to clinically definite multiple sclerosis (CDMS). Patients and methods: One hundred and twelve patients with CIS were prospectively studied with MR imaging and determination of OB. Based on the clinical follow-up (conversion or not conversion to CDMS), we calculated the sensitivity, specificity accuracy, positive and negative predictive value of the OB, and MR imaging criteria proposed by Paty et al, Fazekas et al and Barkhof et al. Results: CDMS developed in 26 (23.2%) patients after a mean follow-up of 31 months (range 12-62). OB were positive in 70 (62.5%) patients and were associated with a higher risk of developing CDMS. OB showed a sensitivity of 81%, specificity of 43%, accuracy of 52%, positive predictive value (PPV) of 30% and negative predictive value (NPV) of 88%. Paty and Fazekas criteria showed the same results with a sensitivity of 77%, specificity of 51%, accuracy of 57%, positive predictive value of 32% and negative predictive value of 88%. Barkhof criteria showed a sensitivity of 65%, specificity of 70%, accuracy of 69%, PPV of 40% and NPV of 87%. The greatest accuracy was achieved when patients with positive OB and three or four Barkhof's criteria were selected. Conclusions: We observed a high prevalence of OB in CIS. OB and MR imaging (Paty's and Fazekas' criteria) have high sensitivity. Barkhof's criteria have a higher specificity. Both OB and MR imaging criteria have a high negative predictive value.

Sensitivity, Specificity, and Positive Predictive Value of Technetium 99-HMPAO SPECT in Discriminating Alzheimer's Disease from other Dementias

1997 ◽  
Vol 10 (1) ◽  
pp. 15-21 ◽  
Author(s):  
Donna L. Masterman ◽  
Mario F. Mendez ◽  
Lynn A. Fairbanks ◽  
Jeffrey L. Cummings
Keyword(s):  
Alzheimer’S Disease ◽  
Alzheimer's Disease ◽  
Positive Predictive Value ◽  
Predictive Value ◽  
Single Photon ◽  
Photon Emission ◽  
High Sensitivity ◽  
Hmpao Spect ◽  
Clinical Diagnoses ◽  
Sensitivity Specificity

Investigators have reported high sensitivity and specificity values for single photon emission computerized tomography (SPECT) when distinguishing Alzheimer's disease (AD) patients from normal elderly controls or from selected patient groups. The role of SPECT in identifying AD among unselected patients with memory complaints requires investigation. We examined 139 consecutive patients with 99Tc-HMPAO SPECT. NINCDS-ADRDA diagnoses were determined blind to SPECT results, and scans were read and classified by visual inspection blind to clinical diagnoses. Bilateral temporoparietal hypoperfusion (TP) occurred in 75% of probable, 65% of possible, and 45% of unlikely AD patients, yielding a sensitivity of 75% and a specificity of 52% when comparing probable AD versus unlikely AD groups. A positive predictive value of 78% was obtained based on a 69% prevalence of AD in our total clinic population. Patients with false-positive results included a variety of dementing illnesses; all patients with bilateral hypoperfusion had dementia. A pattern of TP on SPECT scans is seen in most patients with AD, but could be found in other dementias as well and cannot be regarded as specific to AD. Reduced TP perfusion discriminated between demented and nondemented individuals. Further strategies for SPECT interpretation that improve diagnostic specificity should be sought.

The predictive value of videostroboscopy in the assessment of premalignant lesions and early glottis cancers

2017 ◽  
Vol 71 (4) ◽  
pp. 14-18 ◽  
Author(s):  
Anna Rzepakowska ◽  
Ewelina Sielska-Badurek ◽  
Ewa Osuch-Wójcikiewicz ◽  
Michał Sobol ◽  
Kazimierz Niemczyk
Keyword(s):  
Predictive Value ◽  
Vocal Fold ◽  
Histopathological Examination ◽  
High Sensitivity ◽  
Vocal Folds ◽  
Invasive Cancer ◽  
Premalignant Lesions ◽  
Good Tool ◽  
Malignant Lesions ◽  
Sensitivity Specificity

Objective: To assess the sensitivity and specificity of larngovideostroboscopy (LVS) in the diagnosis of precancerous and malignant lesions of the vocal folds. Material and methods: In 175 patients (128 men and 47 women), aged 19-88 years, mean age 61.5, who were admitted to the clinic with diagnosed premalignant conditions of vocal fold mucosa (leukoplakia, chronic hypertrophic inflammatory lesions) and thickening or tumor on the vocal fold, there was performed LVS before the laryngeal microsurgery. The LVS study included: localization of the leasion, movement of the vocal folds, mucosal wave, shape of glottis clousure, amplitude and symmetry of vocal fold vibration. In the evaluation, a point scale was applied for the individual functional parameters. The scale ranged from 0 to 14. Patients with impaired vocal fold motion or absent mucosal wave were positive on LVS for malignant lesions. Those with limitted mucosal wave were positive on LVS for dysplastic lesions. The results were compared with the final histopathological examination and the sensitivity, specificity, accuracy, positive (PPV) and negative (NPV) predictive value were calculated. Results: On the basis of histopathological examination, benign lesions (normal or inflammatory mucosa) accounted for 20% of diagnoses, hypertrophy and parakeratosis for 28%, low and middle grade dysplasia accounted for 10% and malignant lesions (high-grade dysplasia, pre-invasive cancer, Invasive cancer) was diagnosed in 42% of patients. The overall mean score for LVS was 4.5 and 8.0, respectively for benign and malignant lesions. Sensitivity, specificity, accuracy, PPV and NPV of LVS in detecting malignant lesions were respectively - 95.6%, 23.8%, 61.1%, 57.6% and 83.3% and in detecting both premalignant and malignant lesions were respectively – 90.7%, 31.4%, 78.9%, 84.1% and 45.8%. Conclusions: Because of the high sensitivity of LVS in detecting precancerous and malignant lesions, this method is a very good tool for screening of pathology within the larynx.

Sign in / Sign up

Export Citation Format

Share Document