ROC curves, test accuracy, and the description of diagnostic tests

1991 ◽  
Vol 3 (3) ◽  
pp. 330-333 ◽  
2004 ◽  
Vol 50 (7) ◽  
pp. 1118-1125 ◽  
Author(s):  
Nancy A Obuchowski ◽  
Michael L Lieber ◽  
Frank H Wians

Abstract Background: ROC curves have become the standard for describing and comparing the accuracy of diagnostic tests. Not surprisingly, ROC curves are used often by clinical chemists. Our aims were to observe how the accuracy of clinical laboratory diagnostic tests is assessed, compared, and reported in the literature; to identify common problems with the use of ROC curves; and to offer some possible solutions. Methods: We reviewed every original work using ROC curves and published in Clinical Chemistry in 2001 or 2002. For each article we recorded phase of the research, prospective or retrospective design, sample size, presence/absence of confidence intervals (CIs), nature of the statistical analysis, and major analysis problems. Results: Of 58 articles, 31% were phase I (exploratory), 50% were phase II (challenge), and 19% were phase III (advanced) studies. The studies increased in sample size from phase I to III and showed a progression in the use of prospective designs. Most phase I studies were powered to assess diagnostic tests with ROC areas ≥0.70. Thirty-eight percent of studies failed to include CIs for diagnostic test accuracy or the CIs were constructed inappropriately. Thirty-three percent of studies provided insufficient analysis for comparing diagnostic tests. Other problems included dichotomization of the gold standard scale and inappropriate analysis of the equivalence of two diagnostic tests. Conclusion: We identify available software and make some suggestions for sample size determination, testing for equivalence in diagnostic accuracy, and alternatives to a dichotomous classification of a continuous-scale gold standard. More methodologic research is needed in areas specific to clinical chemistry.


2019 ◽  
Vol 11 (9) ◽  
pp. 2
Author(s):  
Manuel Molina

Aunque las recomendaciones generales para la lectura crítica de un metanálisis de pruebas diagnósticas son similares a las del metanálisis de estudios de tratamiento, existen aspectos específicos que deben conocerse para su correcta valoración. Destacamos el estudio del efecto umbral, la elección de la medida de síntesis y la forma de representar el resultado global con las curvas ROC específicas. ABSTRACT An unfairly treated genius. Meta-analysis of diagnostic test accuracy. Although the general recommendations for the critical appraisal of a meta-analysis of diagnostic tests are similar to those of the treatment meta-analysis, there are specific aspects that should be known for their correct assessment. We highlight the study of the threshold effect, the choice of the synthesis measure and the way to represent the overall result with the specific ROC curves.


1999 ◽  
Vol 45 (7) ◽  
pp. 995-1001 ◽  
Author(s):  
José A Arranz ◽  
Encarnació Riudor ◽  
Margarita Rodés ◽  
Manuel Roig ◽  
Consuelo Climent ◽  
...  

Abstract Background: The diagnosis of heterozygosity for X-linked ornithine carbamoyltransferase (OCT) deficiency has usually been based on measurement of the increase of orotate and orotidine excretion after an allopurinol load. We examined the choices of analyte, cutoff, and test conditions to obtain maximal test accuracy. Methods: Urine orotate/orotidine responses to allopurinol load in 37 children (13 OCT-deficient and 24 non-OCT-deficient) and 24 women (7 at risk for carrier status and 17 not related to OCT-deficient children) were analyzed by liquid chromatography after sample purification by anion-exchange chromatography. Diagnostic accuracy was evaluated by nonparametric ROC curves. Results: Sample purification was necessary to prevent interferences. Orotate and orotidine excretion increased with increased protein intake during the test. At a cutoff of 8 mmol orotidine/mol creatinine, sensitivity was 1.0 and specificity was 0.92 in mild forms of OCT deficiency. Results in monoplex carrier women may differ greatly from those expected because of the genetics of this deficiency. Conclusions: Standardization of protein intake is required in the allopurinol loading test. A negative response in the face of clinical suspicion should be followed with a repeat test during a protein intake not <2.5 g · kg−1 · day−1. Measurements of orotidine provide better clinical sensitivity than measurements of orotate.


1992 ◽  
Vol 9 (4) ◽  
pp. 506-511 ◽  
Author(s):  
YT VAN DER SCHOUW ◽  
ALM VERBEEK ◽  
JHJ RUIJS

2005 ◽  
Vol 24 (1) ◽  
pp. 1-13 ◽  
Author(s):  
Nada Majkic-Singh

Cardiac markers have undergone an amazing transformation from asparatate aminotransferase (AST) and lactate dehydrogenase (LDH) to the three cardiac markers families available at present for routine use in Emergency Department for the evaluation of the chest discomfort: myoglobin, creatine kinase (CK) and the MB isoenzyme of CK (CK-MB), and the troponins I and T (cTnI and cTnT). Each of these has well known kinetics of release from dying myocardial cells and should be carefully applied to each patient as directed by timing of symptoms and presentation. Myoglobin has been touted as an early marker with a high negative predictive value but low specificity. CK and CK-MB represent the "gold standard" for the diagnosis of MI as defined by the WHO criteria. The toponins are cardiac-specific proteins with high degrees of both sensitivity and specificity for myocardial necrosis. These serum markers of necrosis have been well studied in high-risk groups with a high prevalence of AMI. Promising research has also proven benefit in lower-risk patients in the chest pain units. Inflammatory markers such as C-reactive protein (CRP) and markers of platelet such as P-selectin are currently being studied but have not yet been accepted for widespread use. Cardiac markers have proved extremely valuable for diagnosis, risk stratification and treatment of patients in the emergency setting. However, the ideal cardiac marker evaluation protocol varies between institutions, laboratories, patient's populations, and resource availability. Specific marker regimens should be tailored to meet the objectives of diagnosis myocardial infarction and providing risk stratification. New tests are developed at a fast rate and the technology of existing test is continuously being improved. A rigorous evaluation process of diagnostic tests before introduction into clinical practice could not only reduce the number of unwanted clinical consequences related to misleading estimates of test accuracy, but also limit health care costs by preventing unnecessary testing. The evaluation of diagnostic tests is complex but analytical accuracy and diagnostic accuracy is recognized as two of the pillars. Earlier recognition of problems with the quality of reporting of randomized, controlled clinical trials resulted in the Consolited Standards of Reporting Trials (CONSORT) Statement, on the basis of which a checklist of items that should be easily identified in the report of any study on diagnostic accuracy has been developed. The Standards for Reporting of Diagnostic Accuracy (STARD) group has tried to provide the evidence supporting the various components of the Statement. On the basis of these approach, the concept of Evidence- Based Laboratory Medicine (EBLM) should be taken seriously, therefore, for several reasons. First, we should all take pride in producing the best results possible to aid physicians in making diagnostic, prognostic, and treatment decisions. Second, the enormous increase in diagnostic testing is under scrutiny. Third, modern health services question whether laboratory tests offer good value for the money. Biochemical markers of myocardial injury are universally accepted as important for the diagnosis of patients with acute coronary syndromes. In addition to very well established biomarkers, many potential biomarkers are introduced as natriuretic peptides, cardiotonic steroids, cytokines, ischemia-modified albumin, free fatty acids, etc. and their significance and usefulness for acute coronary syndromes will be discussed, as well.


2000 ◽  
Vol 21 (4) ◽  
pp. 278-284 ◽  
Author(s):  
David Birnbaum ◽  
Barry M. Farr ◽  
David E. Shapiro

This article focuses on the selection and interpretation of diagnostic tests, emphasizing the importance of understanding how their mathematical parameters affect the information they provide in various settings. The utility and limitations of sensitivity, specificity, predictive value, and receiver operating characteristic (ROC) curves are discussed using catheter-related bloodstream infections as an example. ROC curves have been used for selecting optimal cutoff values for a positive result and for selecting among several alternative diagnostic tests. For example, 16 different tests have been proposed for diagnosis of catheter-related bloodstream infection; ROC analysis provides an effective way to determine which test offers the best overall performance.


Sign in / Sign up

Export Citation Format

Share Document