scholarly journals ROC Curves in Clinical Chemistry: Uses, Misuses, and Possible Solutions

2004 ◽  
Vol 50 (7) ◽  
pp. 1118-1125 ◽  
Author(s):  
Nancy A Obuchowski ◽  
Michael L Lieber ◽  
Frank H Wians
Keyword(s):  
Phase I ◽  
Sample Size ◽  
Gold Standard ◽  
Diagnostic Tests ◽  
Clinical Laboratory ◽  
Clinical Chemistry ◽  
Roc Curves ◽  
Sample Size Determination ◽  
Phase Iii ◽  
Test Accuracy

Abstract Background: ROC curves have become the standard for describing and comparing the accuracy of diagnostic tests. Not surprisingly, ROC curves are used often by clinical chemists. Our aims were to observe how the accuracy of clinical laboratory diagnostic tests is assessed, compared, and reported in the literature; to identify common problems with the use of ROC curves; and to offer some possible solutions. Methods: We reviewed every original work using ROC curves and published in Clinical Chemistry in 2001 or 2002. For each article we recorded phase of the research, prospective or retrospective design, sample size, presence/absence of confidence intervals (CIs), nature of the statistical analysis, and major analysis problems. Results: Of 58 articles, 31% were phase I (exploratory), 50% were phase II (challenge), and 19% were phase III (advanced) studies. The studies increased in sample size from phase I to III and showed a progression in the use of prospective designs. Most phase I studies were powered to assess diagnostic tests with ROC areas ≥0.70. Thirty-eight percent of studies failed to include CIs for diagnostic test accuracy or the CIs were constructed inappropriately. Thirty-three percent of studies provided insufficient analysis for comparing diagnostic tests. Other problems included dichotomization of the gold standard scale and inappropriate analysis of the equivalence of two diagnostic tests. Conclusion: We identify available software and make some suggestions for sample size determination, testing for equivalence in diagnostic accuracy, and alternatives to a dichotomous classification of a continuous-scale gold standard. More methodologic research is needed in areas specific to clinical chemistry.

Sample size and rejection limits for detecting reagent lot variability: analysis of the applicability of the Clinical and Laboratory Standards Institute (CLSI) EP26-A protocol to real-world clinical chemistry data

2021 ◽  
Vol 59 (1) ◽  
pp. 127-138
Author(s):  
Sollip Kim ◽  
Jeonghyun Chang ◽  
Soo-Kyung Kim ◽  
Sholhui Park ◽  
Jungwon Huh ◽  
...  
Keyword(s):  
Sample Size ◽  
Real World ◽  
Statistical Power ◽  
Clinical Laboratory ◽  
Clinical Chemistry ◽  
Test Results ◽  
Real World Data ◽  
Reference Change Value ◽  
Laboratory Test Results ◽  
Allowable Error

AbstractObjectivesTo maintain the consistency of laboratory test results, between-reagent lot variation should be verified before using new reagent lots in clinical laboratory. Although the Clinical and Laboratory Standards Institute (CLSI) document EP26-A deals with this issue, evaluation of reagent lot-to-lot difference is challenging in reality. We aim to investigate a practical way for determining between-reagent lot variation using real-world data in clinical chemistry.MethodsThe CLSI EP26-A protocol was applied to 83 chemistry tests in three clinical labs. Three criteria were used to define the critical difference (CD) of each test as follows: reference change value and total allowable error, which are based on biological variation, and acceptable limits by external quality assurance agencies. The sample size and rejection limits that could detect CD between-reagent lots were determined.ResultsFor more than half of chemistry tests, reagent lot-to-lot differences could be evaluated using only one patient sample per decision level. In many cases, the rejection limit that could detect reagent lot-to-lot difference with ≥90% probability was 0.6 times CD. However, the sample size and rejection limits vary depending on how the CD is defined. In some cases, impractical sample size or rejection limits were obtained. In some cases, information on sample size and rejection limit that met intended statistical power was not found in EP26-A.ConclusionsThe CLSI EP26-A did not provide all necessary answers. Alternative practical approaches are suggested when CLSI EP26-A does not provide guidance.

scholarly journals Un genio maltratado. El metanálisis de pruebas diagnósticas.

2019 ◽  
Vol 11 (9) ◽  
pp. 2
Author(s):  
Manuel Molina
Keyword(s):  
Diagnostic Test ◽  
Diagnostic Tests ◽  
Critical Appraisal ◽  
Meta Analysis ◽  
Roc Curves ◽  
Threshold Effect ◽  
Diagnostic Test Accuracy ◽  
Test Accuracy ◽  
The Way

Aunque las recomendaciones generales para la lectura crítica de un metanálisis de pruebas diagnósticas son similares a las del metanálisis de estudios de tratamiento, existen aspectos específicos que deben conocerse para su correcta valoración. Destacamos el estudio del efecto umbral, la elección de la medida de síntesis y la forma de representar el resultado global con las curvas ROC específicas. ABSTRACT An unfairly treated genius. Meta-analysis of diagnostic test accuracy. Although the general recommendations for the critical appraisal of a meta-analysis of diagnostic tests are similar to those of the treatment meta-analysis, there are specific aspects that should be known for their correct assessment. We highlight the study of the threshold effect, the choice of the synthesis measure and the way to represent the overall result with the specific ROC curves.

scholarly journals Determining Plasma Protein Variation Parameters as a Prerequisite for Biomarker Studies—A TMT-Based LC-MSMS Proteome Investigation

Proteomes ◽  
2021 ◽  
Vol 9 (4) ◽  
pp. 47
Author(s):  
Lou-Ann C. Andersen ◽  
Nicolai Bjødstrup Palstrøm ◽  
Axel Diederichsen ◽  
Jes Sanddal Lindholt ◽  
Lars Melholt Rasmussen ◽  
...  
Keyword(s):  
Sample Size ◽  
Plasma Protein ◽  
Plasma Proteins ◽  
Clinical Chemistry ◽  
Biological Variation ◽  
Sample Size Determination ◽  
Clinical Proteomics ◽  
European Federation ◽  
Plasma Samples ◽  
Proteomic Approach

Specific plasma proteins serve as valuable markers for various diseases and are in many cases routinely measured in clinical laboratories by fully automated systems. For safe diagnostics and monitoring using these markers, it is important to ensure an analytical quality in line with clinical needs. For this purpose, information on the analytical and the biological variation of the measured plasma protein, also in the context of the discovery and validation of novel, disease protein biomarkers, is important, particularly in relation to for sample size calculations in clinical studies. Nevertheless, information on the biological variation of the majority of medium-to-high abundant plasma proteins is largely absent. In this study, we hypothesized that it is possible to generate data on inter-individual biological variation in combination with analytical variation of several hundred abundant plasma proteins, by applying LC-MS/MS in combination with relative quantification using isobaric tagging (10-plex TMT-labeling) to plasma samples. Using this analytical proteomic approach, we analyzed 42 plasma samples prepared in doublets, and estimated the technical, inter-individual biological, and total variation of 265 of the most abundant proteins present in human plasma thereby creating the prerequisites for power analysis and sample size determination in future clinical proteomics studies. Our results demonstrated that only five samples per group may provide sufficient statistical power for most of the analyzed proteins if relative changes in abundances >1.5-fold are expected. Seventeen of the measured proteins are present in the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Biological Variation Database, and demonstrated remarkably similar biological CV’s to the corresponding CV’s listed in the EFLM database suggesting that the generated proteomic determined variation knowledge is useful for large-scale determination of plasma protein variations.

A phase I QTc study of tivozanib in patients with advanced solid tumors.

2012 ◽  
Vol 30 (5_suppl) ◽  
pp. 445-445
Author(s):  
Manpreet Kaur Chadha ◽  
Jeffrey R. Infante ◽  
Monette M. Cotreau ◽  
Lindsey Jacobson ◽  
Andrew Louis Strahs ◽  
...  
Keyword(s):  
Phase I ◽  
Solid Tumors ◽  
Preliminary Data ◽  
Kinase Inhibitor ◽  
Clinical Laboratory ◽  
New Drugs ◽  
Phase Iii ◽  
Advanced Solid Tumors ◽  
Post Dose ◽  
Clinically Significant

445 Background: Tivozanib is a potent and selective tyrosine kinase inhibitor of vascular endothelial growth factor receptors-1, -2, and -3 that is currently being tested in a Phase III study in patients with renal cell carcinoma and Phase I/II studies of other solid tumors. Preclinical and retrospective electrocardiogram (ECG) analyses suggest no effect of tivozanib on QTc, although this has not been prospectively assessed according to ICH E14 Cardiac Assessment of New Drugs Guidelines. This open-label, non-randomized, single-arm study prospectively investigated the effect of tivozanib on the QTcF interval and its morphology on the ECG and ECG-pharmacokinetic (PK) relationship in patients with advanced solid tumors. Methods: Patients with advanced solid tumors, an ECOG score ≤1 and life expectancy ≥3 months were eligible. Patients received 1.5 mg of tivozanib orally, once daily for 21 days. Serial blood samples and time-matched, triplicate, 12-lead ECGs were collected on: Day 1 20-30 minutes pre-dose (no blood sample collected), immediately pre-dose, and at 2.5, 4, 5, 6, 8, and 10 hours post dose; Day 2 pre-dose evaluation was taken approximately 24 hours post Day 1 dose; Day 8 (±1 day) pre-dose, and at 2.5, 5, and 8 hours post dose; Day 21 pre-dose and at 2.5, 4, 5, 6, 8, and 10 hours post dose; and Day 22 at approximately 24 hours post Day 21 dose. Additional safety parameters were evaluated by assessing clinical laboratory tests, physical examinations, vital signs, and recording of adverse events. Results: Fifty patients with advanced solid tumors (males, 17; median age, 63 years; 94% white) who received ≥ 1 dose of tivozanib were evaluable. Preliminary data showed that there were no clinically significant changes in QTcF from baseline. Further analysis will be completed, and final safety and ECG-PK modeling will be presented. Conclusions: Preliminary data suggest that tivozanib 1.5 mg/d over a 21-day period does not cause clinically significant QT/QTc prolongation over baseline, suggesting that its safety and PK profile is similar to that observed in previous studies, including ECG evaluation in a monkey telemetry study.

Meta-analysis for the comparison of two diagnostic tests to a common gold standard: A generalized linear mixed model approach

2016 ◽  
Vol 27 (5) ◽  
pp. 1410-1421 ◽  
Author(s):  
Annika Hoyer ◽  
Oliver Kuss
Keyword(s):  
Gold Standard ◽  
Diagnostic Tests ◽  
Mixed Model ◽  
Linear Mixed Model ◽  
Meta Analysis ◽  
Roc Curves ◽  
Generalized Linear Mixed Model ◽  
Screening Methods ◽  
Bivariate Model ◽  
Mixed Model Approach

Meta-analysis of diagnostic studies is still a rapidly developing area of biostatistical research. Especially, there is an increasing interest in methods to compare different diagnostic tests to a common gold standard. Restricting to the case of two diagnostic tests, in these meta-analyses the parameters of interest are the differences of sensitivities and specificities (with their corresponding confidence intervals) between the two diagnostic tests while accounting for the various associations across single studies and between the two tests. We propose statistical models with a quadrivariate response (where sensitivity of test 1, specificity of test 1, sensitivity of test 2, and specificity of test 2 are the four responses) as a sensible approach to this task. Using a quadrivariate generalized linear mixed model naturally generalizes the common standard bivariate model of meta-analysis for a single diagnostic test. If information on several thresholds of the tests is available, the quadrivariate model can be further generalized to yield a comparison of full receiver operating characteristic (ROC) curves. We illustrate our model by an example where two screening methods for the diagnosis of type 2 diabetes are compared.

Sign in / Sign up

Export Citation Format

Share Document