Do the American Society of Clinical Oncology (ASCO) Value Framework and the European Society of Medical Oncology (ESMO) Magnitude of Clinical Benefit Scale measure the same construct of clinical benefit?
6509 Background: Whether the American Society of Clinical Oncology (ASCO) Value Framework and the European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS) measure similar constructs of clinical benefit is unclear. It is also unclear how they relate to quality-adjusted life-years (QALYs) and funding recommendations in the UK and Canada. Methods: Randomized clinical trials (RCTs) of oncology drug approvals by the Food and Drug Administration, European Medicines Agency and Health Canada between January 2006 and August 2015 were identified and scored using the ASCO version 1 (v1) framework (August 10, 2015), ASCO version 2 (v2) framework (May 31, 2016) and ESMO-MCBS (May 30, 2015) by at least two independent reviewers. Spearman correlation coefficients were calculated to assess construct (between frameworks) and criterion validity (against incremental QALYs from the National Institute of Clinical Excellence (NICE) and the pan-Canadian Oncology Drug Review (pCODR)). Associations between scores and NICE/pCODR recommendations were examined by logistic regression models. Inter-rater reliability was assessed using intra-class correlation coefficients. Results: From 109 included RCTs, 108 ASCOv1, 111 ASCOv2 and 83 ESMO scores were determined. Correlation coefficients for ASCOv1 vs. ESMO, ASCOv2 vs. ESMO, and ASCOv1 vs. ASCOv2 were 0.36 (95% CI 0.15-0.54), 0.17 (95% CI -0.06-0.37) and 0.50 (95% CI 0.35-0.63), respectively. Compared with NICE QALYs, correlation coefficients were 0.45 (ASCOv1), 0.53 (ASCOv2) and 0.46 (ESMO); with pCODR QALYs, coefficients were 0.19 (ASCOv1), 0.20 (ASCOv2) and 0.36 (ESMO). None of the frameworks were significantly associated with NICE/pCODR recommendations. Inter-rater reliability was good for all frameworks. Conclusions: The weak-to-moderate correlations between the ASCO frameworks and ESMO-MCBS, with QALYs, and with NICE/pCODR funding recommendations suggest different constructs of clinical benefit measured. Construct convergent validity with the ESMO-MCBS in fact did not increase with the updated ASCO framework.