Do the American Society of Clinical Oncology Value Framework and the European Society of Medical Oncology Magnitude of Clinical Benefit Scale Measure the Same Construct of Clinical Benefit?

2017 ◽  
Vol 35 (24) ◽  
pp. 2764-2771 ◽  
Author(s):  
Sierra Cheng ◽  
Erica J. McDonald ◽  
Matthew C. Cheung ◽  
Vanessa S. Arciero ◽  
Mahin Qureshi ◽  
...  
Keyword(s):  
Clinical Trials ◽  
European Society ◽  
Clinical Benefit ◽  
Randomized Clinical Trials ◽  
Medical Oncology ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Rater Reliability ◽  
Oncology Drug ◽  
Life Years

Purpose Whether the ASCO Value Framework and the European Society for Medical Oncology (ESMO) Magnitude of Clinical Benefit Scale (MCBS) measure similar constructs of clinical benefit is unclear. It is also unclear how they relate to quality-adjusted life-years (QALYs) and funding recommendations in the United Kingdom and Canada. Methods Randomized clinical trials of oncology drug approvals by the US Food and Drug Administration, European Medicines Agency, and Health Canada between 2006 and August 2015 were identified and scored using the ASCO version 1 (v1) framework, ASCO version 2 (v2) framework, and ESMO-MCBS by at least two independent reviewers. Spearman correlation coefficients were calculated to assess construct (between frameworks) and criterion validity (against QALYs from the National Institute for Health and Care Excellence [NICE] and the pan-Canadian Oncology Drug Review [pCODR]). Associations between scores and NICE/pCODR recommendations were examined. Inter-rater reliability was assessed using intraclass correlation coefficients. Results From 109 included randomized clinical trials, 108 ASCOv1, 111 ASCOv2, and 83 ESMO scores were determined. Correlation coefficients for ASCOv1 versus ESMO, ASCOv2 versus ESMO, and ASCOv1 versus ASCOv2 were 0.36 (95% CI, 0.15 to 0.54), 0.17 (95% CI, −0.06 to 0.37), and 0.50 (95% CI, 0.35 to 0.63), respectively. Compared with NICE QALYs, correlation coefficients were 0.45 (ASCOv1), 0.53 (ASCOv2), and 0.46 (ESMO); with pCODR QALYs, coefficients were 0.19 (ASCOv1), 0.20 (ASCOv2), and 0.36 (ESMO). None of the frameworks were significantly associated with NICE/pCODR recommendations. Inter-rater reliability was good for all frameworks. Conclusion The weak-to-moderate correlations of the ASCO frameworks with the ESMO-MCBS, as well as their correlations with QALYs and with NICE/pCODR funding recommendations, suggest different constructs of clinical benefit measured. Construct convergent validity with the ESMO-MCBS did not increase with the updated ASCO framework.

Do the American Society of Clinical Oncology (ASCO) Value Framework and the European Society of Medical Oncology (ESMO) Magnitude of Clinical Benefit Scale measure the same construct of clinical benefit?

2017 ◽  
Vol 35 (15_suppl) ◽  
pp. 6509-6509 ◽  
Author(s):  
Sierra Cheng ◽  
Erica McDonald ◽  
Matthew C. Cheung ◽  
Vanessa Sarah Arciero ◽  
Mahin Iqbal Qureshi ◽  
...  
Keyword(s):  
Clinical Oncology ◽  
European Society ◽  
Clinical Benefit ◽  
Randomized Clinical Trials ◽  
Medical Oncology ◽  
Correlation Coefficients ◽  
Rater Reliability ◽  
Oncology Drug ◽  
Life Years ◽  
American Society

6509 Background: Whether the American Society of Clinical Oncology (ASCO) Value Framework and the European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS) measure similar constructs of clinical benefit is unclear. It is also unclear how they relate to quality-adjusted life-years (QALYs) and funding recommendations in the UK and Canada. Methods: Randomized clinical trials (RCTs) of oncology drug approvals by the Food and Drug Administration, European Medicines Agency and Health Canada between January 2006 and August 2015 were identified and scored using the ASCO version 1 (v1) framework (August 10, 2015), ASCO version 2 (v2) framework (May 31, 2016) and ESMO-MCBS (May 30, 2015) by at least two independent reviewers. Spearman correlation coefficients were calculated to assess construct (between frameworks) and criterion validity (against incremental QALYs from the National Institute of Clinical Excellence (NICE) and the pan-Canadian Oncology Drug Review (pCODR)). Associations between scores and NICE/pCODR recommendations were examined by logistic regression models. Inter-rater reliability was assessed using intra-class correlation coefficients. Results: From 109 included RCTs, 108 ASCOv1, 111 ASCOv2 and 83 ESMO scores were determined. Correlation coefficients for ASCOv1 vs. ESMO, ASCOv2 vs. ESMO, and ASCOv1 vs. ASCOv2 were 0.36 (95% CI 0.15-0.54), 0.17 (95% CI -0.06-0.37) and 0.50 (95% CI 0.35-0.63), respectively. Compared with NICE QALYs, correlation coefficients were 0.45 (ASCOv1), 0.53 (ASCOv2) and 0.46 (ESMO); with pCODR QALYs, coefficients were 0.19 (ASCOv1), 0.20 (ASCOv2) and 0.36 (ESMO). None of the frameworks were significantly associated with NICE/pCODR recommendations. Inter-rater reliability was good for all frameworks. Conclusions: The weak-to-moderate correlations between the ASCO frameworks and ESMO-MCBS, with QALYs, and with NICE/pCODR funding recommendations suggest different constructs of clinical benefit measured. Construct convergent validity with the ESMO-MCBS in fact did not increase with the updated ASCO framework.

scholarly journals First-Line Maintenance Treatment in Metastatic Colorectal Cancer (mCRC): Quality and Clinical Benefit Overview

2021 ◽  
Vol 10 (3) ◽  
pp. 470
Author(s):  
Marta Martín-Richard ◽  
Maria Tobeña
Keyword(s):  
Colorectal Cancer ◽  
Clinical Trials ◽  
Clinical Benefit ◽  
Maintenance Treatment ◽  
Randomized Clinical Trials ◽  
Line Treatment ◽  
First Line ◽  
Design Quality ◽  
Primary Endpoints ◽  
First Line Treatment

Different strategies of maintenance therapy (sequential CT, intermittent CT, intermittent CT and MAbs, or de-escalation MAbs monotherapy) after first-line treatment are undertaken. Many randomized clinical trials (RCT), which evaluated these approaches, suffer from incorrect design, heterogenous primary endpoints, inadequate size, and other methodology flaws. Drawing any conclusions becomes challenging and recommendations are mainly vague. We evaluated those studies from another perspective, focusing on the design quality and the clinical benefit measure with a more objective and accurate methodology. These data allowed a clearer and more exact overview of the statement in maintenance treatment.

scholarly journals Adaptation and Psychometric Evaluation of the Chinese Counseling Competencies Scale-Revised

2021 ◽  
Vol 12 ◽  
Author(s):  
Wei Xia ◽  
William Ho Cheung Li ◽  
Tingna Liang ◽  
Yuanhui Luo ◽  
Laurie Long Kwan Ho ◽  
...  
Keyword(s):  
Concurrent Validity ◽  
Convergent Validity ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Psychometric Evaluation ◽  
Counseling Competencies ◽  
Counselors In Training ◽  
Rater Reliability ◽  
Retest Reliability ◽  
Test Retest Reliability

Objectives: This study conducted a linguistic and psychometric evaluation of the Chinese Counseling Competencies Scale-Revised (CCS-R).Methods: The Chinese CCS-R was created from the original English version using a standard forward-backward translation process. The psychometric properties of the Chinese CCS-R were examined in a cohort of 208 counselors-in-training by two independent raters. Fifty-three counselors-in-training were asked to undergo another counseling performance evaluation for the test-retest. The confirmatory factor analysis (CFA) was conducted for the Chinese CCS-R, followed by internal consistency, test-retest reliability, inter-rater reliability, convergent validity, and concurrent validity.Results: The results of the CFA supported the factorial validity of the Chinese CCS-R, with adequate construct replicability. The scale had a McDonald's omega of 0.876, and intraclass correlation coefficients of 0.63 and 0.90 for test-retest reliability and inter-rater reliability, respectively. Significantly positive correlations were observed between the Chinese CCS-R score and scores of performance checklist (Pearson's γ = 0.781), indicating a large convergent validity, and knowledge on drug abuse (Pearson's γ = 0.833), indicating a moderate concurrent validity.Conclusion: The results support that the Chinese CCS-R is a valid and reliable measure of the counseling competencies.Practice implication: The CCS-R provides trainers with a reliable tool to evaluate counseling students' competencies and to facilitate discussions with trainees about their areas for growth.

scholarly journals The QardioArm App in the Assessment of Blood Pressure and Heart Rate: Reliability and Validity Study (Preprint)

2017 ◽  
Author(s):  
Victoria Mazoteras Pardo ◽  
Marta E Losa Iglesias ◽  
José López Chicharro ◽  
Ricardo Becerro de Bengoa Vallejo
Keyword(s):  
Blood Pressure ◽  
Heart Rate ◽  
European Society ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Correlation Coefficients ◽  
Mobile App ◽  
Blood Pressure Monitor ◽  
Pressure Monitor ◽  
Mean Values

BACKGROUND Self-measurement of blood pressure is a priority strategy for managing blood pressure. OBJECTIVE The aim of this study was to evaluate the reliability and validity of blood pressure and heart rate following the European Society of Hypertension’s international validation protocol, as measured with the QardioArm, a fully automatic, noninvasive wireless blood pressure monitor and mobile app. METHODS A total of 100 healthy volunteers older than 25 years from the general population of Ciudad Real, Spain, participated in a test-retest validation study with two measurement sessions separated by 5 to 7 days. In each measurement session, seven systolic blood pressure, diastolic blood pressure, and heart rate assessments were taken, alternating between the two devices. The test device was the QardioArm and the previously validated criterion device was the Omron M3. Sessions took place at a single study site with an evaluation room that was maintained at an appropriate temperature and kept free from noises and distractions. RESULTS The QardioArm displayed very consistent readings both within and across sessions (intraclass correlation coefficients=0.80-0.95, standard errors of measurement=2.5-5.4). The QardioArm measurements corresponded closely to those from the criterion device (r>.96) and mean values for the two devices were nearly identical. The QardioArm easily passed all validation standards set by the European Society of Hypertension International Protocol. CONCLUSIONS The QardioArm mobile app has validity and it can be used free of major measurement error.

Is the location of the signal intensity weighted centroid a reliable measurement of fluid displacement within the disc?

2018 ◽  
Vol 63 (4) ◽  
pp. 453-460 ◽  
Author(s):  
Vahid Abdollah ◽  
Eric C. Parent ◽  
Michele C. Battié
Keyword(s):  
Signal Intensity ◽  
Water Distribution ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Region Of Interest ◽  
Rater Reliability ◽  
Fluid Displacement ◽  
Intraclass Correlation Coefficients ◽  
The Mean ◽  
Standard Error Of Measurement

Abstract Degenerated discs have shorter T2-relaxation time and lower MR signal. The location of the signal-intensity-weighted-centroid reflects the water distribution within a region-of-interest (ROI). This study compared the reliability of the location of the signal-intensity-weighted-centroid to mean signal intensity and area measurements. L4-L5 and L5-S1 discs were measured on 43 mid-sagittal T2-weighted 3T MRI images in adults with back pain. One rater analysed images twice and another once, blinded to measurements. Discs were semi-automatically segmented into a whole disc, nucleus, anterior and posterior annulus. The coordinates of the signal-intensity-weighted-centroid for all regions demonstrated excellent intraclass-correlation-coefficients for intra- (0.99–1.00) and inter-rater reliability (0.97–1.00). The standard error of measurement for the Y-coordinates of the signal-intensity-weighted-centroid for all ROIs were 0 at both levels and 0 to 2.7 mm for X-coordinates. The mean signal intensity and area for the whole disc and nucleus presented excellent intra-rater reliability with intraclass-correlation-coefficients from 0.93 to 1.00, and 0.92 to 1.00 for inter-rater reliability. The mean signal intensity and area had lower reliability for annulus ROIs, with intra-rater intraclass-correlation-coefficient from 0.5 to 0.76 and inter-rater from 0.33 to 0.58. The location of the signal-intensity-weighted-centroid is a reliable biomarker for investigating the effects of disc interventions.

Development and assessment of the inter-rater and intra-rater reproducibility of a self-administration version of the ALSFRS-R

2019 ◽  
Vol 91 (1) ◽  
pp. 75-81 ◽  
Author(s):  
Leonhard A Bakker ◽  
Carin D Schröder ◽  
Harold H G Tan ◽  
Simone M A G Vugts ◽  
Ruben P A van Eijk ◽  
...  
Keyword(s):  
Rating Scale ◽  
Clinical Care ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
The Self ◽  
Coefficient Alpha ◽  
Rater Agreement ◽  
Self Administration ◽  
Limits Of Agreement ◽  
Rater Reliability

ObjectiveThe Amyotrophic Lateral Sclerosis Functional Rating Scale-Revised (ALSFRS-R) is widely applied to assess disease severity and progression in patients with motor neuron disease (MND). The objective of the study is to assess the inter-rater and intra-rater reproducibility, i.e., the inter-rater and intra-rater reliability and agreement, of a self-administration version of the ALSFRS-R for use in apps, online platforms, clinical care and trials.MethodsThe self-administration version of the ALSFRS-R was developed based on both patient and expert feedback. To assess the inter-rater reproducibility, 59 patients with MND filled out the ALSFRS-R online and were subsequently assessed on the ALSFRS-R by three raters. To assess the intra-rater reproducibility, patients were invited on two occasions to complete the ALSFRS-R online. Reliability was assessed with intraclass correlation coefficients, agreement was assessed with Bland-Altman plots and paired samples t-tests, and internal consistency was examined with Cronbach’s coefficient alpha.ResultsThe self-administration version of the ALSFRS-R demonstrated excellent inter-rater and intra-rater reliability. The assessment of inter-rater agreement demonstrated small systematic differences between patients and raters and acceptable limits of agreement. The assessment of intra-rater agreement demonstrated no systematic changes between time points; limits of agreement were 4.3 points for the total score and ranged from 1.6 to 2.4 points for the domain scores. Coefficient alpha values were acceptable.DiscussionThe self-administration version of the ALSFRS-R demonstrates high reproducibility and can be used in apps and online portals for both individual comparisons, facilitating the management of clinical care and group comparisons in clinical trials.

scholarly journals A Comparison of Reliability Coefficients for Ordinal Rating Scales

2021 ◽  
Author(s):  
Alexandra de Raadt ◽  
Matthijs J. Warrens ◽  
Roel J. Bosker ◽  
Henk A. L. Kiers
Keyword(s):  
Empirical Data ◽  
Rating Scales ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
Weighted Kappa ◽  
Rater Reliability ◽  
Intraclass Correlations ◽  
Applied Researcher ◽  
Highly Correlated ◽  
Reliability Coefficients

AbstractKappa coefficients are commonly used for quantifying reliability on a categorical scale, whereas correlation coefficients are commonly applied to assess reliability on an interval scale. Both types of coefficients can be used to assess the reliability of ordinal rating scales. In this study, we compare seven reliability coefficients for ordinal rating scales: the kappa coefficients included are Cohen’s kappa, linearly weighted kappa, and quadratically weighted kappa; the correlation coefficients included are intraclass correlation ICC(3,1), Pearson’s correlation, Spearman’s rho, and Kendall’s tau-b. The primary goal is to provide a thorough understanding of these coefficients such that the applied researcher can make a sensible choice for ordinal rating scales. A second aim is to find out whether the choice of the coefficient matters. We studied to what extent we reach the same conclusions about inter-rater reliability with different coefficients, and to what extent the coefficients measure agreement in a similar way, using analytic methods, and simulated and empirical data. Using analytical methods, it is shown that differences between quadratic kappa and the Pearson and intraclass correlations increase if agreement becomes larger. Differences between the three coefficients are generally small if differences between rater means and variances are small. Furthermore, using simulated and empirical data, it is shown that differences between all reliability coefficients tend to increase if agreement between the raters increases. Moreover, for the data in this study, the same conclusion about inter-rater reliability was reached in virtually all cases with the four correlation coefficients. In addition, using quadratically weighted kappa, we reached a similar conclusion as with any correlation coefficient a great number of times. Hence, for the data in this study, it does not really matter which of these five coefficients is used. Moreover, the four correlation coefficients and quadratically weighted kappa tend to measure agreement in a similar way: their values are very highly correlated for the data in this study.

Evaluation of Anti-cancer Therapies with Reimbursement Limited to Comprehensive Cancer Centres Using the European Society for Medical Oncology Magnitude of Clinical Benefit Scale

2017 ◽  
Vol 30 (5) ◽  
pp. 349-360
Author(s):  
Tomáš Büchler ◽  
Bohuslav Melichar ◽  
David Vrána ◽  
Radmila Lemstrová ◽  
Jindřich Fínek ◽  
...  
Keyword(s):  
European Society ◽  
Clinical Benefit ◽  
Medical Oncology ◽  
Cancer Therapies ◽  
Anti Cancer

scholarly journals The Brazilian version of the Neuropsychiatric Inventory-Clinician rating scale (NPI-C): reliability and validity in dementia

2013 ◽  
Vol 25 (9) ◽  
pp. 1503-1511 ◽  
Author(s):  
Florindo Stella ◽  
Orestes Vicente Forlenza ◽  
Jerson Laks ◽  
Larissa Pires de Andrade ◽  
Michelle A. Ljubetic Avendaño ◽  
...  
Keyword(s):  
Concurrent Validity ◽  
Convergent Validity ◽  
Rating Scale ◽  
Neuropsychiatric Symptoms ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Correlation Coefficients ◽  
Neuropsychiatric Inventory ◽  
Rater Reliability ◽  
Clinician Rating

ABSTRACTBackground:Patients with dementia may be unable to describe their symptoms, and caregivers frequently suffer emotional burden that can interfere with judgment of the patient's behavior. The Neuropsychiatric Inventory-Clinician rating scale (NPI-C) was therefore developed as a comprehensive and versatile instrument to assess and accurately measure neuropsychiatric symptoms (NPS) in dementia, thereby using information from caregiver and patient interviews, and any other relevant available data. The present study is a follow-up to the original, cross-national NPI-C validation, evaluating the reliability and concurrent validity of the NPI-C in quantifying psychopathological symptoms in dementia in a large Brazilian cohort.Methods:Two blinded raters evaluated 312 participants (156 patient-knowledgeable informant dyads) using the NPI-C for a total of 624 observations in five Brazilian centers. Inter-rater reliability was determined through intraclass correlation coefficients for the NPI-C domains and the traditional NPI. Convergent validity included correlations of specific domains of the NPI-C with the Brief Psychiatric Rating Scale (BPRS), the Cohen-Mansfield Agitation Index (CMAI), the Cornell Scale for Depression in Dementia (CSDD), and the Apathy Inventory (AI).Results:Inter-rater reliability was strong for all NPI-C domains. There were high correlations between NPI-C/delusions and BPRS, NPI-C/apathy-indifference with the AI, NPI-C/depression-dysphoria with the CSDD, NPI-C/agitation with the CMAI, and NPI-C/aggression with the CMAI. There was moderate correlation between the NPI-C/aberrant vocalizations and CMAI and the NPI-C/hallucinations with the BPRS.Conclusion:The NPI-C is a comprehensive tool that provides accurate measurement of NPS in dementia with high concurrent validity and inter-rater reliability in the Brazilian setting. In addition to universal assessment, the NPI-C can be completed by individual domains.

Sign in / Sign up

Export Citation Format

Share Document