Extending the I-squared statistic to describe treatment effect heterogeneity in cluster, multi-centre randomized trials and individual patient data meta-analysis

Treatment effect heterogeneity is commonly investigated in meta-analyses to identify if treatment effects vary across studies. When conducting an aggregate level data meta-analysis it is common to describe the magnitude of any treatment effect heterogeneity using the I-squared statistic, which is an intuitive and easily understood concept. The effect of a treatment might also vary across clusters in a cluster randomized trial, or across centres in multi-centre randomized trial, and it can be of interest to explore this at the analysis stage. In cross-over trials and other randomized designs, in which clusters or centres are exposed to both treatment and control conditions, this treatment effect heterogeneity can be identified. Here we derive and evaluate a comparable I-squared measure to describe the magnitude of heterogeneity in treatment effects across clusters or centres in randomized trials. We further show how this methodology can be used to estimate treatment effect heterogeneity in an individual patient data meta-analysis.

Download Full-text

Reappraising Treatment Effect Heterogeneity in Schizophrenia: A Meta-analysis

10.1101/2021.11.03.21265858 ◽

2021 ◽

Author(s):

Robert Ali McCutcheon ◽

Toby Pillinger ◽

Orestis Efthimiou ◽

Marta Maslej ◽

Benoit Mulsant ◽

...

Keyword(s):

Treatment Effect ◽

Treatment Effects ◽

Individual Patient Data ◽

Placebo Response ◽

Patient Data ◽

Antipsychotic Treatment ◽

Level Data ◽

Treatment Effect Heterogeneity ◽

Meta Analyses ◽

Effect Heterogeneity

Objective Determining whether individual patients differ in response to treatment ('treatment effect heterogeneity') is important as it is a prerequisite to developing personalised treatment approaches. Previous variability meta-analyses of response to antipsychotics in schizophrenia found no evidence for treatment effect heterogeneity. Conversely, individual patient data meta-analyses suggest treatment effect heterogeneity does exist. In the current paper we combine individual patient data with study level data to resolve this apparent contradiction and quantitively characterise antipsychotic treatment effect heterogeneity in schizophrenia. Method Individual patient data (IPD) was obtained from the Yale University Open Data Access (YODA) project. Clinical trials were identified in EMBASE, PsycInfo, and PubMed. Treatment effect heterogeneity was estimated from variability ratios derived from study-level data from 66 RCTs of antipsychotics in schizophrenia (N=17,202). This estimation required a correlation coefficient between placebo response and treatment effects to be estimated. We estimated this from both study level estimates of the 66 trials, and individual patient data (N=560). Results Both individual patient (r=-0.32, p=0.002) and study level (r=-0.38, p<0.001) analyses yielded a negative correlation between placebo response and treatment effect. Using these estimates we found evidence of clinically significant treatment effect heterogeneity for total symptoms (our most conservative estimate was SD = 13.5 Positive and Negative Syndrome Scale (PANSS) points). Mean treatment effects were 8.6 points which, given the estimated SD, suggests the top quartile of patients experienced beneficial treatment effects of at least 17.7 PANSS points, while the bottom quartile received no benefit as compared to placebo. Conclusions We found evidence of clinically meaningful treatment effect heterogeneity for antipsychotic treatment of schizophrenia. This suggests efforts to personalise treatment have potential for success, and demonstrates that variability meta-analyses of RCTs need to account for relationships between placebo response and treatment effects.

Download Full-text

Treatment Effect Heterogeneity in Clinical Trials: An Evaluation of 13 Large Clinical Trials Using Individual Patient Data

Value in Health ◽

10.1016/j.jval.2014.08.1756 ◽

2014 ◽

Vol 17 (7) ◽

pp. A543-A544

Author(s):

D.M. Kent ◽

J. Nelson ◽

D.G. Altman ◽

R.A. Hayward

Keyword(s):

Clinical Trials ◽

Treatment Effect ◽

Individual Patient Data ◽

Patient Data ◽

Treatment Effect Heterogeneity ◽

Effect Heterogeneity

Download Full-text

Evaluation of treatment effects in patients with endometrial cancer and POLE mutations: An individual patient data meta‐analysis

Cancer ◽

10.1002/cncr.33516 ◽

2021 ◽

Author(s):

Jessica N. McAlpine ◽

Derek S. Chiu ◽

Remi A. Nout ◽

David N. Church ◽

Pascal Schmidt ◽

...

Keyword(s):

Endometrial Cancer ◽

Treatment Effects ◽

Individual Patient Data ◽

Meta Analysis ◽

Patient Data

Download Full-text

When does the use of individual patient data in network meta-analysis make a difference? A simulation study

BMC Medical Research Methodology ◽

10.1186/s12874-020-01198-2 ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Steve Kanters ◽

Mohammad Ehsanul Karim ◽

Kristian Thorlund ◽

Aslam Anis ◽

Nick Bansback

Keyword(s):

Simulation Study ◽

Treatment Effect ◽

Mean Squared Error ◽

Individual Patient Data ◽

Meta Analysis ◽

Model Performance ◽

Patient Data ◽

Meta Regression ◽

Treatment Comparisons ◽

The Impact

Abstract Background The use of individual patient data (IPD) in network meta-analyses (NMA) is rapidly growing. This study aimed to determine, through simulations, the impact of select factors on the validity and precision of NMA estimates when combining IPD and aggregate data (AgD) relative to using AgD only. Methods Three analysis strategies were compared via simulations: 1) AgD NMA without adjustments (AgD-NMA); 2) AgD NMA with meta-regression (AgD-NMA-MR); and 3) IPD-AgD NMA with meta-regression (IPD-NMA). We compared 108 parameter permutations: number of network nodes (3, 5 or 10); proportion of treatment comparisons informed by IPD (low, medium or high); equal size trials (2-armed with 200 patients per arm) or larger IPD trials (500 patients per arm); sparse or well-populated networks; and type of effect-modification (none, constant across treatment comparisons, or exchangeable). Data were generated over 200 simulations for each combination of parameters, each using linear regression with Normal distributions. To assess model performance and estimate validity, the mean squared error (MSE) and bias of treatment-effect and covariate estimates were collected. Standard errors (SE) and percentiles were used to compare estimate precision. Results Overall, IPD-NMA performed best in terms of validity and precision. The median MSE was lower in the IPD-NMA in 88 of 108 scenarios (similar results otherwise). On average, the IPD-NMA median MSE was 0.54 times the median using AgD-NMA-MR. Similarly, the SEs of the IPD-NMA treatment-effect estimates were 1/5 the size of AgD-NMA-MR SEs. The magnitude of superior validity and precision of using IPD-NMA varied across scenarios and was associated with the amount of IPD. Using IPD in small or sparse networks consistently led to improved validity and precision; however, in large/dense networks IPD tended to have negligible impact if too few IPD were included. Similar results also apply to the meta-regression coefficient estimates. Conclusions Our simulation study suggests that the use of IPD in NMA will considerably improve the validity and precision of estimates of treatment effect and regression coefficients in the most NMA IPD data-scenarios. However, IPD may not add meaningful validity and precision to NMAs of large and dense treatment networks when negligible IPD are used.

Download Full-text

Age and organ damage correlate with poor survival in myeloma patients: meta-analysis of 1435 individual patient data from 4 randomized trials

Haematologica ◽

10.3324/haematol.2012.075051 ◽

2013 ◽

Vol 98 (6) ◽

pp. 980-987 ◽

Cited By ~ 122

Author(s):

S. Bringhen ◽

M. V. Mateos ◽

S. Zweegman ◽

A. Larocca ◽

A. P. Falcone ◽

...

Keyword(s):

Randomized Trials ◽

Individual Patient Data ◽

Meta Analysis ◽

Organ Damage ◽

Patient Data ◽

Poor Survival

Download Full-text

Estimating Treatment Effects in the Presence of Noncompliance and Nonresponse: The Generalized Endogenous Treatment Model

Political Analysis ◽

10.1093/pan/mpr005 ◽

2011 ◽

Vol 19 (2) ◽

pp. 205-226 ◽

Cited By ~ 8

Author(s):

Kevin M. Esterling ◽

Michael A. Neblo ◽

David M. J. Lazer

Keyword(s):

Treatment Effect ◽

Instrumental Variable ◽

Treatment Effects ◽

Mean Squared Error ◽

Principal Stratification ◽

Squared Error ◽

Treatment Effect Heterogeneity ◽

Causal Treatment ◽

Effect Heterogeneity ◽

And Control

If ignored, noncompliance with a treatment or nonresponse on outcome measures can bias estimates of treatment effects in a randomized experiment. To identify and estimate causal treatment effects in the case where compliance and response depend on unobservables, we propose the parametric generalized endogenous treatment (GET) model. GET incorporates behavioral responses within an experiment to measure each subject's latent compliance type and identifies causal effects via principal stratification. Using simulation methods and an application to field experimental data, we show GET has a dramatically lower mean squared error for treatment effect estimates than existing approaches to principal stratification that impute, rather than measure, compliance type. In addition, we show that GET allows one to relax and test the instrumental variable exclusion restriction assumption, to test for the presence of treatment effect heterogeneity across a range of compliance types, and to test for treatment ignorability when treatment and control samples are balanced on observable covariates.

Download Full-text