A framework for extending trial design to facilitate missing data sensitivity analyses

Health economic evaluations face the issues of noncompliance and missing data. Here, noncompliance is defined as non-adherence to a specific treatment, and occurs within randomized controlled trials (RCTs) when participants depart from their random assignment. Missing data arises if, for example, there is loss-to-follow-up, survey non-response, or the information available from routine data sources is incomplete. Appropriate statistical methods for handling noncompliance and missing data have been developed, but they have rarely been applied in health economics studies. Here, we illustrate the issues and outline some of the appropriate methods with which to handle these with application to health economic evaluation that uses data from an RCT. In an RCT the random assignment can be used as an instrument-for-treatment receipt, to obtain consistent estimates of the complier average causal effect, provided the underlying assumptions are met. Instrumental variable methods can accommodate essential features of the health economic context such as the correlation between individuals’ costs and outcomes in cost-effectiveness studies. Methodological guidance for handling missing data encourages approaches such as multiple imputation or inverse probability weighting, which assume the data are Missing At Random, but also sensitivity analyses that recognize the data may be missing according to the true, unobserved values, that is, Missing Not at Random. Future studies should subject the assumptions behind methods for handling noncompliance and missing data to thorough sensitivity analyses. Modern machine-learning methods can help reduce reliance on correct model specification. Further research is required to develop flexible methods for handling more complex forms of noncompliance and missing data.

Download Full-text

Missing data sensitivity analysis for recurrent event data using controlled imputation

Pharmaceutical Statistics ◽

10.1002/pst.1624 ◽

2014 ◽

Vol 13 (4) ◽

pp. 258-264 ◽

Cited By ~ 21

Author(s):

Oliver N. Keene ◽

James H. Roger ◽

Benjamin F. Hartley ◽

Michael G. Kenward

Keyword(s):

Sensitivity Analysis ◽

Missing Data ◽

Recurrent Event ◽

Event Data ◽

Recurrent Event Data ◽

Data Sensitivity

Download Full-text

Data Missing Not at Random in Mobile Health Research: Assessment of the Problem and a Case for Sensitivity Analyses

Journal of Medical Internet Research ◽

10.2196/26749 ◽

2021 ◽

Vol 23 (6) ◽

pp. e26749

Author(s):

Simon B Goldberg ◽

Daniel M Bolt ◽

Richard J Davidson

Keyword(s):

Sensitivity Analysis ◽

Missing Data ◽

Maximum Likelihood ◽

Multiple Imputation ◽

Mixture Model ◽

Mobile Health ◽

Sensitivity Analyses ◽

Research Assessment ◽

Missing Not At Random ◽

Pattern Mixture Model

Background Missing data are common in mobile health (mHealth) research. There has been little systematic investigation of how missingness is handled statistically in mHealth randomized controlled trials (RCTs). Although some missing data patterns (ie, missing at random [MAR]) may be adequately addressed using modern missing data methods such as multiple imputation and maximum likelihood techniques, these methods do not address bias when data are missing not at random (MNAR). It is typically not possible to determine whether the missing data are MAR. However, higher attrition in active (ie, intervention) versus passive (ie, waitlist or no treatment) conditions in mHealth RCTs raise a strong likelihood of MNAR, such as if active participants who benefit less from the intervention are more likely to drop out. Objective This study aims to systematically evaluate differential attrition and methods used for handling missingness in a sample of mHealth RCTs comparing active and passive control conditions. We also aim to illustrate a modern model-based sensitivity analysis and a simpler fixed-value replacement approach that can be used to evaluate the influence of MNAR. Methods We reanalyzed attrition rates and predictors of differential attrition in a sample of 36 mHealth RCTs drawn from a recent meta-analysis of smartphone-based mental health interventions. We systematically evaluated the design features related to missingness and its handling. Data from a recent mHealth RCT were used to illustrate 2 sensitivity analysis approaches (pattern-mixture model and fixed-value replacement approach). Results Attrition in active conditions was, on average, roughly twice that of passive controls. Differential attrition was higher in larger studies and was associated with the use of MAR-based multiple imputation or maximum likelihood methods. Half of the studies (18/36, 50%) used these modern missing data techniques. None of the 36 mHealth RCTs reviewed conducted a sensitivity analysis to evaluate the possible consequences of data MNAR. A pattern-mixture model and fixed-value replacement sensitivity analysis approaches were introduced. Results from a recent mHealth RCT were shown to be robust to missing data, reflecting worse outcomes in missing versus nonmissing scores in some but not all scenarios. A review of such scenarios helps to qualify the observations of significant treatment effects. Conclusions MNAR data because of differential attrition are likely in mHealth RCTs using passive controls. Sensitivity analyses are recommended to allow researchers to assess the potential impact of MNAR on trial results.

Download Full-text

A four-step strategy for handling missing outcome data in randomised trials affected by a pandemic

10.21203/rs.3.rs-32455/v2 ◽

2020 ◽

Author(s):

Suzie Cro ◽

Tim P Morris ◽

Brennan C Kahan ◽

Victoria R Cornelius ◽

James R Carpenter

Keyword(s):

Sensitivity Analysis ◽

Missing Data ◽

Treatment Effect ◽

Missing At Random ◽

Outcome Data ◽

Sensitivity Analyses ◽

Free World ◽

Randomised Trials ◽

Primary Analysis ◽

Missing Not At Random

Abstract Background: The coronavirus pandemic (Covid-19) presents a variety of challenges for ongoing clinical trials, including an inevitably higher rate of missing outcome data, with new and non-standard reasons for missingness. International drug trial guidelines recommend trialists review plans for handling missing data in the conduct and statistical analysis, but clear recommendations are lacking.Methods: We present a four-step strategy for handling missing outcome data in the analysis of randomised trials that are ongoing during a pandemic. We consider handling missing data arising due to (i) participant infection, (ii) treatment disruptions and (iii) loss to follow-up. We consider both settings where treatment effects for a ‘pandemic-free world’ and ‘world including a pandemic’ are of interest. Results: In any trial, investigators should; (1) Clarify the treatment estimand of interest with respect to the occurrence of the pandemic; (2) Establish what data are missing for the chosen estimand; (3) Perform primary analysis under the most plausible missing data assumptions followed by; (4) Sensitivity analysis under alternative plausible assumptions. To obtain an estimate of the treatment effect in a ‘pandemic-free world’, participant data that are clinically affected by the pandemic (directly due to infection or indirectly via treatment disruptions) are not relevant and can be set to missing. For primary analysis, a missing-at-random assumption that conditions on all observed data that are expected to be associated with both the outcome and missingness may be most plausible. For the treatment effect in the ‘world including a pandemic’, all participant data is relevant and should be included in the analysis. For primary analysis, a missing-at-random assumption – potentially incorporating a pandemic time-period indicator and participant infection status – or a missing-not-at-random assumption with a poorer response may be most relevant, depending on the setting. In all scenarios, sensitivity analysis under credible missing-not-at-random assumptions should be used to evaluate the robustness of results. We highlight controlled multiple imputation as an accessible tool for conducting sensitivity analyses.Conclusions: Missing data problems will be exacerbated for trials active during the Covid-19 pandemic. This four-step strategy will facilitate clear thinking about the appropriate analysis for relevant questions of interest.

Download Full-text

Effect of abiraterone acetate (AA) on patient-reported pain in metastatic castration-resistant prostate cancer (mCRPC) post-docetaxel: Results of longitudinal sensitivity analyses.

Journal of Clinical Oncology ◽

10.1200/jco.2013.31.15_suppl.9618 ◽

2013 ◽

Vol 31 (15_suppl) ◽

pp. 9618-9618 ◽

Cited By ~ 2

Author(s):

Yanni Hao ◽

Charles S. Cleeland ◽

Dennis Gagnon ◽

Derek Espindle ◽

Arturo Molina ◽

...

Keyword(s):

Missing Data ◽

Pain Intensity ◽

Short Form ◽

Mixed Effects ◽

Pain Interference ◽

Sensitivity Analyses ◽

Repeated Measure ◽

Patient Reported ◽

Pain Outcomes ◽

The Impact

9618 Background: The COU-AA-301 phase 3 trial showed that AA + prednisone (P) improved overall survival in mCRPC patients (pts) post-docetaxel. Compared with P alone, AA + P also had significant benefits on patient-reported pain. Here we describe post hoc sensitivity analyses of pain data from that trial, using different methods to compensate for the potential impact of missing data. Methods: Pts with mCRPC progressing after docetaxel-based chemotherapy were randomized 2:1 to AA + P or placebo + P. Pain intensity and interference of pain with daily activities were assessed with the Brief Pain Inventory-Short Form (BPI-SF) questionnaire at baseline, Day 15 of Cycle 1, and Day 1 of each 28-day treatment cycle thereafter until treatment discontinuation. The effect of treatment on BPI-SF scores was analyzed using repeated measure mixed-effects (RMM) models, piecewise linear mixed-effects (PWLME) models, and joint mixed-effects and log time-to-dropout (JMEL) models. RMM and PWMLE models assumed missing data (due to death, study dropout, or administrative issues) to be missing at random, the JMEL model to be missing not at random. Model results were compared between treatment arms. Results: 797 pts were randomized to AA + P, and 398 to P only. RMM model estimates suggested statistically significant (p < 0.05) differences in change from baseline for pain intensity and pain interference scores in favor of AA + P at the majority of study visits through cycle 11. PWLME models yielded significantly smaller areas under the curve (AUCs) for AA + P vs P for pain intensity (p = 0.0031) and pain interference (p = 0.0006); smaller AUCs reflect better pain outcomes. Results using JMEL models were nearly identical to those with PWLME models, with AUCs for AA + P significantly smaller than for P alone for pain intensity (p = 0.0031) and pain interference (p = 0.0007). Conclusions: Using various modeling methods that assess the impact of missing data, AA + P showed superior patterns of pain outcomes over time compared with P only in mCRPC pts refractory to docetaxel. These results support the previously reported pain benefits of AA + P over P alone from the same trial. Clinical trial information: NCT00638690.

Download Full-text

Nuclear Data Sensitivity Analyses of OKTAVIAN Shielding Benchmark Experiments

Journal of Nuclear Science and Technology ◽

10.1080/00223131.2008.10875788 ◽

2008 ◽

Vol 45 (sup5) ◽

pp. 70-73

Author(s):

Do Heon Kim ◽

Choong-Sup Gil ◽

Young-Ouk Lee

Keyword(s):

Nuclear Data ◽

Sensitivity Analyses ◽

Data Sensitivity

Download Full-text

Validating and comparing stroke prognosis scales

Neurology ◽

10.1212/wnl.0000000000004332 ◽

2017 ◽

Vol 89 (10) ◽

pp. 997-1002 ◽

Cited By ~ 23

Author(s):

Terence J. Quinn ◽

Sarjit Singh ◽

Kennedy R. Lees ◽

Philip M. Bath ◽

Phyo K. Myint

Keyword(s):

Clinical Trials ◽

Missing Data ◽

Acute Stroke ◽

Clinical Decision Making ◽

Sensitivity Analyses ◽

Systematic Bias ◽

Modified Rankin Scale ◽

Vascular Events ◽

Prognostic Accuracy ◽

Significant Difference

Objective:To compare the prognostic accuracy of various acute stroke prognostic scales using a large, independent, clinical trials dataset.Methods:We directly compared 8 stroke prognostic scales, chosen based on focused literature review (Acute Stroke Registry and Analysis of Lausanne [ASTRAL]; iSCORE; iSCORE-revised; preadmission comorbidities, level of consciousness, age, and neurologic deficit [PLAN]; stroke subtype, Oxfordshire Community Stroke Project, age, and prestroke modified Rankin Scale [mRS] [SOAR]; modified SOAR; Stroke Prognosis Instrument 2 [SPI2]; and Totaled Health Risks in Vascular Events [THRIVE]) using individual patient-level data from a clinical trials archive (Virtual International Stroke Trials Archive [VISTA]). We calculated area under receiver operating characteristic curves (AUROC) for each scale against 90-day outcomes of mRS (dichotomized at mRS >2), Barthel Index (>85), and mortality. We performed 2 complementary analyses: the first limited to patients with complete data for all components of all scales (simultaneous) and the second using as many patients as possible for each individual scale (separate). We compared AUROCs and performed sensitivity analyses substituting extreme outcome values for missing data.Results:In total, 10,777 patients contributed to the analyses. Our simultaneous analyses suggested that ASTRAL had greatest prognostic accuracy for mRS, AUROC 0.78 (95% confidence interval [CI] 0.75–0.82), and SPI2 had poorest AUROC, 0.61 (95% CI 0.57–0.66). Our separate analyses confirmed these results: ASTRAL AUROC 0.79 (95% CI 0.78–0.80 and SPI2 AUROC 0.60 (95% CI 0.59–0.61). On formal comparative testing, there was a significant difference in modified Rankin Scale AUROC between ASTRAL and all other scales. Sensitivity analysis identified no evidence of systematic bias from missing data.Conclusions:Our comparative analyses confirm differences in the prognostic accuracy of stroke scales. However, even the best performing scale had prognostic accuracy that may not be sufficient as a basis for clinical decision-making.

Download Full-text

Bayesian analysis of longitudinal dyadic data with informative missing data using a dyadic shared-parameter model

Statistical Methods in Medical Research ◽

10.1177/0962280217715051 ◽

2017 ◽

Vol 28 (1) ◽

pp. 70-83 ◽

Cited By ~ 1

Author(s):

Jaeil Ahn ◽

Satoshi Morita ◽

Wenyi Wang ◽

Ying Yuan

Keyword(s):

Missing Data ◽

Random Effects ◽

Metastatic Breast ◽

Measurement Model ◽

Repeated Measurements ◽

Sensitivity Analyses ◽

Simulation Studies ◽

Dyadic Data ◽

Longitudinal Measurement ◽

Informative Missing Data

Analyzing longitudinal dyadic data is a challenging task due to the complicated correlations from repeated measurements and within-dyad interdependence, as well as potentially informative (or non-ignorable) missing data. We propose a dyadic shared-parameter model to analyze longitudinal dyadic data with ordinal outcomes and informative intermittent missing data and dropouts. We model the longitudinal measurement process using a proportional odds model, which accommodates the within-dyad interdependence using the concept of the actor-partner interdependence effects, as well as dyad-specific random effects. We model informative dropouts and intermittent missing data using a transition model, which shares the same set of random effects as the longitudinal measurement model. We evaluate the performance of the proposed method through extensive simulation studies. As our approach relies on some untestable assumptions on the missing data mechanism, we perform sensitivity analyses to evaluate how the analysis results change when the missing data mechanism is misspecified. We demonstrate our method using a longitudinal dyadic study of metastatic breast cancer.

Download Full-text