Sensitivity to missing not at random dropout in clinical trials: use and interpretation of the Trimmed Means Estimator

AbstractOutcome values in randomized controlled trials (RCTs) may be missing not at random (MNAR), if patients with extreme outcome values are more likely to drop out (e.g., due to perceived ineffectiveness of treatment, or adverse effects). In such scenarios, estimates from complete case analysis (CCA) and multiple imputation (MI) will be biased. The trimmed means (TM) estimator operates by setting missing values to the most extreme value, and then “trimming” away equal fractions of both treatment groups, estimating the treatment effect using the remaining data. The TM estimator relies on two assumptions, which we term the “strong MNAR” and “location shift” assumptions. In this article, we derive formulae for the bias resulting from the violation of these assumptions for normally distributed outcomes. We propose an adjusted estimator, which relaxes the location shift assumption and detail how our bias formulae can be used to establish the direction of bias of CCA, MI and TM estimates under a range of plausible data scenarios, to inform sensitivity analyses. The TM approach is illustrated with simulations and in a sensitivity analysis of the CoBalT RCT of cognitive behavioural therapy (CBT) in 469 individuals with 46 months follow-up. Results were consistent with a beneficial CBT treatment effect. The MI estimates are closer to the null than the CCA estimate, whereas the TM estimate was further from the null. We propose using the TM estimator as a sensitivity analysis for data where it is suspected that extreme outcome values are missing.

Download Full-text

A four-step strategy for handling missing outcome data in randomised trials affected by a pandemic

10.21203/rs.3.rs-32455/v2 ◽

2020 ◽

Author(s):

Suzie Cro ◽

Tim P Morris ◽

Brennan C Kahan ◽

Victoria R Cornelius ◽

James R Carpenter

Keyword(s):

Sensitivity Analysis ◽

Missing Data ◽

Treatment Effect ◽

Missing At Random ◽

Outcome Data ◽

Sensitivity Analyses ◽

Free World ◽

Randomised Trials ◽

Primary Analysis ◽

Missing Not At Random

Abstract Background: The coronavirus pandemic (Covid-19) presents a variety of challenges for ongoing clinical trials, including an inevitably higher rate of missing outcome data, with new and non-standard reasons for missingness. International drug trial guidelines recommend trialists review plans for handling missing data in the conduct and statistical analysis, but clear recommendations are lacking.Methods: We present a four-step strategy for handling missing outcome data in the analysis of randomised trials that are ongoing during a pandemic. We consider handling missing data arising due to (i) participant infection, (ii) treatment disruptions and (iii) loss to follow-up. We consider both settings where treatment effects for a ‘pandemic-free world’ and ‘world including a pandemic’ are of interest. Results: In any trial, investigators should; (1) Clarify the treatment estimand of interest with respect to the occurrence of the pandemic; (2) Establish what data are missing for the chosen estimand; (3) Perform primary analysis under the most plausible missing data assumptions followed by; (4) Sensitivity analysis under alternative plausible assumptions. To obtain an estimate of the treatment effect in a ‘pandemic-free world’, participant data that are clinically affected by the pandemic (directly due to infection or indirectly via treatment disruptions) are not relevant and can be set to missing. For primary analysis, a missing-at-random assumption that conditions on all observed data that are expected to be associated with both the outcome and missingness may be most plausible. For the treatment effect in the ‘world including a pandemic’, all participant data is relevant and should be included in the analysis. For primary analysis, a missing-at-random assumption – potentially incorporating a pandemic time-period indicator and participant infection status – or a missing-not-at-random assumption with a poorer response may be most relevant, depending on the setting. In all scenarios, sensitivity analysis under credible missing-not-at-random assumptions should be used to evaluate the robustness of results. We highlight controlled multiple imputation as an accessible tool for conducting sensitivity analyses.Conclusions: Missing data problems will be exacerbated for trials active during the Covid-19 pandemic. This four-step strategy will facilitate clear thinking about the appropriate analysis for relevant questions of interest.

Download Full-text

A four-step strategy for handling missing outcome data in randomised trials affected by a pandemic

10.21203/rs.3.rs-32455/v1 ◽

2020 ◽

Author(s):

Suzie Cro ◽

Tim P Morris ◽

Brennan C Kahan ◽

Victoria R Cornelius ◽

James R Carpenter

Keyword(s):

Sensitivity Analysis ◽

Missing Data ◽

Treatment Effect ◽

Missing At Random ◽

Outcome Data ◽

Sensitivity Analyses ◽

Free World ◽

Randomised Trials ◽

Primary Analysis ◽

Missing Not At Random

Abstract Background The coronavirus pandemic (Covid-19) presents a variety of challenges for ongoing clinical trials, including an inevitably higher rate of missing outcome data, with new and non-standard reasons for missingness. International drug trial guidelines recommend trialists review plans for handling missing data in the conduct and statistical analysis, but clear recommendations are lacking. Methods We present a four-step strategy for handling missing outcome data in the analysis of randomised trials that are ongoing during a pandemic. We consider handling missing data arising due to (i) participant infection, (ii) treatment disruptions and (iii) loss to follow-up. We consider both settings where treatment effects for a ‘pandemic-free world’ and ‘world including a pandemic’ are of interest. Results In any trial, investigators should; (1) Clarify the treatment estimand of interest; (2) Establish what data are missing for the estimand at hand; (3) Perform primary analysis under the most plausible missing data assumptions followed by; (4) Sensitivity analysis under alternative plausible assumptions. To obtain an estimate of the treatment effect in a ‘pandemic-free world’, data from participants clinically affected by the pandemic (directly via infection or indirectly via treatment disruptions) are not relevant and can be set to missing. For primary analysis, a missing-at-random assumption that conditions on all observed data that are expected to be associated with both the outcome and missingness may be most plausible. For the treatment effect in the ‘world including a pandemic’, all participant data is relevant and should be included in the analysis. For primary analysis, a missing-at-random assumption – potentially incorporating a pandemic time-period indicator and participant infection status – or a missing-not-at-random assumption with a poorer response may be most relevant, depending on the setting. In all scenarios, sensitivity analysis under credible missing-not-at-random assumptions should be used to evaluate the robustness of results. We highlight controlled multiple imputation as an accessible tool for conducting sensitivity analyses. Conclusions Missing data problems will be exacerbated for trials active during the Covid-19 pandemic. This four-step strategy will facilitate clear thinking about the appropriate analysis for relevant questions of interest.

Download Full-text

What we know, what we do not know, and where are we heading? Efficacy and acceptability of psychological interventions for depression

Epidemiology and Psychiatric Sciences ◽

10.1017/s2045796015000815 ◽

2015 ◽

Vol 25 (4) ◽

pp. 301-308 ◽

Cited By ~ 5

Author(s):

N. Solomonov ◽

J. P. Barber

Keyword(s):

Randomized Clinical Trials ◽

Behavioural Therapy ◽

Drop Out ◽

Interpersonal Therapy ◽

Problem Solving Therapy ◽

Cognitive Behavioural ◽

Treatment Of Depression ◽

Personalised Treatment ◽

Meta Analyses ◽

And Control

In the past several decades, increasing evidence supports the efficacy of psychotherapies for depression. The vast majority of findings from meta-analyses, randomized clinical trials (RCTs) and naturalistic studies have demonstrated that well-established psychotherapies (behavioural activation, problem-solving therapy, psychodynamic therapy, cognitive-behavioural therapy, interpersonal therapy and emotion-focused therapy) are superior to no-treatment and control conditions, and are in most cases equally effective in treating depression. However, despite this abundant support for psychotherapies, studies have also consistently shown high drop-out rates, high percentages of non-respondent patients who experience treatment failures, and mixed findings regarding the enduring effects of psychotherapy. Thus, there is a need to develop more personalised treatment models tailored to patients’ needs. A new integrative sequential stepwise approach to the treatment of depression is suggested.

Download Full-text

Measuring the sensitivity of difference-in-difference estimates to the parallel trends assumption

Research Methods in Medicine & Health Sciences ◽

10.1177/26320843211061306 ◽

2021 ◽

pp. 263208432110613

Author(s):

Landon Gibson ◽

Frederick Zimmerman

Keyword(s):

Sensitivity Analysis ◽

Treatment Period ◽

Treatment Effect ◽

Statistical Tests ◽

Sensitivity Analyses ◽

Difference In Difference ◽

Control Groups ◽

Pre Treatment ◽

Background Difference ◽

And Control

Background. Difference-in-Difference makes a critical assumption that the changes in the outcomes, over the post-treatment period, are similar between the treated and control groups—the parallel trends assumption. Evaluation of this assumption is often done either by graphical examination or by statistical tests in the pre-treatment period. They result in a binary conclusion about the validity of the assumption. Purpose. This paper proposes a sensitivity analysis that quantifies the departure from parallel trends necessary to meaningfully change the estimated treatment effect. Results. Sensitivity analyses have an advantage over traditional parallel trends tests: they use all available data and thereby work even if only one pre-period is available, and they quantify the strength of unobserved confounder(s) required to change the conclusions of a study. Conclusions. We apply the sensitivity analysis metrics developed by Cinelli and Hazlett (2020) and illustrate them on two studies.

Download Full-text

Computerised cognitive-behavioural therapy for depression: systematic review

The British Journal of Psychiatry ◽

10.1192/bjp.bp.106.025981 ◽

2008 ◽

Vol 193 (3) ◽

pp. 181-184 ◽

Cited By ~ 125

Author(s):

Eva Kaltenthaler ◽

Glenys Parry ◽

Catherine Beverley ◽

Michael Ferriter

Keyword(s):

Cognitive Behavioural Therapy ◽

Treatment Options ◽

Computer Software ◽

Behavioural Therapy ◽

Drop Out ◽

Potential Range ◽

Treatment As Usual ◽

Cognitive Behavioural ◽

Randomised Controlled ◽

Depression Education

BackgroundComputerised cognitive–behavioural therapy (CCBT) is used for treating depression and provides a potentially useful alternative to therapist cognitive–behavioural therapy (CBT).AimsTo systematically review the evidence for the effectiveness of CCBT for the treatment of mild to moderate depression.MethodElectronic databases were searched to identify randomised controlled trials. Selected studies were quality assessed and data extracted by two reviewers.ResultsFour studies of three computer software packages met the inclusion criteria. Comparators were treatment as usual, using a depression education website and an attention placebo.ConclusionsThere is some evidence to support the effectiveness of CCBT for the treatment of depression. However, all studies were associated with considerable drop-out rates and little evidence was presented regarding participants' preferences and the acceptability of the therapy. More research is needed to determine the place of CCBT in the potential range of treatment options offered to individuals with depression.

Download Full-text

Does Starting With the Behavioural Component of Cognitive Behavioural Therapy (CBT) Increase Patients’ Retention in Therapy?

Behaviour Change ◽

10.1017/bec.2012.23 ◽

2012 ◽

Vol 29 (4) ◽

pp. 238-257 ◽

Cited By ~ 1

Author(s):

Nusrat Yasmeen Ahmed ◽

Sharon Lawn

Keyword(s):

Cognitive Behavioural Therapy ◽

Behavioural Therapy ◽

Drop Out ◽

Psychosocial Assessment ◽

Anxiety And Depression ◽

Behavioural Intervention ◽

Behavioural Interventions ◽

Comorbid Anxiety ◽

Cognitive Behavioural ◽

Different Types

This study examined whether starting with the behavioural component of cognitive behavioural therapy (CBT) decreases the drop-out rate in outpatients with comorbid anxiety and depression. Retrospective data were collected on 60 patients with anxiety and depression. Mean values of different psychosocial assessment scales during screening, mid-session and discharge session were compared between the patients receiving and not receiving any type of behavioural interventions and among the patients receiving different types of behavioural interventions. A significant relationship was found (p < .05) between behavioural interventions and retention in therapy. Patients who did not receive any sort of behavioural intervention showed a greater rate of drop-out than those who received behavioural interventions. In the group of patients receiving different types of behavioural interventions, there was significant improvement in mental health scores between the screening and discharge sessions in those who received exposure therapy. The study findings will be helpful to retain patients with comorbid anxiety and depression in an outpatient therapy setting. If patient retention is increased, CBT can be more effectively delivered and thereby achieve better health outcomes for patients, more effective use of therapy service resources, and decrease the socioeconomic burden of anxiety and depression on the community.

Download Full-text

Data Missing Not at Random in Mobile Health Research: Assessment of the Problem and a Case for Sensitivity Analyses

Journal of Medical Internet Research ◽

10.2196/26749 ◽

2021 ◽

Vol 23 (6) ◽

pp. e26749

Author(s):

Simon B Goldberg ◽

Daniel M Bolt ◽

Richard J Davidson

Keyword(s):

Sensitivity Analysis ◽

Missing Data ◽

Maximum Likelihood ◽

Multiple Imputation ◽

Mixture Model ◽

Mobile Health ◽

Sensitivity Analyses ◽

Research Assessment ◽

Missing Not At Random ◽

Pattern Mixture Model

Background Missing data are common in mobile health (mHealth) research. There has been little systematic investigation of how missingness is handled statistically in mHealth randomized controlled trials (RCTs). Although some missing data patterns (ie, missing at random [MAR]) may be adequately addressed using modern missing data methods such as multiple imputation and maximum likelihood techniques, these methods do not address bias when data are missing not at random (MNAR). It is typically not possible to determine whether the missing data are MAR. However, higher attrition in active (ie, intervention) versus passive (ie, waitlist or no treatment) conditions in mHealth RCTs raise a strong likelihood of MNAR, such as if active participants who benefit less from the intervention are more likely to drop out. Objective This study aims to systematically evaluate differential attrition and methods used for handling missingness in a sample of mHealth RCTs comparing active and passive control conditions. We also aim to illustrate a modern model-based sensitivity analysis and a simpler fixed-value replacement approach that can be used to evaluate the influence of MNAR. Methods We reanalyzed attrition rates and predictors of differential attrition in a sample of 36 mHealth RCTs drawn from a recent meta-analysis of smartphone-based mental health interventions. We systematically evaluated the design features related to missingness and its handling. Data from a recent mHealth RCT were used to illustrate 2 sensitivity analysis approaches (pattern-mixture model and fixed-value replacement approach). Results Attrition in active conditions was, on average, roughly twice that of passive controls. Differential attrition was higher in larger studies and was associated with the use of MAR-based multiple imputation or maximum likelihood methods. Half of the studies (18/36, 50%) used these modern missing data techniques. None of the 36 mHealth RCTs reviewed conducted a sensitivity analysis to evaluate the possible consequences of data MNAR. A pattern-mixture model and fixed-value replacement sensitivity analysis approaches were introduced. Results from a recent mHealth RCT were shown to be robust to missing data, reflecting worse outcomes in missing versus nonmissing scores in some but not all scenarios. A review of such scenarios helps to qualify the observations of significant treatment effects. Conclusions MNAR data because of differential attrition are likely in mHealth RCTs using passive controls. Sensitivity analyses are recommended to allow researchers to assess the potential impact of MNAR on trial results.

Download Full-text

Early Improvement in Eating Attitudes during Cognitive Behavioural Therapy for Eating Disorders: The Impact of Personality Disorder Cognitions

Behavioural and Cognitive Psychotherapy ◽

10.1017/s1352465812001117 ◽

2013 ◽

Vol 42 (2) ◽

pp. 224-237 ◽

Cited By ~ 3

Author(s):

Emma C. Park ◽

Glenn Waller ◽

Kenneth Gannon

Keyword(s):

Eating Disorders ◽

Eating Disorder ◽

Personality Disorder ◽

Personality Disorders ◽

Cognitive Behavioural Therapy ◽

Behavioural Therapy ◽

Eating Attitudes ◽

Drop Out ◽

Cognitive Behavioural ◽

The Impact

Background: The personality disorders are commonly comorbid with the eating disorders. Personality disorder pathology is often suggested to impair the treatment of axis 1 disorders, including the eating disorders. Aims: This study examined whether personality disorder cognitions reduce the impact of cognitive behavioural therapy (CBT) for eating disorders, in terms of treatment dropout and change in eating disorder attitudes in the early stages of treatment. Method: Participants were individuals with a diagnosed eating disorder, presenting for individual outpatient CBT. They completed measures of personality disorder cognitions and eating disorder attitudes at sessions one and six of CBT. Drop-out rates prior to session six were recorded. Results: CBT had a relatively rapid onset of action, with a significant reduction in eating disorder attitudes over the first six sessions. Eating disorder attitudes were most strongly associated with cognitions related to anxiety-based personality disorders (avoidant, obsessive-compulsive and dependent). Individuals who dropped out of treatment prematurely had significantly higher levels of dependent personality disorder cognitions than those who remained in treatment. For those who remained in treatment, higher levels of avoidant, histrionic and borderline personality disorder cognitions were associated with a greater change in global eating disorder attitudes. Conclusions: CBT's action and retention of patients might be improved by consideration of such personality disorder cognitions when formulating and treating the eating disorders.

Download Full-text