scholarly journals Missing Data in Experiments: Challenges and Solutions

2019 ◽  
Author(s):  
Robin Gomila ◽  
Chelsey S. Clark
Keyword(s):  
Missing Data ◽  
Inverse Probability Weighting ◽  
Effect Sizes ◽  
Double Sampling ◽  
Probability Weighting ◽  
Random Assignment ◽  
Causal Inferences ◽  
Standard Methods ◽  
Inverse Probability

Missing data is a common feature of experimental datasets. Standard methods used by psychology researchers to handle missingness rely on unrealistic assumptions, invalidate random assignment procedures, and bias estimates of effect sizes. In this tutorial, we describe different classes of missing data typically encountered in experimental datasets, and we discuss how each of them impacts researchers' causal inferences. We provide concrete guidelines for handling each class of missingness, focusing on two methods that make realistic assumptions: i) Inverse Probability Weighting (IPW) for mild instances of missingness, and ii) Double Sampling and Bounds for severe instances of missingness. After reviewing the reasons why these methods increase the accuracy of researchers' estimates of effect sizes, we provide lines of R code that researchers may use in their own analyses.

Adjusting for selection bias due to missing data in electronic health records-based research

2021 ◽  
Vol 30 (10) ◽  
pp. 2221-2238
Author(s):  
Sarah B Peskoe ◽  
David Arterburn ◽  
Karen J Coleman ◽  
Lisa J Herrinton ◽  
Michael J Daniels ◽  
...  
Keyword(s):  
Missing Data ◽  
Electronic Health Records ◽  
Selection Bias ◽  
Inverse Probability Weighting ◽  
Small Sample ◽  
Data Provenance ◽  
Probability Weighting ◽  
Health Records ◽  
Inverse Probability ◽  
Electronic Health

While electronic health records data provide unique opportunities for research, numerous methodological issues must be considered. Among these, selection bias due to incomplete/missing data has received far less attention than other issues. Unfortunately, standard missing data approaches (e.g. inverse-probability weighting and multiple imputation) generally fail to acknowledge the complex interplay of heterogeneous decisions made by patients, providers, and health systems that govern whether specific data elements in the electronic health records are observed. This, in turn, renders the missing-at-random assumption difficult to believe in standard approaches. In the clinical literature, the collection of decisions that gives rise to the observed data is referred to as the data provenance. Building on a recently-proposed framework for modularizing the data provenance, we develop a general and scalable framework for estimation and inference with respect to regression models based on inverse-probability weighting that allows for a hierarchy of missingness mechanisms to better align with the complex nature of electronic health records data. We show that the proposed estimator is consistent and asymptotically Normal, derive the form of the asymptotic variance, and propose two consistent estimators. Simulations show that naïve application of standard methods may yield biased point estimates, that the proposed estimators have good small-sample properties, and that researchers may have to contend with a bias-variance trade-off as they consider how to handle missing data. The proposed methods are motivated by an on-going, electronic health records-based study of bariatric surgery.

scholarly journals Responsiveness-informed multiple imputation and inverse probability-weighting in cohort studies with missing data that are non-monotone or not missing at random

2016 ◽  
Vol 27 (2) ◽  
pp. 352-363 ◽  
Author(s):  
James C Doidge
Keyword(s):  
Missing Data ◽  
Data Collection ◽  
Cohort Studies ◽  
Multiple Imputation ◽  
Missing At Random ◽  
Inverse Probability Weighting ◽  
Probability Weighting ◽  
Inverse Probability ◽  
Not Missing At Random ◽  
Over Time

Population-based cohort studies are invaluable to health research because of the breadth of data collection over time, and the representativeness of their samples. However, they are especially prone to missing data, which can compromise the validity of analyses when data are not missing at random. Having many waves of data collection presents opportunity for participants’ responsiveness to be observed over time, which may be informative about missing data mechanisms and thus useful as an auxiliary variable. Modern approaches to handling missing data such as multiple imputation and maximum likelihood can be difficult to implement with the large numbers of auxiliary variables and large amounts of non-monotone missing data that occur in cohort studies. Inverse probability-weighting can be easier to implement but conventional wisdom has stated that it cannot be applied to non-monotone missing data. This paper describes two methods of applying inverse probability-weighting to non-monotone missing data, and explores the potential value of including measures of responsiveness in either inverse probability-weighting or multiple imputation. Simulation studies are used to compare methods and demonstrate that responsiveness in longitudinal studies can be used to mitigate bias induced by missing data, even when data are not missing at random.

scholarly journals Study Designs for Extending Causal Inferences from a Randomized Trial to a Target Population

2020 ◽  
Author(s):  
Issa J Dahabreh ◽  
Sebastien J P A Haneuse ◽  
James M Robins ◽  
Sarah E Robertson ◽  
Ashley L Buchanan ◽  
...  
Keyword(s):  
Randomized Trial ◽  
Study Design ◽  
Target Population ◽  
Inverse Probability Weighting ◽  
Trial Participation ◽  
Probability Weighting ◽  
Causal Inferences ◽  
Inverse Probability ◽  
Study Designs

Abstract We examine study designs for extending (generalizing or transporting) causal inferences from a randomized trial to a target population. Specifically, we consider nested trial designs, where randomized individuals are nested within a sample from the targetpopulation, and non-nested trial designs, including composite dataset designs, where a randomized trial is combined with a separately obtained sample of non-randomized individuals from the target population. We show that the counterfactual quantities that can be identified in each study design depend on what is known about the probability of sampling non-randomized individuals. For each study design, we examine identification of counterfactual outcome means via the g-formula and inverse probability weighting. Last, we explore the implications of the sampling properties underlying the designs for the identification and estimation of the probability of trial participation.

Scaled Inverse Probability Weighting: A Method to Assess Potential Bias Due to Event Nonreporting in Ecological Momentary Assessment Studies

2017 ◽  
Vol 43 (3) ◽  
pp. 354-381 ◽  
Author(s):  
Stephanie A. Kovalchik ◽  
Steven C. Martino ◽  
Rebecca L. Collins ◽  
William G. Shadel ◽  
Elizabeth J. D’Amico ◽  
...  
Keyword(s):  
Missing Data ◽  
Ecological Momentary Assessment ◽  
Inverse Probability Weighting ◽  
Assessment Method ◽  
Probability Weighting ◽  
Inverse Probability ◽  
Traditional Assessment ◽  
Urge To Smoke ◽  
Ecological Momentary ◽  
Momentary Assessment

Ecological momentary assessment (EMA) is a popular assessment method in psychology that aims to capture events, emotions, and cognitions in real time, usually repeatedly throughout the day. Because EMA typically involves more intensive monitoring than traditional assessment methods, missing data are commonly an issue and this missingness may bias results. EMA can involve two types of missing data: known missingness, arising from nonresponse to scheduled prompts, and hidden missingness, arising from nonreporting of focal events (e.g., an urge to smoke or a meal). Prior research on missing data in EMA has focused almost exclusively on nonresponse to scheduled prompts. In this study, we introduce a scaled inverse probability weighting approach to assess the risk of bias due to nonreporting of events due to fatigue on estimates of exposure or correlates of exposure. In our proposed approach, the inverse probability is the estimated probability of compliance with random prompts from a model that uses participant and contextual factors to predict this compliance and a fatigue factor that adjusts for attrition in event reporting over time. We demonstrate the use and utility of our bias assessment method with the Tracking and Recording Alcohol Communications Study, an EMA study of adolescent exposure to alcohol advertising.

scholarly journals A Case Study of the Impact of Data-Adaptive Versus Model-Based Estimation of the Propensity Scores on Causal Inferences from Three Inverse Probability Weighting Estimators

2016 ◽  
Vol 12 (1) ◽  
pp. 131-155 ◽  
Author(s):  
Romain Neugebauer ◽  
Julie A. Schmittdiel ◽  
Mark J. van der Laan
Keyword(s):  
Propensity Scores ◽  
Adaptive Estimation ◽  
Inverse Probability Weighting ◽  
Model Specification ◽  
Probability Weighting ◽  
Consistent Estimation ◽  
Causal Inferences ◽  
Inverse Probability ◽  
Estimation Strategy ◽  
Data Adaptive

Abstract:Objective: Consistent estimation of causal effects with inverse probability weighting estimators is known to rely on consistent estimation of propensity scores. To alleviate the bias expected from incorrect model specification for these nuisance parameters in observational studies, data-adaptive estimation and in particular an ensemble learning approach known as Super Learning has been proposed as an alternative to the common practice of estimation based on arbitrary model specification. While the theoretical arguments against the use of the latter haphazard estimation strategy are evident, the extent to which data-adaptive estimation can improve inferences in practice is not. Some practitioners may view bias concerns over arbitrary parametric assumptions as academic considerations that are inconsequential in practice. They may also be wary of data-adaptive estimation of the propensity scores for fear of greatly increasing estimation variability due to extreme weight values. With this report, we aim to contribute to the understanding of the potential practical consequences of the choice of estimation strategy for the propensity scores in real-world comparative effectiveness research.Method: We implement secondary analyses of Electronic Health Record data from a large cohort of type 2 diabetes patients to evaluate the effects of four adaptive treatment intensification strategies for glucose control (dynamic treatment regimens) on subsequent development or progression of urinary albumin excretion. Three Inverse Probability Weighting estimators are implemented using both model-based and data-adaptive estimation strategies for the propensity scores. Their practical performances for proper confounding and selection bias adjustment are compared and evaluated against results from previous randomized experiments.Conclusion: Results suggest both potential reduction in bias and increase in efficiency at the cost of an increase in computing time when using Super Learning to implement Inverse Probability Weighting estimators to draw causal inferences.

scholarly journals On weighting approaches for missing data

2011 ◽  
Vol 22 (1) ◽  
pp. 14-30 ◽  
Author(s):  
Lingling Li ◽  
Changyu Shen ◽  
Xiaochun Li ◽  
James M Robins
Keyword(s):  
Missing Data ◽  
Selection Bias ◽  
Inverse Probability Weighting ◽  
Probability Weighting ◽  
Full Data ◽  
Inverse Probability ◽  
Intuitive Idea ◽  
Complex Settings ◽  
Conceptual Overview

We review the class of inverse probability weighting (IPW) approaches for the analysis of missing data under various missing data patterns and mechanisms. The IPW methods rely on the intuitive idea of creating a pseudo-population of weighted copies of the complete cases to remove selection bias introduced by the missing data. However, different weighting approaches are required depending on the missing data pattern and mechanism. We begin with a uniform missing data pattern (i.e. a scalar missing indicator indicating whether or not the full data is observed) to motivate the approach. We then generalise to more complex settings. Our goal is to provide a conceptual overview of existing IPW approaches and illustrate the connections and differences among these approaches.

Combining Multiple Imputation and Inverse-Probability Weighting for Analyzing Response with Missing in the Presence of Covariates

2020 ◽  
Author(s):  
Freshteh Osmani ◽  
Ebrahim Hajizadeh
Keyword(s):  
Missing Data ◽  
Missing Values ◽  
Clinical Trial Data ◽  
Real Data ◽  
Inverse Probability Weighting ◽  
Trial Data ◽  
Data Sets ◽  
Probability Weighting ◽  
Inverse Probability ◽  
Doubly Robust

Introduction: Missing values are frequently seen in data sets of research studiesespecially in medical studies.Therefore, it is essential that the data, especially in medical research should evaluate in terms of the structure of missingness.This study aims to provide new statistical methods for analyzing such data. Methods:Multiple imputation (MI) and inverse-probability weighting (IPW)aretwo common methods whichused to deal with missing data. MI method is more effectiveand complexthan IPW.MI requires a model for the joint distribution of the missing data given the observed data.While IPW need only a model for the probability that a subject has fulldata .Inefficacy in each of these models may causeto serious bias if missingness in dataset is large .Anothermethod that combines these approaches to give a doubly robust estimator.In addition, using of these methodswill demonstrate in the clinical trial data related to postpartum bleeding. Results:In this article, we examine the performance of IPW/MI relative to MI and IPW alone in terms of bias and efficiency.According to the results of simulation can be said that that IPW/MI have advantages over alternatives.Also results of real data showed that,results of MI/MI doesnot differ with the results ofIPW/MIsignificantly. Conclusion:Problem of missing data are in many studies that causes bias and decreasing efficacy inmodel.In this study, after comparing the results of these techniques,it was concludedthat IPW/MI method has better performance than other methods.

Sign in / Sign up

Export Citation Format

Share Document