scholarly journals Design and Analysis of Experiments in Networks: Reducing Bias from Interference

2016 ◽  
Vol 5 (1) ◽  
Author(s):  
Dean Eckles ◽  
Brian Karrer ◽  
Johan Ugander
Keyword(s):  
Sufficient Conditions ◽  
Bias Reduction ◽  
Experimental Unit ◽  
Randomized Experiments ◽  
Cluster Randomization ◽  
Graph Cluster ◽  
Unbiased Estimates ◽  
Bias Variance ◽  
Substantial Bias ◽  
Biased Estimates

AbstractEstimating the effects of interventions in networks is complicated due to interference, such that the outcomes for one experimental unit may depend on the treatment assignments of other units. Familiar statistical formalism, experimental designs, and analysis methods assume the absence of this interference, and result in biased estimates of causal effects when it exists. While some assumptions can lead to unbiased estimates, these assumptions are generally unrealistic in the context of a network and often amount to assuming away the interference. In this work, we evaluate methods for designing and analyzing randomized experiments under minimal, realistic assumptions compatible with broad interference, where the aim is to reduce bias and possibly overall error in estimates of average effects of a global treatment. In design, we consider the ability to perform random assignment to treatments that is correlated in the network, such as through graph cluster randomization. In analysis, we consider incorporating information about the treatment assignment of network neighbors. We prove sufficient conditions for bias reduction through both design and analysis in the presence of potentially global interference; these conditions also give lower bounds on treatment effects. Through simulations of the entire process of experimentation in networks, we measure the performance of these methods under varied network structure and varied social behaviors, finding substantial bias reductions and, despite a bias–variance tradeoff, error reductions. These improvements are largest for networks with more clustering and data generating processes with both stronger direct effects of the treatment and stronger interactions between units.

Predicting Network Events to Assess Goodness of Fit of Relational Event Models

2019 ◽  
Vol 27 (4) ◽  
pp. 556-571 ◽  
Author(s):  
Laurence Brandenberger
Keyword(s):  
Goodness Of Fit ◽  
Temporal Dynamics ◽  
Simple Procedure ◽  
Model Fit ◽  
Network Characteristics ◽  
Event Sequences ◽  
Simulated Event ◽  
Unbiased Estimates ◽  
Event Models ◽  
Biased Estimates

Relational event models are becoming increasingly popular in modeling temporal dynamics of social networks. Due to their nature of combining survival analysis with network model terms, standard methods of assessing model fit are not suitable to determine if the models are specified sufficiently to prevent biased estimates. This paper tackles this problem by presenting a simple procedure for model-based simulations of relational events. Predictions are made based on survival probabilities and can be used to simulate new event sequences. Comparing these simulated event sequences to the original event sequence allows for in depth model comparisons (including parameter as well as model specifications) and testing of whether the model can replicate network characteristics sufficiently to allow for unbiased estimates.

Doubly Robust Estimation with the R Package drgee

2015 ◽  
Vol 4 (1) ◽  
Author(s):  
Johan Zetterqvist ◽  
Arvid Sjölander
Keyword(s):  
Robust Estimation ◽  
Linear Models ◽  
Model Misspecification ◽  
R Package ◽  
Epidemiologic Research ◽  
Important Concern ◽  
Doubly Robust Estimation ◽  
Unbiased Estimates ◽  
Doubly Robust ◽  
Biased Estimates

AbstractA common goal of epidemiologic research is to study the association between a certain exposure and a certain outcome, while controlling for important covariates. This is often done by fitting a restricted mean model for the outcome, as in generalized linear models (GLMs) and in generalized estimating equations (GEEs). If the covariates are high-dimensional, then it may be difficult to well specify the model. This is an important concern, since model misspecification may lead to biased estimates. Doubly robust estimation is an estimation technique that offers some protection against model misspecification. It utilizes two models, one for the outcome and one for the exposure, and produces unbiased estimates of the exposure-outcome association if either model is correct, not necessarily both. Despite its obvious appeal, doubly robust estimation is not used on a regular basis in applied epidemiologic research. One reason for this could be the lack of up-to-date software. In this paper we describe a new

Using Bayesian Correspondence Criteria to Compare Results From a Randomized Experiment and a Quasi-Experiment Allowing Self-Selection

2018 ◽  
Vol 42 (2) ◽  
pp. 248-280 ◽  
Author(s):  
David M. Rindskopf ◽  
William R. Shadish ◽  
M. H. Clark
Keyword(s):  
Comparison Group ◽  
Statistical Significance ◽  
Randomized Experiment ◽  
Randomized Experiments ◽  
Bayesian Approaches ◽  
Self Selection ◽  
Frequentist Statistics ◽  
Unbiased Estimates ◽  
Quantities Of Interest ◽  
Quasi Experiment

Background: Randomized experiments yield unbiased estimates of treatment effect, but such experiments are not always feasible. So researchers have searched for conditions under which randomized and nonrandomized experiments can yield the same answer. This search requires well-justified and informative correspondence criteria, that is, criteria by which we can judge if the results from an appropriately adjusted nonrandomized experiment well-approximate results from randomized experiments. Past criteria have relied exclusively on frequentist statistics, using criteria such as whether results agree in sign or statistical significance or whether results differ significantly from each other. Objectives: In this article, we show how Bayesian correspondence criteria offer more varied, nuanced, and informative answers than those from frequentist approaches. Research design: We describe the conceptual bases of Bayesian correspondence criteria and then illustrate many possibilities using an example that compares results from a randomized experiment to results from a parallel nonequivalent comparison group experiment in which participants could choose their condition. Results: Results suggest that, in this case, the quasi-experiment reasonably approximated the randomized experiment. Conclusions: We conclude with a discussion of the advantages (computation of relevant quantities, interpretation, and estimation of quantities of interest for policy), disadvantages, and limitations of Bayesian correspondence criteria. We believe that in most circumstances, the advantages of Bayesian approaches far outweigh the disadvantages.

scholarly journals Evaluation of statistical methods used in the analysis of interrupted time series studies: a simulation study

2020 ◽  
Author(s):  
Simon L Turner ◽  
Andrew B Forbes ◽  
Amalia Karahalios ◽  
Monica Taljaard ◽  
Joanne E McKenzie
Keyword(s):  
Time Series ◽  
Statistical Methods ◽  
Simulated Data ◽  
Population Level ◽  
Interrupted Time Series ◽  
Ordinary Least Squares ◽  
In Series ◽  
Time Series Studies ◽  
Unbiased Estimates ◽  
Biased Estimates

AbstractInterrupted time series (ITS) studies are frequently used to evaluate the effects of population-level interventions or exposures. To our knowledge, no studies have compared the performance of different statistical methods for this design. We simulated data to compare the performance of a set of statistical methods under a range of scenarios which included different level and slope changes, varying lengths of series and magnitudes of autocorrelation. We also examined the performance of the Durbin-Watson (DW) test for detecting autocorrelation. All methods yielded unbiased estimates of the level and slope changes over all scenarios. The magnitude of autocorrelation was underestimated by all methods, however, restricted maximum likelihood (REML) yielded the least biased estimates. Underestimation of autocorrelation led to standard errors that were too small and coverage less than the nominal 95%. All methods performed better with longer time series, except for ordinary least squares (OLS) in the presence of autocorrelation and Newey-West for high values of autocorrelation. The DW test for the presence of autocorrelation performed poorly except for long series and large autocorrelation. From the methods evaluated, OLS was the preferred method in series with fewer than 12 points, while in longer series, REML was preferred. The DW test should not be relied upon to detect autocorrelation, except when the series is long. Care is needed when interpreting results from all methods, given confidence intervals will generally be too narrow. Further research is required to develop better performing methods for ITS, especially for short series.

scholarly journals Generalized Inflated Discrete Models: A Strategy to Work with Multimodal Discrete Distributions

2017 ◽  
Author(s):  
Tianji Cai ◽  
Yiwei Xia ◽  
Yisu Zhou
Keyword(s):  
Large Scale ◽  
Behavioral Outcomes ◽  
Discrete Distributions ◽  
Model Fit ◽  
Discrete Models ◽  
Multimodal Data ◽  
Delinquent Behaviors ◽  
Monte Carlo Experiments ◽  
Substantial Bias ◽  
Biased Estimates

Analysts of discrete data often face the challenge of managing the tendency of inflation on certain values. When treated improperly, such phenomenon may lead to biased estimates and incorrect inferences. This study extends the existing literature on single value inflated models, and develops a general framework to handle variables with more than one inflated values. To assess the performance of the proposed maximum likelihood estimator, we conducted Monte Carlo experiments under several scenarios for different levels of inflated probabilities under Multinomial, Ordinal, Poisson, and Zero-Truncated Poisson outcomes with covariates. We found that ignoring the inflations leads to substantial bias and poor inference if the inflations—not only for the intercept(s) of the inflated categories, but other coefficients as well. Specifically, higher values of inflated probabilities are associated with larger biases. By contrast, the Generalized Inflated Discrete models (GIDM) perform well with unbiased estimates and satisfactory coverages even when the number of parameters that need to be estimated is quite large. We showed that model fit criteria such as AIC could be used in selecting appropriate specification of inflated models. Lastly, GIDM was implemented to a large-scale health survey data to compare with conventional modeling approach such as various Poisson, and Ordered Logit models. We showed that GIDM fits the data better in general. The current work provides a practical approach to analyze multimodal data existing in many fields, such as heaping in self-reported behavioral outcomes, inflated categories of indifference and neutral in attitude survey, large amount of zero and low occurance of delinquent behaviors, etc.

scholarly journals Revised Calculation of Kalinowski’s Ancestral and New Inbreeding Coefficients

Diversity ◽  
2020 ◽  
Vol 12 (4) ◽  
pp. 155 ◽  
Author(s):  
Harmen P. Doekes ◽  
Ino Curik ◽  
István Nagy ◽  
János Farkas ◽  
György Kövér ◽  
...  
Keyword(s):  
Software Package ◽  
Stochastic Approach ◽  
Inbreeding Coefficient ◽  
Data Sets ◽  
Inbreeding Coefficients ◽  
Holstein Friesian ◽  
Unbiased Estimates ◽  
Biased Estimates ◽  
Friesian Cattle

To test for the presence of purging in populations, the classical pedigree-based inbreeding coefficient (F) can be decomposed into Kalinowski’s ancestral (FANC) and new (FNEW) inbreeding coefficients. The FANC and FNEW can be calculated by a stochastic approach known as gene dropping. However, the only publicly available algorithm for the calculation of FANC and FNEW, implemented in GRain v 2.1 (and also incorporated in the PEDIG software package), has produced biased estimates. The FANC was systematically underestimated and consequently, FNEW was overestimated. To illustrate this bias, we calculated FANC and FNEW by hand for simple example pedigrees. We revised the GRain program so that it now provides unbiased estimates. Correlations between the biased and unbiased estimates of FANC and FNEW, obtained for example data sets of Hungarian Pannon White rabbits (22,781 individuals) and Dutch Holstein Friesian cattle (37,061 individuals), were high, i.e., >0.96. Although the magnitude of bias appeared to be small, results from studies based on biased estimates should be interpreted with caution. The revised GRain program (v 2.2) is now available online and can be used to calculate unbiased estimates of FANC and FNEW.

network scale-up method

2004 ◽  
Vol 1 (2) ◽  
pp. 395-405
Author(s):  
Silvia Snidero ◽  
Roberto Corradetti ◽  
Dario Gregori
Keyword(s):  
Social Network ◽  
Scale Up ◽  
Monte Carlo Simulation Study ◽  
The Social ◽  
Network Scale ◽  
Ideal Number ◽  
Unbiased Estimates ◽  
The Mean ◽  
Ml Estimator ◽  
Biased Estimates

The network scale-up method is a social network estimator for the size of hidden or hard-to-count subpopulations. These estimators are based on a simple model which have however strong assumptions. The basic idea is that the proportion of the mean number of people known by respondent in a subpopulation E of T of size e is the same of the proportion that the e subpopulation E forms in general population T of size t: mc = t , where c is the number of persons known by each respondent and m is the mean number of persons known by each respondent in the subpopulation E. The persons known by every subject is called the "social network", and its size is c, estimated by several estimators proposed in the recent literature. In this paper we present a Monte Carlo simulation study aimed at understanding the behavior of the scale-up method type estimators under several conditions. The first goal was to understand what would be the ideal number of subpopulations of known size to be used in planning the research. The second goal was to analyze what happens when we use overlapped subpopulations. Our results showed that with the scale-up estimator we always obtain biased estimates for any number of subpopulations employed in estimates. With the Killworth's ML estimator, the improvement of scale-up method, we have substantially unbiased estimates under any condition. Also in case of overlapping, and increasing the degree of it among subpopulations, bias raises with scale-up method, instead it remains close to zero with ML estimator.

What if Subjects Can't be Randomly Assigned?

1977 ◽  
Vol 19 (3) ◽  
pp. 227-234 ◽  
Author(s):  
Russell S. Vaught
Keyword(s):  
Monte Carlo ◽  
Experimental Design ◽  
Treatment Effects ◽  
Experimental Designs ◽  
Random Assignment ◽  
Monte Carlo Studies ◽  
Quasi Experimental ◽  
Unbiased Estimates ◽  
Evaluative Research ◽  
Biased Estimates

Random assignment to treatment is not always possible in evaluative research. The semi-experimental design discussed here has aspects of both full experimental and quasi-experimental designs. Monte Carlo studies are used to explore and exemplify the strengths and weaknesses of the design. The analysis suggested is found to give unbiased estimates of treatment effects and error mean squares but biased estimates of assignment effects. Some further aspects of the design and its use are discussed.

scholarly journals Revisiting Bias in Odds Ratios

2021 ◽  
Author(s):  
Ivo M Foppa ◽  
Fredrick S Dahlgren
Keyword(s):  
Bias Correction ◽  
Odds Ratio ◽  
Effect Modification ◽  
Sample Sizes ◽  
Unbiased Estimates ◽  
Expected Values ◽  
Substantial Bias ◽  
False Premise ◽  
Method Bias ◽  
Almost Unbiased

AbstractRatio measures of effect, such as the odds ratio (OR), are consistent, but the presumption of their unbiasedness is founded on a false premise: The equality of the expected value of a ratio and the ratio of expected values. We show that the invalidity of this assumptions is an important source of empirical bias in ratio measures of effect, which is due to properties of the expectation of ratios of count random variables. We investigate ORs (unconfounded, no effect modification), proposing a correction that leads to “almost unbiased” estimates. We also explore ORs with covariates. We find substantial bias in OR estimates for smaller sample sizes, which can be corrected by the proposed method. Bias correction is more elusive for adjusted analyses. The notion of unbiasedness of OR for the effect of interest for smaller sample sizes is challenged.

Sign in / Sign up

Export Citation Format

Share Document