scholarly journals Performance of Matching Methods as Compared With Unmatched Ordinary Least Squares Regression Under Constant Effects

2019 ◽  
Vol 188 (7) ◽  
pp. 1345-1354 ◽  
Author(s):  
Anusha M Vable ◽  
Mathew V Kiang ◽  
M Maria Glymour ◽  
Joseph Rigdon ◽  
Emmanuel F Drabo ◽  
...  
Keyword(s):  
Propensity Score ◽  
Least Squares ◽  
Propensity Score Matching ◽  
Confidence Intervals ◽  
Error Rate ◽  
Type I Error ◽  
Ordinary Least Squares ◽  
Type I ◽  
Exact Matching ◽  
Common Support

AbstractMatching methods are assumed to reduce the likelihood of a biased inference compared with ordinary least squares (OLS) regression. Using simulations, we compared inferences from propensity score matching, coarsened exact matching, and unmatched covariate-adjusted OLS regression to identify which methods, in which scenarios, produced unbiased inferences at the expected type I error rate of 5%. We simulated multiple data sets and systematically varied common support, discontinuities in the exposure and/or outcome, exposure prevalence, and analytical model misspecification. Matching inferences were often biased in comparison with OLS, particularly when common support was poor; when analysis models were correctly specified and common support was poor, the type I error rate was 1.6% for propensity score matching (statistically inefficient), 18.2% for coarsened exact matching (high), and 4.8% for OLS (expected). Our results suggest that when estimates from matching and OLS are similar (i.e., confidence intervals overlap), OLS inferences are unbiased more often than matching inferences; however, when estimates from matching and OLS are dissimilar (i.e., confidence intervals do not overlap), matching inferences are unbiased more often than OLS inferences. This empirical “rule of thumb” may help applied researchers identify situations in which OLS inferences may be unbiased as compared with matching inferences.

scholarly journals Consequences of Ignoring Clustering in Linear Regression

2020 ◽  
Author(s):  
Georgia Ntani ◽  
Hazel Inskip ◽  
Clive Osmond ◽  
David Coggon
Keyword(s):  
Linear Regression ◽  
Statistical Inference ◽  
Confidence Intervals ◽  
Type I Error ◽  
Explanatory Variable ◽  
Simulated Data ◽  
Ordinary Least Squares ◽  
Outcome Variable ◽  
Type I ◽  
Random Intercept

Abstract BackgroundClustering of observations is a common phenomenon in epidemiological and clinical research. Previous studies have highlighted the importance of using multilevel analysis to account for such clustering, but in practice, methods ignoring clustering are often used. We used simulated data to explore the circumstances in which failure to account for clustering in linear regression analysis could lead to importantly erroneous conclusions. MethodsWe simulated data following the random-intercept model specification under different scenarios of clustering of a continuous outcome and a single continuous or binary explanatory variable. We fitted random-intercept (RI) and cluster-unadjusted ordinary least squares (OLS) models and compared the derived estimates of effect, as quantified by regression coefficients, and their estimated precision. We also assessed the extent to which coverage by 95% confidence intervals and rates of Type I error were appropriate. ResultsWe found that effects estimated from OLS linear regression models that ignored clustering were on average unbiased. The precision of effect estimates from the OLS model was overestimated when both the outcome and explanatory variable were continuous. By contrast, in linear regression with a binary explanatory variable, in most circumstances, the precision of effects was somewhat underestimated by the OLS model. The magnitude of bias, both in point estimates and their precision, increased with greater clustering of the outcome variable, and was influenced also by the amount of clustering in the explanatory variable. The cluster-unadjusted model resulted in poor coverage rates by 95% confidence intervals and high rates of Type I error especially when the explanatory variable was continuous. ConclusionsIn this study we identified situations in which an OLS regression model is more likely to affect statistical inference, namely when the explanatory variable is continuous, and its intraclass correlation coefficient is higher than 0.01. Situations in which statistical inference is less likely to be affected have also been identified.

scholarly journals Multinomial Propensity Score for Ternary Exposure for Genetic Study

2020 ◽  
Author(s):  
Cathy S. J. Fann ◽  
Thai Son Dinh ◽  
Yu-Hsien Chang ◽  
Jia Jyun Sie ◽  
Ie-Bin Lian
Keyword(s):  
Propensity Score ◽  
Error Rate ◽  
Type I Error ◽  
Type I ◽  
Matched Cohort ◽  
Nucleotide Polymorphisms ◽  
Single Nucleotide ◽  
Type I Error Rate ◽  
Regression Adjustment ◽  
Selection Of

Abstract Background: Propensity score (PS) is a popular method for reducing multiple confounding effects in observational studies. It is applicable mainly for situations wherein the exposure/treatment of interest is dichotomous and the PS can be estimated through logistic regression. However, multinomial exposures with 3 or more levels are not rare, e.g., when considering genetic variants, such as single nucleotide polymorphisms (SNPs), which have 3 levels (aa/aA/AA), as an exposure. Conventional PS is inapplicable for this situation unless the 3 levels are collapsed into 2 classes first. Methods: A simulation study was conducted to compare the performance of the proposed multinomial propensity score (MPS) method under various contrast codings and approaches, including regression adjustment and matching.Results: MPS methods had more reasonable type I error rate than the non-MPS methods, of which the latter could be as high as 30~50%. Compared with MPS-direct adjusted methods, MPS-matched cohort methods have better power but larger type I error rate. Performance of contrast codings depend on the selection of MPS models. Conclusions: In general, two combinations had relatively better performance in our simulation of ternary exposure: MPS-matched cohort method with Helmert contrast and MPS-direct adjusted regression with treatment contrasts. Compared with the latter, the former had better power but larger type I error rate as a trade-off.

Correction: “Influence of Selection Bias on the Test Decision – A Simulation Study”

2014 ◽  
Vol 53 (05) ◽  
pp. 343-343
Keyword(s):  
Selection Bias ◽  
Simulation Study ◽  
Error Rate ◽  
Type I Error ◽  
Block Size ◽  
Error Rates ◽  
Type I ◽  
Type I Error Rate ◽  
Representation Error ◽  
Numeric Representation

We have to report marginal changes in the empirical type I error rates for the cut-offs 2/3 and 4/7 of Table 4, Table 5 and Table 6 of the paper “Influence of Selection Bias on the Test Decision – A Simulation Study” by M. Tamm, E. Cramer, L. N. Kennes, N. Heussen (Methods Inf Med 2012; 51: 138 –143). In a small number of cases the kind of representation of numeric values in SAS has resulted in wrong categorization due to a numeric representation error of differences. We corrected the simulation by using the round function of SAS in the calculation process with the same seeds as before. For Table 4 the value for the cut-off 2/3 changes from 0.180323 to 0.153494. For Table 5 the value for the cut-off 4/7 changes from 0.144729 to 0.139626 and the value for the cut-off 2/3 changes from 0.114885 to 0.101773. For Table 6 the value for the cut-off 4/7 changes from 0.125528 to 0.122144 and the value for the cut-off 2/3 changes from 0.099488 to 0.090828. The sentence on p. 141 “E.g. for block size 4 and q = 2/3 the type I error rate is 18% (Table 4).” has to be replaced by “E.g. for block size 4 and q = 2/3 the type I error rate is 15.3% (Table 4).”. There were only minor changes smaller than 0.03. These changes do not affect the interpretation of the results or our recommendations.

Controlling type I error rate for fast track drug development programmes

2003 ◽  
Vol 22 (5) ◽  
pp. 665-675 ◽  
Author(s):  
Weichung J. Shih ◽  
Peter Ouyang ◽  
Hui Quan ◽  
Yong Lin ◽  
Bart Michiels ◽  
...  
Keyword(s):  
Drug Development ◽  
Error Rate ◽  
Fast Track ◽  
Type I Error ◽  
Type I ◽  
Type I Error Rate

Alternative models and randomization techniques for Bayesian response-adaptive randomization with binary outcomes

2021 ◽  
pp. 174077452110101
Author(s):  
Jennifer Proper ◽  
John Connett ◽  
Thomas Murray
Keyword(s):  
Logistic Regression ◽  
Sample Size ◽  
Error Rate ◽  
Adaptive Design ◽  
Type I Error ◽  
Probability Model ◽  
Binary Outcomes ◽  
Type I ◽  
Operating Characteristics ◽  
Type I Error Rate

Background: Bayesian response-adaptive designs, which data adaptively alter the allocation ratio in favor of the better performing treatment, are often criticized for engendering a non-trivial probability of a subject imbalance in favor of the inferior treatment, inflating type I error rate, and increasing sample size requirements. The implementation of these designs using the Thompson sampling methods has generally assumed a simple beta-binomial probability model in the literature; however, the effect of these choices on the resulting design operating characteristics relative to other reasonable alternatives has not been fully examined. Motivated by the Advanced R2 Eperfusion STrategies for Refractory Cardiac Arrest trial, we posit that a logistic probability model coupled with an urn or permuted block randomization method will alleviate some of the practical limitations engendered by the conventional implementation of a two-arm Bayesian response-adaptive design with binary outcomes. In this article, we discuss up to what extent this solution works and when it does not. Methods: A computer simulation study was performed to evaluate the relative merits of a Bayesian response-adaptive design for the Advanced R2 Eperfusion STrategies for Refractory Cardiac Arrest trial using the Thompson sampling methods based on a logistic regression probability model coupled with either an urn or permuted block randomization method that limits deviations from the evolving target allocation ratio. The different implementations of the response-adaptive design were evaluated for type I error rate control across various null response rates and power, among other performance metrics. Results: The logistic regression probability model engenders smaller average sample sizes with similar power, better control over type I error rate, and more favorable treatment arm sample size distributions than the conventional beta-binomial probability model, and designs using the alternative randomization methods have a negligible chance of a sample size imbalance in the wrong direction. Conclusion: Pairing the logistic regression probability model with either of the alternative randomization methods results in a much improved response-adaptive design in regard to important operating characteristics, including type I error rate control and the risk of a sample size imbalance in favor of the inferior treatment.

scholarly journals Can the Ceftriaxone Breakpoints Be Increased Without Compromising Patient Outcomes?

2018 ◽  
Vol 5 (6) ◽  
Author(s):  
Pranita D Tamma ◽  
Virginia M Pierce ◽  
Sara E Cosgrove ◽  
Ebbing Lautenbach ◽  
Anthony Harris ◽  
...  
Keyword(s):  
Propensity Score ◽  
Propensity Score Matching ◽  
Patient Outcomes ◽  
Nearest Neighbor ◽  
Clinical Laboratory ◽  
Severity Of Illness ◽  
Bloodstream Infections ◽  
Source Control ◽  
Exact Matching ◽  
Extended Spectrum

Abstract Background In 2010, the Clinical Laboratory and Standards Institute recommended a 3-fold lowering of ceftriaxone breakpoints to 1 mcg/mL for Enterobacteriaceae. Supportive clinical data at the time were from fewer than 50 patients. We compared the clinical outcomes of adults with Enterobacteriaceae bloodstream infections treated with ceftriaxone compared with matched patients (with exact matching on ceftriaxone minimum inhibitory concentrations [MICs]) treated with extended-spectrum agents to determine if ceftriaxone breakpoints could be increased without negatively impacting patient outcomes. Methods A retrospective cohort study was conducted at 3 large academic medical centers and included patients with Enterobacteriaceae bacteremia with ceftriaxone MICs of 2 mcg/mL treated with ceftriaxone or extended-spectrum β-lactams (ie, cefepime, piperacillin/tazobactam, meropenem, or imipenem/cilastatin) between 2008 and 2014; 1:2 nearest neighbor propensity score matching was performed to estimate the odds of recurrent bacteremia and mortality within 30 days. Results Propensity score matching yielded 108 patients in the ceftriaxone group and 216 patients in the extended-spectrum β-lactam group, with both groups well-balanced on demographics, preexisting medical conditions, severity of illness, source of bacteremia, and source control interventions. No difference in recurrent bacteremia (odds ratio [OR], 1.16; 95% confidence interval [CI], 0.49–2.73) or mortality (OR, 1.27; 95% CI, 0.56–2.91) between the treatment groups was observed for patients with isolates with ceftriaxone MICs of 2 mcg/mL. Only 6 isolates (1.6%) with ceftriaxone MICs of 2 mcg/mL were extended-spectrum β-lactamase (ESBL)–producing. Conclusions Our findings suggest that patient outcomes are similar when receiving ceftriaxone vs extended-spectrum agents for the treatment of Enterobacteriaceae bloodstream infections with ceftriaxone MICs of 2 mcg/mL. This warrants consideration of adjusting the ceftriaxone susceptibility breakpoint from 1 to 2 mcg/mL, as a relatively small increase in the antibiotic breakpoint could have the potential to limit the use of large numbers of extended-spectrum antibiotic agents.

Does Family Decision-Making Power Improve Women’s Happiness?

2021 ◽  
pp. 0192513X2110300
Author(s):  
Zhongwu Li
Keyword(s):  
Decision Making ◽  
Propensity Score ◽  
Least Squares ◽  
Logit Model ◽  
Self Esteem ◽  
Ordinary Least Squares ◽  
Matching Method ◽  
Family Decision ◽  
Family Decision Making ◽  
Heterogeneity Analysis

It is almost a consensus that the stronger family decision-making power a woman has, the happier she will be. While using the China Family Panel Studies, this study reveals a long-overlooked fact that women’s control over more family decision-making power does not necessarily improve their happiness. The results of the ordinary least squares and ordinal logit model confirm this finding, and the propensity score matching method corroborates the conclusion. Heterogeneity analysis shows that among those women with less education and lower social status, the negative happiness effect of women’s family decision-making power is particularly significant. Women’s traditional attitudes and self-esteem are two important factors which hinder women’s family decision-making power from enhancing their happiness.

Anova Tests for Homogeneity of Variance: Nonnormality and Unequal Samples

1977 ◽  
Vol 2 (3) ◽  
pp. 187-206 ◽  
Author(s):  
Charles G. Martin ◽  
Paul A. Games
Keyword(s):  
Error Rate ◽  
Type I Error ◽  
Type I ◽  
Empirical Comparison ◽  
Jackknife Test ◽  
Type I Error Rate ◽  
Power And Control ◽  
Homogeneity Of Variance ◽  
Test Use ◽  
And Control

This paper presents an exposition and an empirical comparison of two potentially useful tests for homogeneity of variance. Control of Type I error rate, P(EI), and power are investigated for three forms of the Box test and for two forms of the jackknife test with equal and unequal n's under conditions of normality and nonnormality. The Box test is shown to be robust to violations of the assumption of normality. The jackknife test is shown not to be robust. When n's are unequal, the problem of heterogeneous within-cell variances of the transformed values and unequal n's affects the jackknife and Box tests. Previously reported suggestions for selecting subsample sizes for the Box test are shown to be inappropriate, producing an inflated P(EI). Two procedures which alleviate this problem are presented for the Box test. Use of the jack-knife test with a reduced alpha is shown to provide power and control of P(EI) at approximately the same level as the Box test. Recommendations for the use of these techniques and computational examples of each are provided.

scholarly journals Non-inferiority test for a continuous variable with a flexible margin in an active controlled trial : An application to the "Stratall ANRS 12110 / ESTHER" trial

2020 ◽  
Author(s):  
Arsene Sandie ◽  
Nicholas Molinari ◽  
Anthony Wanjoya ◽  
Charles Kouanfack ◽  
Christian Laurent ◽  
...  
Keyword(s):  
Confidence Interval ◽  
Active Control ◽  
Error Rate ◽  
Type I Error ◽  
Clinical Monitoring ◽  
Small Scale ◽  
Type I ◽  
Test Procedures ◽  
Power Estimate ◽  
Control Intervention

Abstract Background: The non-inferiority trials are becoming increasingly popular in public health and clinical research. The choice of the non-inferiority margin is the cornerstone of the non-inferiority trial. When the effect of active control intervention is unknown, it can be interesting to choose the non-inferiority margin as a function of the active control intervention effect. In this case, the uncertainty surrounding the non-inferiority margin should be accounted for in statistical tests. In this work, we explored how to perform the non-inferiority test with a flexible margin for continuous endpoint.Methods: It was proposed in this study two procedures for the non-inferiority test with a flexible margin for the continuous endpoint. The proposed test procedures are based on test statistic and confidence interval approach. Simulations have been used to assess the performances and properties of the proposed test procedures. An application was done on clinical real data, which the purpose was to assess the efficacy of clinical monitoring alone versus laboratory and clinical monitoring in HIV-infected adult patients.Results: Basically, the two proposed test procedures have good properties. In the test based on a statistic, the actual type 1 error rate estimate is approximatively equal to the nominal value. It has been found that the confidence interval level determines approximately the level of significance. The 80%, 90%, and 95%one-sided confidence interval levels led approximately to a type I error of 10%, 5% and 2.5% respectively. The power estimate was almost 100% for two proposed tests, except for the small scale values of the reference treatment where the power was relatively low when the sample sizes were small.Conclusions: Based on type I error rate and power estimates, the proposed non-inferiority hypothesis test procedures have good performance and are applicable in practice.Trial registration: The trial data used in this study was from the ”Stratall ANRS 12110 / ESTHER”, registered with ClinicalTrials.gov, number NCT00301561. Date : March 13, 2006, url : https://clinicaltrials.gov/ct2/show/NCT00301561.

Sign in / Sign up

Export Citation Format

Share Document