Estimating Bayes Factors for Linear Models with Random Slopes on Continuous Predictors

Using mixed-effects models and Bayesian statistics has been advocated by statisticians in recent years. Mixed-effects models allow researchers to adequately account for the structure in the data. Bayesian statistics – in contrast to frequentist statistics – can state the evidence in favor of or against an effect of interest. For frequentist statistical methods, it is known that mixed models can lead to serious over-estimation of evidence in favor of an effect (i.e., inflated Type-I error rate) when models fail to include individual differences in the effect sizes of predictors ("random slopes") that are actually present in the data. Here, we show through simulation that the same problem exists for Bayesian mixed models. Yet, at present there is no easy-to-use application that allows for the estimation of Bayes Factors for mixed models with random slopes on continuous predictors. Here, we close this gap by introducing a new R package called BayesRS. We tested its functionality in four simulation studies. They show that BayesRS offers a reliable and valid tool to compute Bayes Factors. BayesRS also allows users to account for correlations between random effects. In a fifth simulation study we show, however, that doing so leads to slight underestimation of the evidence in favor of an actually present effect. We only recommend modeling correlations between random effects when they are of primary interest and when sample size is large enough. BayesRS is available under https://cran.r-project.org/web/packages/BayesRS/, R code for all simulations is available under https://osf.io/nse5x/?view_only=b9a7caccd26a4764a084de3b8d459388

Download Full-text

Benefits of Bayesian Model Selection and Averaging for Mixed-Effects Modeling

10.31234/osf.io/zusd2 ◽

2021 ◽

Author(s):

Daniel W. Heck ◽

Florence Bockting

Keyword(s):

Model Selection ◽

Random Effects ◽

Mixed Models ◽

Bayesian Model ◽

Fixed Effects ◽

Bayes Factor ◽

Mixed Effects ◽

Bayes Factors ◽

Mixed Effects Models ◽

Within Subjects

Bayes factors allow researchers to test the eﬀects of experimental manipulations in within-subjects designs using mixed-eﬀects models. van Doorn et al. (2021) showed that such hypothesis tests can be performed by comparing diﬀerent pairs of models which vary in the speciﬁcation of the ﬁxed- and random-eﬀect structure for the within-subjects factor. To discuss the question of which of these model comparisons is most appropriate, van Doorn et al. used a case study to compare the corresponding Bayes factors. We argue that researchers should not only focus on pairwise comparisons of two nested models but rather use the Bayes factor for performing model selection among a larger set of mixed models that represent diﬀerent auxiliary assumptions. In a standard one-factorial, repeated-measures design, the comparison should include four mixed-eﬀects models: ﬁxed-eﬀects H0, ﬁxed-eﬀects H1, random-eﬀects H0, and random-eﬀects H1. Thereby, the Bayes factor enables testing both the average eﬀect of condition and the heterogeneity of eﬀect sizes across individuals. Bayesian model averaging provides an inclusion Bayes factor which quantiﬁes the evidence for or against the presence of an eﬀect of condition while taking model-selection uncertainty about the heterogeneity of individual eﬀects into account. We present a simulation study showing that model selection among a larger set of mixed models performs well in recovering the true, data-generating model.

Download Full-text

Detecting the violation of variance homogeneity in mixed models

Statistical Methods in Medical Research ◽

10.1177/0962280214526194 ◽

2016 ◽

Vol 25 (6) ◽

pp. 2506-2520 ◽

Cited By ~ 4

Author(s):

Xicheng Fang ◽

Jialiang Li ◽

Weng Kee Wong ◽

Bo Fu

Keyword(s):

Random Effects ◽

Mixed Models ◽

Systematic Approach ◽

Covariance Structure ◽

Mixed Effects ◽

Mixed Effects Models ◽

Simulation Studies ◽

Homogeneity Assumption ◽

Variance Homogeneity

Mixed-effects models are increasingly used in many areas of applied science. Despite their popularity, there is virtually no systematic approach for examining the homogeneity of the random-effects covariance structure commonly assumed for such models. We propose two tests for evaluating the homogeneity of the covariance structure assumption across subjects: one is based on the covariance matrices computed from the fitted model and the other is based on the empirical variation computed from the estimated random effects. We used simulation studies to compare performances of the two tests for detecting violations of the homogeneity assumption in the mixed-effects models and showed that they were able to identify abnormal clusters of subjects with dissimilar random-effects covariance structures; in particular, their removal from the fitted model might change the signs and the magnitudes of important predictors in the analysis. In a case study, we applied our proposed tests to a longitudinal cohort study of rheumatoid arthritis patients and compared their abilities to ascertain whether the assumption of covariance homogeneity for subject-specific random effects holds.

Download Full-text

Bayes factors for mixed-effects models

10.31234/osf.io/m4hju ◽

2021 ◽

Author(s):

Catriona Silvey ◽

Zoltan Dienes ◽

Elizabeth Wonnacott

Keyword(s):

Significance Test ◽

Mixed Effects ◽

Bayes Factors ◽

Mixed Effects Models ◽

Effect Sizes ◽

States Of Affairs ◽

Simple Method ◽

Frequentist Statistics ◽

To Come

In psychology, we often want to know whether or not an effect exists. The traditional way of answering this question is to use frequentist statistics. However, a significance test against a null hypothesis of no effect cannot distinguish between two states of affairs: evidence of absence of an effect, and absence of evidence for or against an effect. Bayes factors can make this distinction; however, uptake of Bayes factors in psychology has so far been low for two reasons. Firstly, they require researchers to specify the range of effect sizes their theory predicts. Researchers are often unsure about how to do this, leading to the use of inappropriate default values which may give misleading results. Secondly, many implementations of Bayes factors have a substantial technical learning curve. We present a case study and simulations demonstrating a simple method for generating a range of plausible effect sizes based on the output from frequentist mixed-effects models. Bayes factors calculated using these estimates provide intuitively reasonable results across a range of real effect sizes. The approach provides a solution to the problem of how to come up with principled estimates of effect size, and produces comparable results to a state-of-the-art method without requiring researchers to learn novel statistical software.

Download Full-text

The Use of Theory of Linear Mixed-Effects Models to Detect Fraudulent Erasures at an Aggregate Level

Educational and Psychological Measurement ◽

10.1177/0013164421994893 ◽

2021 ◽

pp. 001316442199489

Author(s):

Luyao Peng ◽

Sandip Sinharay

Keyword(s):

Type I Error ◽

Real Data ◽

Mixed Effects ◽

Error Rates ◽

Mixed Effects Models ◽

Type I ◽

Aggregate Level ◽

Linear Mixed Effects Models ◽

Linear Mixed Effects ◽

Best Linear Unbiased

Wollack et al. (2015) suggested the erasure detection index (EDI) for detecting fraudulent erasures for individual examinees. Wollack and Eckerly (2017) and Sinharay (2018) extended the index of Wollack et al. (2015) to suggest three EDIs for detecting fraudulent erasures at the aggregate or group level. This article follows up on the research of Wollack and Eckerly (2017) and Sinharay (2018) and suggests a new aggregate-level EDI by incorporating the empirical best linear unbiased predictor from the literature of linear mixed-effects models (e.g., McCulloch et al., 2008). A simulation study shows that the new EDI has larger power than the indices of Wollack and Eckerly (2017) and Sinharay (2018). In addition, the new index has satisfactory Type I error rates. A real data example is also included.

Download Full-text

Design and analysis of a 2-year parallel follow-up of repeated ivermectin mass drug administrations for control of malaria: Small sample considerations for cluster-randomized trials with count data

Clinical Trials ◽

10.1177/17407745211028581 ◽

2021 ◽

pp. 174077452110285

Author(s):

Conner L Jackson ◽

Kathryn Colborn ◽

Dexiang Gao ◽

Sangeeta Rao ◽

Hannah C Slater ◽

...

Keyword(s):

Randomized Trial ◽

Cluster Randomized Trial ◽

Mixed Effects ◽

Small Sample ◽

Mixed Effects Models ◽

Type I ◽

Cluster Randomized ◽

Cluster Level ◽

Generalized Estimating

Background: Cluster-randomized trials allow for the evaluation of a community-level or group-/cluster-level intervention. For studies that require a cluster-randomized trial design to evaluate cluster-level interventions aimed at controlling vector-borne diseases, it may be difficult to assess a large number of clusters while performing the additional work needed to monitor participants, vectors, and environmental factors associated with the disease. One such example of a cluster-randomized trial with few clusters was the “efficacy and risk of harms of repeated ivermectin mass drug administrations for control of malaria” trial. Although previous work has provided recommendations for analyzing trials like repeated ivermectin mass drug administrations for control of malaria, additional evaluation of the multiple approaches for analysis is needed for study designs with count outcomes. Methods: Using a simulation study, we applied three analysis frameworks to three cluster-randomized trial designs (single-year, 2-year parallel, and 2-year crossover) in the context of a 2-year parallel follow-up of repeated ivermectin mass drug administrations for control of malaria. Mixed-effects models, generalized estimating equations, and cluster-level analyses were evaluated. Additional 2-year parallel designs with different numbers of clusters and different cluster correlations were also explored. Results: Mixed-effects models with a small sample correction and unweighted cluster-level summaries yielded both high power and control of the Type I error rate. Generalized estimating equation approaches that utilized small sample corrections controlled the Type I error rate but did not confer greater power when compared to a mixed model approach with small sample correction. The crossover design generally yielded higher power relative to the parallel equivalent. Differences in power between analysis methods became less pronounced as the number of clusters increased. The strength of within-cluster correlation impacted the relative differences in power. Conclusion: Regardless of study design, cluster-level analyses as well as individual-level analyses like mixed-effects models or generalized estimating equations with small sample size corrections can both provide reliable results in small cluster settings. For 2-year parallel follow-up of repeated ivermectin mass drug administrations for control of malaria, we recommend a mixed-effects model with a pseudo-likelihood approximation method and Kenward–Roger correction. Similarly designed studies with small sample sizes and count outcomes should consider adjustments for small sample sizes when using a mixed-effects model or generalized estimating equation for analysis. Although the 2-year parallel follow-up of repeated ivermectin mass drug administrations for control of malaria is already underway as a parallel trial, applying the simulation parameters to a crossover design yielded improved power, suggesting that crossover designs may be valuable in settings where the number of available clusters is limited. Finally, the sensitivity of the analysis approach to the strength of within-cluster correlation should be carefully considered when selecting the primary analysis for a cluster-randomized trial.

Download Full-text