The Importance of Random Slopes in Mixed Models for Bayesian Hypothesis Testing

Mixed models are gaining popularity in psychology. For frequentist mixed models, Barr, Levy, Scheepers, and Tily (2013) showed that excluding random slopes – differences between individuals in the direction and size of an effect – from a model when they are in the data can lead to a substantial increase in false-positive conclusions in null-hypothesis tests. Here I demonstrate through five simulations that the same is true for Bayesian hypothesis testing with mixed models, often yielding Bayes factors reflecting very strong evidence for a mean effect on the population level even if there was no such effect. Including random slopes in the model largely eliminates the risk of strong false positives, but reduces the chance of obtaining strong evidence for true effects. I recommend starting analysis with testing the support for random slopes in the data, and removing them from the models only if there is clear evidence against them.

Preprint and additional materials for: Positivity in peer perceptions over time: Personality explains variation at zero-acquaintance, popularity explains differential change.

People have characteristic ways of perceiving others’ personalities. When judging others on several traits, some perceivers tend to form globally positive and others tend to form globally negative impressions. These differences, often termed perceiver effects, have mostly been conceptualized as a static construct that taps perceivers’ personal stereotypes about the average other. Here, we assessed perceiver effects repeatedly in small groups of strangers who got to know each other over the course of 2 to 3 weeks and examined the degree to which positivity differences were stable vs. developed systematically over time. Using second order latent growth curve modelling, we tested whether initial positivity (i.e., random intercepts) could be explained by several personality variables and whether change (i.e., random slopes) could be explained by these personality variables and by perceivers’ social experiences within the group. Across three studies (ns = 439, 257, and 311), personality variables characterized by specific beliefs about others, such as agreeableness and narcissistic rivalry, were found to explain initial positivity but personality was not reliably linked to changes in positivity over time. Instead, feeling liked and, to a lesser extent, being liked by one’s peers, partially explained changes in positivity. The results suggest that perceiver effects are best conceptualized as reflecting personal generalized stereotypes at an initial encounter but group-specific stereotypes that are fueled by social experiences as groups get acquainted. More generally, these findings suggest that perceiver effects might be a key variable to understanding reciprocal dynamics of small groups and interpersonal functioning.

The role of incubator classification on performance of incubators in Kenya.

The study sought to assess the importance of classifying incubators based on the programs offered for optimum performance. Client selection criteria were assessed through three constructs namely: models that fit program goals, uniqueness of ideas, and standard selection tool. A mixed cross-sectional and causal design was adopted and a census was carried out targeting all the 51 incubators. Primary data was collected with an incubator program as a grouping/ cluster variable yielding a multilevel data structure with incubator centres nested in programs. Linear mixed effect models were fitted using Stata to assess the study objective taking into account the fixed effects for the incubator centre level (level-1) and random effects for the program level (level-2). The uniqueness of ideas was found to have a significant fixed effect on performance at level one while at level two, the study found significant random intercepts of incubator centre performance across the programs. Models that match program goals and standard selection tools were also found to have significant random slopes as level two random covariates in the model. Based on the findings of significant random slopes, the study concluded that incubator classification is key for client selection criteria and enhances incubator performance.

COVID-19 vaccine uptake in United States counties: geospatial vaccination patterns and trajectories towards herd immunity

Following the COVID-19 pandemic, safe and effective vaccines were developed and authorized for use in the general population. Studying factors that encourage community acceptance of these vaccines is needed to prevent proliferation of SARS-CoV-2 variants, to safely relax local restrictions, and to return to pre-pandemic living conditions. To our knowledge, United States (US) county-level disparities in vaccination are yet to be investigated. Our data span February - May 2021 across 3138 US counties. We consider percentage of residents with at least one dose of an authorized COVID vaccine as the outcome. Spatio-temporal models were used to determine associations of vaccination rates with time-fixed and time-varying covariates. Spatial variability was modelled via Conditional Auto-regressive models; county trajectories over time were specified using random slopes. Greater vaccination rates occur in counties with older residents, high educational attainment, and high proportion of minority residents. Vaccination rates change with COVID risk metrics, suggesting continued slowing of vaccine uptake due to decreasing incidence and infection rates. County effects reveal strong regional patterns in average vaccination rates and trajectories. Although local herd immunity can be expected in August 2021 for counties with typical uptake rates, these counties are clustered in relatively few areas of the country.

Who Is and Is Not "Average'"? Random Effects Selection with Spike-and-Slab Priors

Mixed effects models are often employed to study individual differences in psychological science. Such analyses commonly entail testing whether between-subject variability exists, but this is typically the extent of such analyses. We argue that researchers have much to gain by explicitly focusing on the individual in individual differences research. To this end, we propose the spike-and-slab prior distribution for random effect selection in (generalized) mixed-effects models as a means to gain a more nuanced perspective of individual differences. The prior for each random effect, or deviation away from the fixed effect, is a two-component mixture consisting of a point-mass 'spike' centered at zero and a diffuse 'slab' capturing non-zero values. Effectively, such an approach allows researchers to answer questions about each particular individual; specifically, "who is average?'" in the sense of deviating from an average effect, such as the population-averaged or common slope. We begin with an illustrative example, where the spike-and-slab formulation is used to select random intercepts in logistic regression. This demonstrates the utility of the proposed methodology in a simple setting while also highlighting its flexibility in fitting different kinds of models. We then extend the approach to random slopes that capture experimental effects. In two cognitive tasks, we show that despite there being little variability in the slopes, there were many individual differences in performance. Most notably, over 25% of the sample differed from the common slope in their experimental effect. We conclude with future directions for the presented methodology.

The development of a standard procedure for the optimal reliability-feasibility trade-off in Multilevel Linear Models analyses in Psychology and Neuroscience.

The use of Multilevel Linear Models (MLMs) is increasing in Psychology and Neuroscience research. A key aspect of MLMs is choosing a random effects structure according to the experimental needs. To date, opposite suggestions are present in the literature, spanning from keeping all random effects, which produces several singularity and convergence issues and often requires high computational resources, to removing random effects until the best fit is found, with the risk of inflating first-type error. However, defining the random structure to fit a non-singular and convergent model is not straightforward. Moreover, the lack of a standard approach may lead the researcher to make decisions that potentially inflate first-type errors and generate distortions in the estimates. To date, how to deal with singular and non-converging models is an ongoing debate.We introduce a new way to control for first-type error inflation during model reduction, namely transforming random slopes in complex random intercepts (CRIs). These are multiple random intercepts that represent the complexity of the factors within a given grouping factor. We demonstrate that CRIs can produce reliable results, require less computational and timing resources, and we provide a straightforward procedure to use CRIs in MLMs reduction. Importantly we outline a few criteria and clear recommendations on how and when scholars should reduce singular and non-converging models.We validated this approach by extensive simulations, a real-case application, and a comprehensive procedure that defines new solutions to avoid overinflated results and potentially standardise the use of MLMs in Psychology and Neuroscience.

Power formulas for mixed effects models with random slope and intercept comparing rate of change across groups

We have previously derived power calculation formulas for cohort studies and clinical trials using the longitudinal mixed effects model with random slopes and intercepts to compare rate of change across groups [Ard & Edland, Power calculations for clinical trials in Alzheimer’s disease. J Alzheim Dis 2011;21:369–77]. We here generalize these power formulas to accommodate 1) missing data due to study subject attrition common to longitudinal studies, 2) unequal sample size across groups, and 3) unequal variance parameters across groups. We demonstrate how these formulas can be used to power a future study even when the design of available pilot study data (i.e., number and interval between longitudinal observations) does not match the design of the planned future study. We demonstrate how differences in variance parameters across groups, typically overlooked in power calculations, can have a dramatic effect on statistical power. This is especially relevant to clinical trials, where changes over time in the treatment arm reflect background variability in progression observed in the placebo control arm plus variability in response to treatment, meaning that power calculations based only on the placebo arm covariance structure may be anticonservative. These more general power formulas are a useful resource for understanding the relative influence of these multiple factors on the efficiency of cohort studies and clinical trials, and for designing future trials under the random slopes and intercepts model.

Usefulness of Linear Mixed-Effects Models to Assess the Relationship between Objective and Subjective Internal Load in Team Sports

Internal load can be objectively measured by heart rate-based models, such as Edwards’ summated heart rate zones, or subjectively by session rating of perceived exertion. The relationship between internal loads assessed via heart rate-based models and session rating of perceived exertion is usually studied through simple correlations, although the Linear Mixed Model could represent a more appropriate statistical procedure to deal with intrasubject variability. This study aimed to compare conventional correlations and the Linear Mixed Model to assess the relationships between objective and subjective measures of internal load in team sports. Thirteen male youth beach handball players (15.9 ± 0.3 years) were monitored (14 training sessions; 7 official matches). Correlation coefficients were used to correlate the objective and subjective internal load. The Linear Mixed Model was used to model the relationship between objective and subjective measures of internal load data by considering each player individual response as random effect. Random intercepts were used and then random slopes were added. The likelihood-ratio test was used to compare statistical models. The correlation coefficient for the overall relationship between the objective and subjective internal data was very large (r = 0.74; ρ = 0.78). The Linear Mixed Model using both random slopes and random intercepts better explained (p < 0.001) the relationship between internal load measures. Researchers are encouraged to apply the Linear Mixed Models rather than correlation to analyze internal load relationships in team sports since it allows for the consideration of the individuality of players.

Specifying the random effect structure in linear mixed effect models for analyzing psycholinguistic data

Linear Mixed Effect Models (LMEM) have become a popular method for analyzing nested experimental data, which are often encountered in psycholinguistics and other fields. This approach allows experimental results to be generalized to the greater population of both subjects and experimental stimuli. In an influential paper Bar and his colleagues (2013; https://doi.org/10.1016/j.jml.2012.11.001) recommend specifying the maximal random effect structure allowed by the experimental design, which includes random intercepts and random slopes for all within-subjects and within-items experimental factors, as well as correlations between the random effects components. The goal of this paper is to formally investigate whether their recommendations can be generalized to wider variety of experimental conditions. The simulation results revealed that complex models (i.e., with more parameters) lead to a dramatic increase in the non-convergence rate. Furthermore, AIC and BIC were found to select the true model in the majority of cases, although selection accuracy varied by LMEM random effect structure.

Discrepant Inferences When Modeling Associations Between Time-Varying Exposures and Linear Growth Trajectories in Infancy Using Length-For-Age Z Scores Versus Raw Length

Objectives To compare inferences from longitudinal models of the relationships between biomarkers of interest and linear growth outcomes in infancy using length-for-age z-scores (LAZ) based on age- and sex-specific growth standards, or raw length. Methods This was a secondary analysis of data from a study of the association between bone-related biomarkers and infant linear growth trajectories up to 1 year of life in a subset of infants (n = 820) enrolled in the Maternal Vitamin D for Infant Growth trial. The linear growth outcome (LAZ or raw length) was modelled as a function of the interaction between each biomarker and age using linear mixed effect models with restricted cubic splines. Models were specified to obtain the best fit and reconcile discrepancies in results from LAZ and length models. Inferences from marginal effects at birth, 3 months, 6 months, and 12 months were compared, for a total 4 effect estimates from each of 10 pairs of LAZ and length models, resulting in 40 pairs of estimates. The following biomarkers were included: fibroblast growth factor 21 (FGF21), fibroblast growth factor 23 (FGF23), N-terminal propeptide of C-type natriuretic peptide (NT-proCNP), osteocalcin, osteoprotegerin, receptor activator of nuclear activator kappa-b ligand (RANKL), 25-hydroxyvitamin D (25OHD), C-reactive protein (CRP), Interleukin 6 (IL6), and insulin-like growth factor-1 (IGF1). Biomarkers were time-varying, measured in cord blood and at 3 and 6 months of age. Results The best fitting model for LAZ had 3 knots with random slopes, and the best fitting model for raw length had 5 knots, random slopes, and an exponential residual covariance structure. Comparisons of the pairs of marginal estimates from the LAZ vs length models resulted in discrepant inferences for 25% of effect estimates (10/40). Results were consistently concordant only for FGF23, 25OHD, CRP, and IL6. Conclusions Length and LAZ represent the same biological construct but their use in longitudinal modelling may lead to different inferences about associations between time-varying exposures and infant growth, possibly due to residual confounding by age. These findings raise concerns about the reliability of studies of determinants or markers of infant linear growth based on longitudinal modelling of growth trajectories. Funding Sources Bill and Melinda Gates Foundation.