A brief introduction to mixed effects modelling and multi-model inference in ecology

The use of linear mixed effects models (LMMs) is increasingly common in the analysis of biological data. Whilst LMMs offer a flexible approach to modelling a broad range of data types, ecological data are often complex and require complex model structures, and the fitting and interpretation of such models is not always straightforward. The ability to achieve robust biological inference requires that practitioners know how and when to apply these tools. Here, we provide a general overview of current methods for the application of LMMs to biological data, and highlight the typical pitfalls that can be encountered in the statistical modelling process. We tackle several issues relating to the use of information theory and multi-model inference in ecology, and demonstrate the tendency for data dredging to lead to greatly inflated Type I error rate (false positives) and impaired inference. We offer practical solutions and direct the reader to key references that provide further technical detail for those seeking a deeper understanding. This overview should serve as a widely accessible code of best practice for applying LMMs to complex biological problems and model structures, and in doing so improve the robustness of conclusions drawn from studies investigating ecological and evolutionary questions.

Download Full-text

A brief introduction to mixed effects modelling and multi-model inference in ecology

10.7287/peerj.preprints.3113 ◽

2018 ◽

Cited By ~ 1

Author(s):

Xavier A Harrison ◽

Lynda Donaldson ◽

Maria Eugenia Correa-Cano ◽

Julian Evans ◽

David N Fisher ◽

...

Keyword(s):

Best Practice ◽

Statistical Modelling ◽

Mixed Effects ◽

Complex Model ◽

Biological Data ◽

Type I ◽

Data Types ◽

Ecological Data ◽

Model Inference ◽

Model Structures

The use of linear mixed effects models (LMMs) is increasingly common in the analysis of biological data. Whilst LMMs offer a flexible approach to modelling a broad range of data types, ecological data are often complex and require complex model structures, and the fitting and interpretation of such models is not always straightforward. The ability to achieve robust biological inference requires that practitioners know how and when to apply these tools. Here, we provide a general overview of current methods for the application of LMMs to biological data, and highlight the typical pitfalls that can be encountered in the statistical modelling process. We tackle several issues relating to the use of information theory and multi-model inference in ecology, and demonstrate the tendency for data dredging to lead to greatly inflated Type I error rate (false positives) and impaired inference. We offer practical solutions and direct the reader to key references that provide further technical detail for those seeking a deeper understanding. This overview should serve as a widely accessible code of best practice for applying LMMs to complex biological problems and model structures, and in doing so improve the robustness of conclusions drawn from studies investigating ecological and evolutionary questions.

Download Full-text

A brief introduction to mixed effects modelling and multi-model inference in ecology

PeerJ ◽

10.7717/peerj.4794 ◽

2018 ◽

Vol 6 ◽

pp. e4794 ◽

Cited By ~ 343

Author(s):

Xavier A. Harrison ◽

Lynda Donaldson ◽

Maria Eugenia Correa-Cano ◽

Julian Evans ◽

David N. Fisher ◽

...

Keyword(s):

Best Practice ◽

Statistical Modelling ◽

Mixed Effects ◽

Complex Model ◽

Biological Data ◽

Data Types ◽

Ecological Data ◽

Model Inference ◽

Model Structures ◽

Modelling Process

The use of linear mixed effects models (LMMs) is increasingly common in the analysis of biological data. Whilst LMMs offer a flexible approach to modelling a broad range of data types, ecological data are often complex and require complex model structures, and the fitting and interpretation of such models is not always straightforward. The ability to achieve robust biological inference requires that practitioners know how and when to apply these tools. Here, we provide a general overview of current methods for the application of LMMs to biological data, and highlight the typical pitfalls that can be encountered in the statistical modelling process. We tackle several issues regarding methods of model selection, with particular reference to the use of information theory and multi-model inference in ecology. We offer practical solutions and direct the reader to key references that provide further technical detail for those seeking a deeper understanding. This overview should serve as a widely accessible code of best practice for applying LMMs to complex biological problems and model structures, and in doing so improve the robustness of conclusions drawn from studies investigating ecological and evolutionary questions.

Download Full-text

Supplemental Material for Type I Error Inflation in the Traditional By-Participant Analysis to Metamemory Accuracy: A Generalized Mixed-Effects Model Perspective

Journal of Experimental Psychology Learning Memory and Cognition ◽

10.1037/a0036914.supp ◽

2014 ◽

Keyword(s):

Type I Error ◽

Mixed Effects ◽

Mixed Effects Model ◽

Type I

Download Full-text

The Use of Theory of Linear Mixed-Effects Models to Detect Fraudulent Erasures at an Aggregate Level

Educational and Psychological Measurement ◽

10.1177/0013164421994893 ◽

2021 ◽

pp. 001316442199489

Author(s):

Luyao Peng ◽

Sandip Sinharay

Keyword(s):

Type I Error ◽

Real Data ◽

Mixed Effects ◽

Error Rates ◽

Mixed Effects Models ◽

Type I ◽

Aggregate Level ◽

Linear Mixed Effects Models ◽

Linear Mixed Effects ◽

Best Linear Unbiased

Wollack et al. (2015) suggested the erasure detection index (EDI) for detecting fraudulent erasures for individual examinees. Wollack and Eckerly (2017) and Sinharay (2018) extended the index of Wollack et al. (2015) to suggest three EDIs for detecting fraudulent erasures at the aggregate or group level. This article follows up on the research of Wollack and Eckerly (2017) and Sinharay (2018) and suggests a new aggregate-level EDI by incorporating the empirical best linear unbiased predictor from the literature of linear mixed-effects models (e.g., McCulloch et al., 2008). A simulation study shows that the new EDI has larger power than the indices of Wollack and Eckerly (2017) and Sinharay (2018). In addition, the new index has satisfactory Type I error rates. A real data example is also included.

Download Full-text

Design and analysis of a 2-year parallel follow-up of repeated ivermectin mass drug administrations for control of malaria: Small sample considerations for cluster-randomized trials with count data

Clinical Trials ◽

10.1177/17407745211028581 ◽

2021 ◽

pp. 174077452110285

Author(s):

Conner L Jackson ◽

Kathryn Colborn ◽

Dexiang Gao ◽

Sangeeta Rao ◽

Hannah C Slater ◽

...

Keyword(s):

Randomized Trial ◽

Cluster Randomized Trial ◽

Mixed Effects ◽

Small Sample ◽

Mixed Effects Models ◽

Type I ◽

Cluster Randomized ◽

Cluster Level ◽

Generalized Estimating

Background: Cluster-randomized trials allow for the evaluation of a community-level or group-/cluster-level intervention. For studies that require a cluster-randomized trial design to evaluate cluster-level interventions aimed at controlling vector-borne diseases, it may be difficult to assess a large number of clusters while performing the additional work needed to monitor participants, vectors, and environmental factors associated with the disease. One such example of a cluster-randomized trial with few clusters was the “efficacy and risk of harms of repeated ivermectin mass drug administrations for control of malaria” trial. Although previous work has provided recommendations for analyzing trials like repeated ivermectin mass drug administrations for control of malaria, additional evaluation of the multiple approaches for analysis is needed for study designs with count outcomes. Methods: Using a simulation study, we applied three analysis frameworks to three cluster-randomized trial designs (single-year, 2-year parallel, and 2-year crossover) in the context of a 2-year parallel follow-up of repeated ivermectin mass drug administrations for control of malaria. Mixed-effects models, generalized estimating equations, and cluster-level analyses were evaluated. Additional 2-year parallel designs with different numbers of clusters and different cluster correlations were also explored. Results: Mixed-effects models with a small sample correction and unweighted cluster-level summaries yielded both high power and control of the Type I error rate. Generalized estimating equation approaches that utilized small sample corrections controlled the Type I error rate but did not confer greater power when compared to a mixed model approach with small sample correction. The crossover design generally yielded higher power relative to the parallel equivalent. Differences in power between analysis methods became less pronounced as the number of clusters increased. The strength of within-cluster correlation impacted the relative differences in power. Conclusion: Regardless of study design, cluster-level analyses as well as individual-level analyses like mixed-effects models or generalized estimating equations with small sample size corrections can both provide reliable results in small cluster settings. For 2-year parallel follow-up of repeated ivermectin mass drug administrations for control of malaria, we recommend a mixed-effects model with a pseudo-likelihood approximation method and Kenward–Roger correction. Similarly designed studies with small sample sizes and count outcomes should consider adjustments for small sample sizes when using a mixed-effects model or generalized estimating equation for analysis. Although the 2-year parallel follow-up of repeated ivermectin mass drug administrations for control of malaria is already underway as a parallel trial, applying the simulation parameters to a crossover design yielded improved power, suggesting that crossover designs may be valuable in settings where the number of available clusters is limited. Finally, the sensitivity of the analysis approach to the strength of within-cluster correlation should be carefully considered when selecting the primary analysis for a cluster-randomized trial.

Download Full-text

Protocol for a two-arm pragmatic stepped-wedge hybrid effectiveness-implementation trial evaluating Engagement and Collaborative Management to Proactively Advance Sepsis Survivorship (ENCOMPASS)

BMC Health Services Research ◽

10.1186/s12913-021-06521-1 ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Marc Kowalkowski ◽

Tara Eaton ◽

Andrew McWilliams ◽

Hazel Tapp ◽

Aleta Rios ◽

...

Keyword(s):

Usual Care ◽

Best Practice ◽

Healthcare Delivery ◽

Collaborative Management ◽

High Morbidity ◽

Type I ◽

Stepped Wedge ◽

Sepsis Management ◽

Sepsis Survivors ◽

Sepsis Care

Abstract Background Sepsis survivors experience high morbidity and mortality, and healthcare systems lack effective strategies to address patient needs after hospital discharge. The Sepsis Transition and Recovery (STAR) program is a navigator-led, telehealth-based multicomponent strategy to provide proactive care coordination and monitoring of high-risk patients using evidence-driven, post-sepsis care tasks. The purpose of this study is to evaluate the effectiveness of STAR to improve outcomes for sepsis patients and to examine contextual factors that influence STAR implementation. Methods This study uses a hybrid type I effectiveness-implementation design to concurrently test clinical effectiveness and gather implementation data. The effectiveness evaluation is a two-arm, pragmatic, stepped-wedge cluster randomized controlled trial at eight hospitals in North Carolina comparing clinical outcomes between sepsis survivors who receive Usual Care versus care delivered through STAR. Each hospital begins in a Usual Care control phase and transitions to STAR in a randomly assigned sequence (one every 4 months). During months that a hospital is allocated to Usual Care, all eligible patients will receive usual care. Once a hospital transitions to STAR, all eligible patients will receive STAR during their hospitalization and extending through 90 days from discharge. STAR includes centrally located nurse navigators using telephonic counseling and electronic health record-based support to facilitate best-practice post-sepsis care strategies including post-discharge review of medications, evaluation for new impairments or symptoms, monitoring existing comorbidities, and palliative care referral when appropriate. Adults admitted with suspected sepsis, defined by clinical criteria for infection and organ failure, are included. Planned enrollment is 4032 patients during a 36-month period. The primary effectiveness outcome is the composite of all-cause hospital readmission or mortality within 90 days of discharge. A mixed-methods implementation evaluation will be conducted before, during, and after STAR implementation. Discussion This pragmatic evaluation will test the effectiveness of STAR to reduce combined hospital readmissions and mortality, while identifying key implementation factors. Results will provide practical information to advance understanding of how to integrate post-sepsis management across care settings and facilitate implementation, dissemination, and sustained utilization of best-practice post-sepsis management strategies in other heterogeneous healthcare delivery systems. Trial registration NCT04495946. Submitted July 7, 2020; Posted August 3, 2020.

Download Full-text

Practical R for biologists: an introduction

10.1079/9781789245349.0000 ◽

2021 ◽

Keyword(s):

Statistical Tests ◽

Statistical Modelling ◽

Biological Data ◽

Early Years ◽

Main Text ◽

Biological Data Analysis ◽

Base Functions ◽

Almost All ◽

Selection Of ◽

R Functions

Abstract R is an open-source statistical environment modelled after the previously widely used commercial programs S and S-Plus, but in addition to powerful statistical analysis tools, it also provides powerful graphics outputs. In addition to its statistical and graphical capabilities, R is a programming language suitable for medium-sized projects. This book presents a set of studies that collectively represent almost all the R operations that beginners, analysing their own data up to perhaps the early years of doing a PhD, need. Although the chapters are organized around topics such as graphing, classical statistical tests, statistical modelling, mapping and text parsing, examples have been chosen based largely on real scientific studies at the appropriate level and within each the use of more R functions is nearly always covered than are simply necessary just to get a p-value or a graph. R comes with around a thousand base functions which are automatically installed when R is downloaded. This book covers the use of those of most relevance to biological data analysis, modelling and graphics. Throughout each chapter, the functions introduced and used in that chapter are summarized in Tool Boxes. The book also shows the user how to adapt and write their own code and functions. A selection of base functions relevant to graphics that are not necessarily covered in the main text are described in Appendix 1, and additional housekeeping functions in Appendix 2.

Download Full-text

Incremental model breakdown to assess the multi-hypotheses problem

Hydrology and Earth System Sciences ◽

10.5194/hess-22-4565-2018 ◽

2018 ◽

Vol 22 (8) ◽

pp. 4565-4581 ◽

Cited By ~ 3

Author(s):

Florian U. Jehn ◽

Lutz Breuer ◽

Tobias Houska ◽

Konrad Bestian ◽

Philipp Kraft

Keyword(s):

Hydrological Modeling ◽

Model Performance ◽

Complex Model ◽

Model Parameters ◽

Objective Functions ◽

Final Model ◽

Incremental Model ◽

Model Efficiency ◽

Improved Model ◽

Model Structures

Abstract. The ambiguous representation of hydrological processes has led to the formulation of the multiple hypotheses approach in hydrological modeling, which requires new ways of model construction. However, most recent studies focus only on the comparison of predefined model structures or building a model step by step. This study tackles the problem the other way around: we start with one complex model structure, which includes all processes deemed to be important for the catchment. Next, we create 13 additional simplified models, where some of the processes from the starting structure are disabled. The performance of those models is evaluated using three objective functions (logarithmic Nash–Sutcliffe; percentage bias, PBIAS; and the ratio between the root mean square error and the standard deviation of the measured data). Through this incremental breakdown, we identify the most important processes and detect the restraining ones. This procedure allows constructing a more streamlined, subsequent 15th model with improved model performance, less uncertainty and higher model efficiency. We benchmark the original Model 1 and the final Model 15 with HBV Light. The final model is not able to outperform HBV Light, but we find that the incremental model breakdown leads to a structure with good model performance, fewer but more relevant processes and fewer model parameters.

Download Full-text

Estimating Bayes Factors for Linear Models with Random Slopes on Continuous Predictors

10.31234/osf.io/4xqvr ◽

2017 ◽

Author(s):

Mirko Thalmann ◽

Marcel Niklaus ◽

Klaus Oberauer

Keyword(s):

Bayesian Statistics ◽

Random Effects ◽

Mixed Models ◽

Mixed Effects ◽

Bayes Factors ◽

Mixed Effects Models ◽

Type I ◽

Frequentist Statistics ◽

Random Slopes ◽

Continuous Predictors

Using mixed-effects models and Bayesian statistics has been advocated by statisticians in recent years. Mixed-effects models allow researchers to adequately account for the structure in the data. Bayesian statistics – in contrast to frequentist statistics – can state the evidence in favor of or against an effect of interest. For frequentist statistical methods, it is known that mixed models can lead to serious over-estimation of evidence in favor of an effect (i.e., inflated Type-I error rate) when models fail to include individual differences in the effect sizes of predictors ("random slopes") that are actually present in the data. Here, we show through simulation that the same problem exists for Bayesian mixed models. Yet, at present there is no easy-to-use application that allows for the estimation of Bayes Factors for mixed models with random slopes on continuous predictors. Here, we close this gap by introducing a new R package called BayesRS. We tested its functionality in four simulation studies. They show that BayesRS offers a reliable and valid tool to compute Bayes Factors. BayesRS also allows users to account for correlations between random effects. In a fifth simulation study we show, however, that doing so leads to slight underestimation of the evidence in favor of an actually present effect. We only recommend modeling correlations between random effects when they are of primary interest and when sample size is large enough. BayesRS is available under https://cran.r-project.org/web/packages/BayesRS/, R code for all simulations is available under https://osf.io/nse5x/?view_only=b9a7caccd26a4764a084de3b8d459388

Download Full-text