Linear Model Complexities

This book began with the simplest types of linear model: one-way ANOVA and its simple linear regression equivalent. However, once more complex ANOVA and ANCOVA designs were encountered some complexities arose that were then skipped over. This chapter explores these complexities of linear-model analysis and some additional ones that arise with unbalanced designs—those with unequal numbers of replicates in the different treatment groups.

Reevaluating the SIBTEST Classification Heuristics for Dichotomous Differential Item Functioning

A simulation study was conducted to investigate the heuristics of the SIBTEST procedure and how it compares with ETS classification guidelines used with the Mantel–Haenszel procedure. Prior heuristics have been used for nearly 25 years, but they are based on a simulation study that was restricted due to computer limitations and that modeled item parameters from estimates of ACT and ASVAB tests from 1987 and 1984, respectively. Further, suggested heuristics for data fitting a two-parameter logistic model (2PL) have essentially went unused since their original presentation. This simulation study incorporates a wide range of data conditions to recommend heuristics for both 2PL and three-parameter logistic (3PL) data that correspond with ETS’s Mantel–Haenszel heuristics. Levels of agreement between the new SIBTEST heuristics and Mantel–Haenszel heuristics were similar for 2PL data and higher than prior SIBTEST heuristics for 3PL data. The new recommendations provide higher true-positive rates for 2PL data. Conversely, they displayed decreased true-positive rates for 3PL data. False-positive rates, overall, remained below the level of significance for the new heuristics. Unequal group sizes resulted in slightly larger false-positive rates than balanced designs for both prior and new SIBTEST heuristics, with rates less than alpha levels for equal ability distributions and unbalanced designs versus false-positive rates slightly higher than alpha with unequal ability distributions and unbalanced designs.

Statistical modeling of vignette data in psychology

Vignette methods are widely used in psychology and the social sciences to obtain responses to multi-dimensional scenarios or situations. Where quantitative data are collected this presents challenges to the selection of an appropriate statistical model. This depends on subtle details of the design and allocation of vignettes to participants. A key distinction is between factorial survey experiments where each participant receives a different allocation of vignettes from the full universe of possible vignettes and experimental vignette studies where this restriction is relaxed. The former leads to nested designs with a single random factor and the latter to designs with two crossed random factors. In addition, the allocation of vignettes to participants may lead to fractional or unbalanced designs and a consequent loss of efficiency or aliasing of the effects of interest. Many vignette studies (including some factorial survey experiments) include unmodeled heterogeneity between vignettes leading to potentially serious problems if traditional regression approaches are adopted. These issues are reviewed and recommendations are made for the efficient design of vignette studies including the allocation of vignettes to participants. Multilevel models are proposed as a general approach to handling nested and crossed designs including unbalanced and fractional designs. This is illustrated with a small vignette data set looking at judgements of online and offline bullying and harassment.

Multiple imputation to balance unbalanced designs for two-way analysis of variance

A balanced ANOVA design provides an unambiguous interpretation of the F-tests, and has more power than an unbalanced design. In earlier literature, multiple imputation was proposed to create balance in unbalanced designs, as an alternative to Type-III sum of squares. In the current simulation study we studied four pooled statistics for multiple imputation, namely D₀, D₁, D₂, and D₃ in unbalanced data, and compared them with Type-III sum of squares. Statistics D₁ and D₂ generally performed best regarding Type-I error rates, and had power rates closest to that of Type-III sum of squares. Additionally, for the interaction, D₁ produced power rates higher than Type-III sum of squares. For multiply imputed datasets D₁ and D₂ may be the best methods for pooling the results in multiply imputed datasets, and for unbalanced data, D₁ might be a good alternative to Type-III sum of squares regarding the interaction.

WE-ASCA: The Weighted-Effect ASCA for Analyzing Unbalanced Multifactorial Designs—A Raman Spectra-Based Example

Analyses of multifactorial experimental designs are used as an explorative technique describing hypothesized multifactorial effects based on their variation. The procedure of analyzing multifactorial designs is well established for univariate data, and it is known as analysis of variance (ANOVA) tests, whereas only a few methods have been developed for multivariate data. In this work, we present the weighted-effect ASCA, named WE-ASCA, as an enhanced version of ANOVA-simultaneous component analysis (ASCA) to deal with multivariate data in unbalanced multifactorial designs. The core of our work is to use general linear models (GLMs) in decomposing the response matrix into a design matrix and a parameter matrix, while the main improvement in WE-ASCA is to implement the weighted-effect (WE) coding in the design matrix. This WE-coding introduces a unique solution to solve GLMs and satisfies a constrain in which the sum of all level effects of a categorical variable equal to zero. To assess the WE-ASCA performance, two applications were demonstrated using a biomedical Raman spectral data set consisting of mice colorectal tissue. The results revealed that WE-ASCA is ideally suitable for analyzing unbalanced designs. Furthermore, if WE-ASCA is applied as a preprocessing tool, the classification performance and its reproducibility can significantly improve.

Much ado about nothing: Multiple imputation to balance unbalanced designs for two-way analysis of variance

In earlier literature, multiple imputation was proposed to create balance in unbalanced designs, as an alternative to Type III sum of squares in two-way ANOVA. In the current simulation study we studied four pooled statistics for multiple imputation, namely D₀, D₁, D₂, and D₃ in unbalanced data, and compared these statistics with Type III sum of squares. Statistics D₀ and D₂ generally performed best regarding Type-I error rates, and had power rates closest to that of Type III sum of squares. However, none of the statistics produced power rates higher than Type III sum of squares. The results lead to the conclusion that for multiply imputed datasets D₀ and D₂ may be the best methods for pooling the results of multiparameter estimates in multiply imputed datasets, and that for unbalanced data, Type III sum of square is to be preferred over using multiple imputation in obtaining ANOVA results.