The primary objective of this work is to extend classical test theory (CTT), in particular, forthe case of repeated measurement studies. The guiding idea that motivates this work is that anytheory ought to be expanded when it is not compatible with commonly observed phenomena-namely, that homogeneous variance components appear to be the exception and not the rule inpsychological applications. Additionally, advancements in methodology should also be consideredin light of theory expansion, when appropriate. We argue both goals can be accomplishedby merging heterogeneous variance modeling with the central tenants of CTT. To this end, weintroduce novel methodology that is based on the mixed-effects location scale model. This allows for fitting explanatory models to the true score (between-group) and error (within-group)variance. Two illustrative examples, that span from educational research to infant cognition,highlight such possibilities. The results revealed that there can be substantial individual differences in error variance, which necessarily implies the same for reliability, and that true scorevariance can be a function of covariates. We incorporate this variance heterogeneity into novel reliability indices that can be used to forecast group or person-specific reliability. These extend traditional formulations that assume the variance components are homogeneous. This powerful approach can be used to identify predictors of true score and error variance, which can then be used to refine measurement. The methods are implemented in the user-friendly R packageICCier.
Visualising two approaches to explore reliability-power relationships