Stepwise methods can limit power for hypothesis tests of cross-level interactions

In multilevel models, stepwise methods are commonly used to test cross-level interactions, where a cluster level variable explains differences in the effect of an observation level variable on the outcome.Researchers often wish to establish that there is between cluster variance in slopes before testing whether an observed cluster level variable explains between cluster variance in slopes.In the stepwise method, between cluster slope variance (i.e. random slopes) is required before a cross-level interaction is tested.We argue that this requirement unnecessarily reduces the power to detect true cross-level interactions, because it imposes an unnecessary constraint on the power to detect valid interactions.In short, the stepwise approach would only be valid if, in the same data, the stepwise approach always identifies true interactions that are also identified by a direct test of the interaction model. Using Monte Carlo simulations, we demonstrate that this is not the case.The power to detect a true interaction was especially low when the residual slope variance (i.e. variance unexplained by the interaction), the variance of the moderator, the number of observations per cluster, or the number of clusters was small.We recommend that researchers directly test interactions that are of interest, regardless of the presence of random slope variance.