Although the use of structural equation modeling has increased during the last decades, the typical procedure to investigate mean differences across groups is still to create an observed composite score from several indicators and to compare the composite’s mean across the groups. Whereas the structural equation modeling literature has emphasized that a comparison of latent means presupposes equal factor loadings and indicator intercepts for most of the indicators (i.e., partial invariance), it is still unknown if partial invariance is sufficient when relying on observed composites. This Monte-Carlo study investigated whether one or two unequal factor loadings and indicator intercepts in a composite can lead to wrong conclusions regarding latent mean differences. Results show that unequal indicator intercepts substantially affect the composite mean difference and the probability of a significant composite difference. In contrast, unequal factor loadings demonstrate only small effects. It is concluded that analyses of composite differences are only warranted in conditions of full measurement invariance, and the author recommends the analyses of latent mean differences with structural equation modeling instead.
A Strategy for Using Bias and RMSE as Outcomes in Monte Carlo Studies in Statistics