The Reliability of Multidimensional Scales: A Comparison of Confidence Intervals and a Bayesian Alternative
The reliability of a multidimensional test instrument is commonly estimated using coefficients ωt (total) and ωh (hierarchical) based on a factor model approach. However, point estimates for the coefficients are rarely accompanied by uncertainty estimates. In this study we compare several methods to obtain confidence intervals for the two coefficients: bootstrap and normal-theory intervals. In addition, we adapt methodology from Bayesian structural equation modeling to develop Bayesian versions of coefficients ωt and ωh by sampling from a second-order factor model. Results from a comprehensive simulation study show that the bootstrap standard error confidence interval, the bootstrap standard error log-transformed confidence interval, the Wald confidence interval, and the Bayesian credible interval perform well across a wide range of conditions. This study provides researchers with more information about the ωt and ωh confidence intervals they wish to report in their research. Moreover, the study introduces ωt and ωh credible intervals that are easy to use and come with all the benefits of Bayesian parameter estimation.