Development and refinement of self-report measures generally involves selecting a subset of indicators from a larger set. Despite the importance of this task, methods applied to accomplish this are often idiosyncratic and ad hoc, or based on incomplete statistical criteria. We describe a structural equation modeling (SEM)-based technique, based on the standardized residual variance–covariance matrix, which subsumes multiple traditional psychometric criteria: item homogeneity, reliability, convergent, and discriminant validity. SEMs with a fixed structure, but with substituted candidate items, can be used to evaluate the relative performance of those items. Using simulated data sets, we demonstrate a simple progressive elimination algorithm, which demonstrably optimizes item choice across multiple psychometric criteria. This method is then applied to the task of short-form development of the multidimensional “4Es” (Excitement, Esteem, Escape, Excess) scale, which are understood as indicators of psychological vulnerability to gambling problems. It is concluded that the proposed SEM-based algorithm provides an automatic and efficient approach to the item-reduction stage of scale development and should be similarly useful for the development of short forms of preexisting scales. Broader use of such an algorithm would promote more transparent, consistent, and replicable scale development.
The Adolescent Distress-Eustress Scale: Development and Validation