Measurement Invariance with Ordered Categorical Variables

Multi-item ordered categorical scales and structural equation modelling approaches are often used in panel research for the analysis of latent variables over time. The accuracy of such models depends on the assumption of longitudinal measurement invariance (LMI), which states that repeatedly measured latent variables should effectively represent the same construct in the same metric at each time point. Previous research has widely contributed to the LMI literature for continuous variables, but these findings might not be generalized to ordered categorical data. Treating ordered categorical data as continuous contradicts the assumption of multivariate normality and could potentially produce inaccuracies and distortions in both invariance testing results and structural parameter estimates. However, there is still little research that examines and compares criteria for establishing LMI with ordinal categorical data. Drawing on this lack of evidence, the present chapter offers a detailed description of the main procedures used to test for LMI with ordered categorical variables, accompanied by examples of their practical application in a two-wave longitudinal survey administered to 1,912 Italian middle school teachers. The empirical study evaluates whether different testing procedures, when applied to ordered categorical data, lead to similar conclusions about model fit, invariance, and structural parameters over time.

Continuous and Ordered Categorical Data in Network Psychometrics: Which Estimation Method to Choose? Deriving Guidelines for Applied Researchers

The Gaussian Graphical Model (GGM) has recently grown popular in psychological research, with a large body of estimation methods being proposed and discussed across various fields of study, and several algorithms being identified and recommend as applicable to psychological datasets. Such high-dimensional model estimation, however, is not trivial, and algorithms tend to perform differently in different settings. In addition, psychological research poses unique challenges, including placing a strong focus on weak edges (e.g., bridge edges), handling data measured on ordered scales, and relatively limited sample sizes. As a result, there is currently no consensus regarding which estimation procedure performs best in which setting. In this large-scale simulation study, we aimed to overcome this gap in the literature by comparing the performance of several estimation algorithms suitable for gaussian and skewed ordered categorical data across a multitude of settings, as to arrive at concrete guidelines from applied researchers. In total, we investigated 60 different metrics across 564,000 simulated datasets. We summarized our findings through a platform that allows for manually exploring simulation results. Overall, we found that an exchange between discovery (e.g., sensitivity, edge weight correlation) and caution (e.g., specificity, precision) should always be expected and achieving both¬—which is a requirement for perfect replicability—is difficult. Further, we identified that the estimation method is best chosen in light of each research question and highlighted, alongside desirable asymptotic properties and low sample size discovery, results according to most common research questions in the field.

Using Pooled Heteroskedastic Ordered Probit Models to Improve Small-Sample Estimates of Latent Test Score Distributions

This article describes an extension to the use of heteroskedastic ordered probit (HETOP) models to estimate latent distributional parameters from grouped, ordered-categorical data by pooling across multiple waves of data. We illustrate the method with aggregate proficiency data reporting the number of students in schools or districts scoring in each of a small number of ordered “proficiency” levels. HETOP models can be used to estimate means and standard deviations of the underlying (latent) test score distributions but may yield biased or very imprecise estimates when group sample sizes are small. A simulation study demonstrates that the pooled HETOP models described here can reduce the bias and sampling error of standard deviation estimates when group sample sizes are small. Analyses of real test score data demonstrate the use of the models and suggest the pooled models are likely to improve estimates in applied contexts.

A comparison of generalised linear models and compositional models for ordered categorical data

Ordered categorical data occur in many applied fields, such as geochemistry, econometrics, sociology and demography or even transportation research, for example, in the form of results from various questionnaires. There are different possibilities for modelling proportions of individual categories. Generalised linear models (GLMs) are traditionally used for this purpose, but also methods of compositional data analysis (CoDa) can be considered. Here, both approaches are compared in depth. Particularly, different assumptions of the models on variability are highlighted. Advantages and disadvantages of individual models are pointed out. While the CoDa model may be inappropriate when the variability of the compositional coordinates depends on the regressors, for example, due to different total counts on which the coordinates are based, the GLM may underestimate the uncertainty of the predictions considerably in case of large-scale data.