The Explanatory Generalized Graded Unfolding Model: Incorporating Collateral Information to Improve the Latent Trait Estimation Accuracy

Collateral information has been used to address subpopulation heterogeneity and increase estimation accuracy in some large-scale cognitive assessments. The methodology that takes collateral information into account has not been developed and explored in published research with models designed specifically for noncognitive measurement. Because the accurate noncognitive measurement is becoming increasingly important, we sought to examine the benefits of using collateral information in latent trait estimation with an item response theory model that has proven valuable for noncognitive testing, namely, the generalized graded unfolding model (GGUM). Our presentation introduces an extension of the GGUM that incorporates collateral information, henceforth called Explanatory GGUM. We then present a simulation study that examined Explanatory GGUM latent trait estimation as a function of sample size, test length, number of background covariates, and correlation between the covariates and the latent trait. Results indicated the Explanatory GGUM approach provides scoring accuracy and precision superior to traditional expected a posteriori (EAP) and full Bayesian (FB) methods. Implications and recommendations are discussed.

Accounting for Item Response Process and Response Styles Using the Unfolding Item Response Tree (UIRTree) Model

Many researchers have found that unfolding models may better represent how respondents answer Liker-type items and response styles (RSs) often have moderate to strong presence in responses to such items. However, the two research lines have been growing largely in parallel. The present study proposed an unfolding item response tree (UIRTree) model that can account for unfolding response process and RSs simultaneously. An empirical illustration showed that the UIRTree model could fit a personality dataset well and produced more reasonable parameter estimates. Strong presence of the extreme response style (ERS) was also revealed by the UIRTree model. We further conducted a Monte Carlo simulation study to examine the performance of the UIRTree model compared to three other models for Likert-scale responses: the Samejima’s graded response model, the generalized graded unfolding model, and the dominance item response tree (DIRTree) model. Results showed that when data followed unfolding response process and contained the ERS, the AIC was able to select the UIRTree model, while BIC was biased towards the DIRTree model in many conditions. In addition, model parameters in the UIRTree model could be accurately recovered under realistic conditions, and wrongly assuming the item response process or ignoring RSs was detrimental to the estimation of key parameters. In general, the UIRTree model is expected to help in better understanding of responses to Liker-type items theoretically and contribute to better scale development practically. Future studies on multi-trait UIRTree models and UIRTree models accounting for different types of RSs are expected.

bmggum: An R Package for Bayesian Estimation of the Multidimensional Generalized Graded Unfolding Model With Covariates

Over the past couple of decades, there has been an increasing interest in adopting ideal point models to represent noncognitive constructs, as they have been demonstrated to better measure typical behaviors than traditional dominance models do. The generalized graded unfolding model ( GGUM) has consistently been the most popular ideal point model among researchers and practitioners. However, the GGUM2004 software and the later developed GGUM package in R can only handle unidimensional models despite the fact that many noncognitive constructs are multidimensional in nature. In addition, GGUM2004 and the GGUM package often yield unreasonable estimates of item parameters and standard errors. To address these issues, we developed the new open-source bmggum R package that is capable of estimating both unidimensional and multidimensional GGUM using a fully Bayesian approach, with supporting capabilities of stabilizing parameterization, incorporating person covariates, estimating constrained models, providing fit diagnostics, producing convergence metrics, and effectively handling missing data.

Though Forced, Still Valid: Psychometric Equivalence of Forced-Choice and Single-Statement Measures

Forced-choice (FC) measures are gaining popularity as an alternative assessment format to single-statement (SS) measures. However, a fundamental question remains to be answered: Do FC and SS instruments measure the same underlying constructs? In addition, FC measures are theorized to be more cognitively challenging, so how would this feature influence respondents’ reactions to FC measures compared to SS? We used both between- and within-subjects designs to examine the equivalence of the FC format and the SS format. As the results illustrate, FC measures scored by the multi-unidimensional pairwise preference (MUPP) model and SS measures scored with the generalized graded unfolding model (GGUM) showed strong equivalence. Specifically, both formats demonstrated similar marginal reliabilities and test-retest reliabilities, high convergent validities, good discriminant validities, and similar criterion-related validities with theoretically relevant criteria. In addition, the formats had little differential impact on respondents’ general emotional and cognitive reactions except that the FC format was perceived to be slightly more difficult and more time-saving.

GGUM: An R Package for Fitting the Generalized Graded Unfolding Model

In this article, the newly created GGUM R package is presented. This package finally brings the generalized graded unfolding model (GGUM) to the front stage for practitioners and researchers. It expands the possibilities of fitting this type of item response theory (IRT) model to settings that, up to now, were not possible (thus, beyond the limitations imposed by the widespread GGUM2004 software). The outcome is therefore a unique software, not limited by the dimensions of the data matrix or the operating system used. It includes various routines that allow fitting the model, checking model fit, plotting the results, and also interacting with GGUM2004 for those interested. The software should be of interest to all those who are interested in IRT in general or to ideal point models in particular.

Evaluating Anchor-Item Designs for Concurrent Calibration With the GGUM

Concurrent calibration using anchor items has proven to be an effective alternative to separate calibration and linking for developing large item banks, which are needed to support continuous testing. In principle, anchor-item designs and estimation methods that have proven effective with dominance item response theory (IRT) models, such as the 3PL model, should also lead to accurate parameter recovery with ideal point IRT models, but surprisingly little research has been devoted to this issue. This study, therefore, had two purposes: (a) to develop software for concurrent calibration with, what is now the most widely used ideal point model, the generalized graded unfolding model (GGUM); (b) to compare the efficacy of different GGUM anchor-item designs and develop empirically based guidelines for practitioners. A Monte Carlo study was conducted to compare the efficacy of three anchor-item designs in vertical and horizontal linking scenarios. The authors found that a block-interlaced design provided the best parameter recovery in nearly all conditions. The implications of these findings for concurrent calibration with the GGUM and practical recommendations for pretest designs involving ideal point computer adaptive testing (CAT) applications are discussed.

The lz(p)* Person-Fit Statistic in an Unfolding Model Context

Although person-fit analysis has a long-standing tradition within item response theory, it has been applied in combination with dominance response models almost exclusively. In this article, a popular log likelihood-based parametric person-fit statistic under the framework of the generalized graded unfolding model is used. Results from a simulation study indicate that the person-fit statistic performed relatively well in detecting midpoint response style patterns and not so well in detecting extreme response style patterns.

Constructing a Scale of Attitudes toward School Science Using the General Graded Unfolding Model

The present study aimed at constructing an attitude scale toward school science using the generalized graded unfolding model (GGUM). A 47-item scale (24 positive, 23 negative) with 4-point response format was used to measure attitudes toward science among 9th (n=424) and 10th (n=420) grade students in 38 sections distributed randomly over 22 schools in Irbid district. Respondents selected one of four options to represent their level of agreement with each item. The findings support the hypothesis that the data form a single unidimensional unfolding model. Furthermore, the findings showed that the GGUM didn’t fit the data of 7 items, leaving the final scale with 40 items, where accurate estimates of these item parameters were derived and the GGUM was appropriate. Cronbach's alpha for the internal consistency, and the test retest reliability coefficients of the final scale were 0.932 and 0.875, respectively.

A Comparison of the LR and DFIT Frameworks of Differential Functioning Applied to the Generalized Graded Unfolding Model

Recently, applied psychological measurement researchers have become interested in the application of the generalized graded unfolding model (GGUM), a parametric item response theory model that posits an ideal point conception of the relationship between latent attributes and observed item responses. Little attention has been given to considerations for the detection of differential item functioning (DIF) under the GGUM. In this article, the authors present a Monte Carlo simulation meant to assess the efficacy of the likelihood ratio (LR) and differential functioning of items and tests (DFIT) frameworks, two popular ways of detecting DIF. Findings indicate a marked superiority of the LR approach over DFIT in terms of true and false positive rates under the GGUM. The discussion centers on possible explanations for the poor performance of the DFIT framework in detecting DIF under the GGUM and addresses limitations of the current study as well as future research directions.