Accounting for Item Response Process and Response Styles Using the Unfolding Item Response Tree (UIRTree) Model

Many researchers have found that unfolding models may better represent how respondents answer Liker-type items and response styles (RSs) often have moderate to strong presence in responses to such items. However, the two research lines have been growing largely in parallel. The present study proposed an unfolding item response tree (UIRTree) model that can account for unfolding response process and RSs simultaneously. An empirical illustration showed that the UIRTree model could fit a personality dataset well and produced more reasonable parameter estimates. Strong presence of the extreme response style (ERS) was also revealed by the UIRTree model. We further conducted a Monte Carlo simulation study to examine the performance of the UIRTree model compared to three other models for Likert-scale responses: the Samejima’s graded response model, the generalized graded unfolding model, and the dominance item response tree (DIRTree) model. Results showed that when data followed unfolding response process and contained the ERS, the AIC was able to select the UIRTree model, while BIC was biased towards the DIRTree model in many conditions. In addition, model parameters in the UIRTree model could be accurately recovered under realistic conditions, and wrongly assuming the item response process or ignoring RSs was detrimental to the estimation of key parameters. In general, the UIRTree model is expected to help in better understanding of responses to Liker-type items theoretically and contribute to better scale development practically. Future studies on multi-trait UIRTree models and UIRTree models accounting for different types of RSs are expected.

Modeling Self-Determination Theory Motivation Data by Using Unfolding IRT

Self-determination theory (SDT) suggests that the degree of autonomous behavior regulation is a characteristic of distinct motivation types which thus can be ordered on the so-called Autonomy-Control Continuum (ACC). The present study employs an item response theory (IRT) model under the ideal point response/unfolding paradigm in order to model the response process to SDT motivation items in theoretical accordance with the ACC. Using data from two independent student samples (measuring SDT motivation for the academic subjects of Mathematics and German as a native language), it was found that an unfolding model exhibited a relatively better fit compared to a dominance model. The item location parameters under the unfolding paradigm showed clusters of items representing the different regulation types on the ACC to be (almost perfectly) empirically separable, as suggested by SDT. Besides theoretical implications, perspectives for the application of ideal point response/unfolding models in the development of measures for non-cognitive constructs are addressed.

Exploring the Impersonal Judgments and Personal Preferences of Raters in Rater-Mediated Assessments With Unfolding Models

The purpose of this study is to explore the use of unfolding models for evaluating the quality of ratings obtained in rater-mediated assessments. Two different judgmental processes can be used to conceptualize ratings: impersonal judgments and personal preferences. Impersonal judgments are typically expected in rater-mediated assessments, and these ratings reflect a cumulative response process. However, raters may also be influenced by their personal preferences in providing ratings, and these ratings may reflect a noncumulative or unfolding response process. The goal of rater training in rater-mediated assessments is to stress impersonal judgments represented by scoring rubrics and to minimize the personal preferences that may represent construct-irrelevant variance in the assessment system. In this study, we explore the use of unfolding models as a framework for evaluating the quality of ratings in rater-mediated assessments. Data from a large-scale assessment of writing in the United States are used to illustrate our approach. The results suggest that unfolding models offer a useful way to evaluate rater-mediated assessments in order to initially explore the judgmental processes underlying the ratings. The data also indicate that there are significant relationships between some essay features (e.g., word count, syntactic simplicity, word concreteness, and verb cohesion) and essay orderings based on the personal preferences of raters. The implications of unfolding models for theory and practice in rater-mediated assessments are discussed.

Unfolding Models for Asymmetric Dissimilarity Data With External Information Based on Path Structures

This article contains asymmetric dissimilarity data which is observed in various situations. In asymmetric dissimilarity data, dissimilarity from subject i to j and from subject j to i are not the same necessarily. Asymmetric multidimensional scaling (AMDS) is a visualization method for describing the asymmetric relations between subjects, given asymmetric dissimilarity data for subjects. It is sure that AMDS is a useful tool for interpreting the asymmetric relation, however, existing AMDS cannot be considered for the external information, even if the external information of the same subjects for the asymmetric dissimilarity data is given. If the estimated coordinates can be interpreted from the loading matrix for the external information like principal component analysis (PCA), the AMDS become more useful. This is because we can interpret the relation between the estimated asymmetries and the factors of the external information on the low dimensions. In this article, we proposed new AMDS with external information. In addition to that, the proposed method can consider the path structure for variables like SEM.