MUPPscore: An R script for expected a posteriori scoring of multidimensional pairwise preference items

In this manual, we present a flexible and freely available tool for obtaining latent trait scores from multi-unidimensional pairwise preference (MUPP) tests: An R script named MUPPscore. The development of the MUPPscore script provides a solution to the issue that is the previously inconvenient estimation of forced choice item pairs. Instead of using the computationally-intensive multidimensional Bayes modal procedure, the MUPPscore script employs the expected a posterior (EAP) scoring procedure, which provides plausible latent trait score estimates and is also consistent with scoring algorithms used in existing software programs intended for single stimulus measures (e.g., GGUM2004, IRTPRO). The MUPPscore script also returns the empirical marginal reliability of EAP theta estimates and outputs a series of files that can be used to easily create and modify three-dimensional surface charts for plotting MUPP item response function (IRF) in Microsoft Excel.

Approximating Bifactor IRT True-Score Equating With a Projective Item Response Model

Item response theory (IRT) true-score equating for the bifactor model is often conducted by first numerically integrating out specific factors from the item response function and then applying the unidimensional IRT true-score equating method to the marginalized bifactor model. However, an alternative procedure for obtaining the marginalized bifactor model is through projecting the nuisance dimensions of the bifactor model onto the dominant dimension. Projection, which can be viewed as an approximation to numerical integration, has an advantage over numerical integration in providing item parameters for the marginalized bifactor model; therefore, projection could be used with existing equating software packages that require item parameters. In this paper, IRT true-score equating results obtained with projection are compared to those obtained with numerical integration. Simulation results show that the two procedures provide very similar equating results.

Detection Rates of the M2 Test for Nonzero Lower Asymptotes Under Normal and Nonnormal Ability Distributions in the Applications of IRT

When considering the two-parameter or the three-parameter logistic model for item responses from a multiple-choice test, one may want to assess the need for the lower asymptote parameters in the item response function and make sure the use of the three-parameter item response model. This study reports the degree of sensitivity of an overall model test M2 to detecting the presence of nonzero asymptotes in the item response function under normal and nonnormal ability distribution conditions.

The Delta-Scoring Method of Tests With Binary Items: A Note on True Score Estimation and Equating

This article presents some new developments in the methodology of an approach to scoring and equating of tests with binary items, referred to as delta scoring (D-scoring), which is under piloting with large-scale assessments at the National Center for Assessment in Saudi Arabia. This presentation builds on a previous work on delta scoring and adds procedures for scaling and equating, item response function, and estimation of true values and standard errors of D scores. Also, unlike the previous work on this topic, where D-scoring involves estimates of item and person parameters in the framework of item response theory, the approach presented here does not require item response theory calibration.

Nonparametric Item Response Function Estimates with the EM Algorithm

The methods of functional data analysis are used to estimate item response functions (IRFs) nonparametrically. The EM algorithm is used to maximize the penalized marginal likelihood of the data. The penalty controls the smoothness of the estimated IRFs, and is chosen so that, as the penalty is increased, the estimates converge to shapes closely represented by the three-parameter logistic family. The one-dimensional latent trait model is recast as a problem of estimating a space curve or manifold, and, expressed in this way, the model no longer involves any latent constructs, and is invariant with respect to choice of latent variable. Some results from differential geometry are used to develop a data-anchored measure of ability and a new technique for assessing item discriminability. Functional data-analytic techniques are used to explore the functional variation in the estimated IRFs. Applications involving simulated and actual data are included.

A Generalized Formula for the Mantel-Haenszel Differential Item Functioning Parameter

The present study derives a general formula for the population parameter being estimated by the Mantel-Haenszel (MH) differential item functioning (DIF) statistic. Because the formula is general, it is appropriate for either uniform DIF (defined as a difference in item response theory item difficulty values) or nonuniform DIF; and it can be used regardless of the form of the item response function. In the case of uniform DIF modeled with two-parameter-logistic response functions, the parameter is well known to be linearly related to the difference in item difficulty between the focal and reference groups. Even though this relationship is known to not strictly hold true in the case of three-parameter-logistic (3PL) uniform DIE the degree of the departure from this relationship has not been known and has been generally believed to be small By evaluating the MH DIF parameter, we show that for items of medium or high difficulty, the parameter is much smaller in absolute value than expected based on the difference in item difficulty between the two groups. These results shed new light on results from previous simulation studies that showed the MH DIF statistic has a tendency to shrink toward zero with increasing difficulty level when used with 3PL data.