scholarly journals autoFC: An R Package for Automatic Item Pairing in Forced-Choice Test Construction

2021 ◽  
Author(s):  
Mengtong Li ◽  
Tianjun Sun ◽  
Bo Zhang

Recently, there has been increasing interest in adopting the forced-choice (FC) test format in non-cognitive assessments, as it demonstrates faking resistance when well-designed. However, traditional or manual pairing approaches to FC test construction are time- and effort- intensive, and often involves insufficient considerations. To address these issues, we developed the new open-source autoFC R package to facilitate automated and optimized item pairing strategies. The autoFC package is intended as a practical tool for FC test constructions. Users can easily obtain automatically optimized FC tests by simply inputting the item characteristics of interest. Customizations are also available for considerations on matching indices and the behaviors of the optimization process. The autoFC package should be of interest to researchers and practitioners constructing FC scales with potentially many metrics to match on and/or many items to pair, essentially exempting users from the burden of manual item pairing and reducing the computational costs and biases induced by simple ranking methods.

2021 ◽  
pp. 014662162110517
Author(s):  
Mengtong Li ◽  
Tianjun Sun ◽  
Bo Zhang

Recently, there has been increasing interest in adopting the forced-choice (FC) test format in non-cognitive assessments, as it demonstrates faking resistance when well-designed. However, traditional or manual pairing approaches to FC test construction are time- and effort- intensive and often involve insufficient considerations. To address these issues, we developed the new open-source autoFC R package to facilitate automated and optimized item pairing strategies. The autoFC package is intended as a practical tool for FC test constructions. Users can easily obtain automatically optimized FC tests by simply inputting the item characteristics of interest. Customizations are also available for considerations on matching rules and the behaviors of the optimization process. The autoFC package should be of interest to researchers and practitioners constructing FC scales with potentially many metrics to match on and/or many items to pair, essentially exempting users from the burden of manual item pairing and reducing the computational costs and biases induced by simple ranking methods.


2021 ◽  
Vol 183 ◽  
pp. 111114
Author(s):  
Goran Pavlov ◽  
Dexin Shi ◽  
Alberto Maydeu-Olivares ◽  
Amanda Fairchild

2021 ◽  
Author(s):  
Goran Pavlov ◽  
Dexin Shi ◽  
ALBERTO MAYDEU-OLIVARES

The forced-choice method has been proposed as a viable strategy to prevent socially desirable responding (SDR) on self-report non-cognitive measures. The ability of the method to eliminate SDR may largely depend on how closely items comprising forced-choice item-blocks are matched in terms of perceived desirability. The gold standard in quantifying similarity between items in terms of desirability has been the mean difference index, that is, the absolute difference between items’ mean desirability ratings. The mean difference index relies on the assumption that items have one “true” desirability value, as represented by their means, and may fail if this assumption does not hold. Instead, we propose indexing within-rater agreement with several robust agreement indices to appropriately quantify similarity between items in terms of desirability (i.e., inter-item agreement). On a set of empirically derived desirability ratings, we show that relying on the mean difference index may lead to suboptimal forced-choice item assembly. Implications of our findings and future research directions are discussed. R code for computing the proposed indices on a set of desirability ratings is provided.


1968 ◽  
Vol 28 (4) ◽  
pp. 1103-1110 ◽  
Author(s):  
Leonard V. Gordon ◽  
Richard J. Hofmann
Keyword(s):  

1994 ◽  
Vol 8 (1) ◽  
pp. 118-125 ◽  
Author(s):  
Richard I. Frederick ◽  
Stephen D. Sarfaty ◽  
J. Dennis Johnston ◽  
Jeffrey Powel

1991 ◽  
Vol 3 (4) ◽  
pp. 596-602 ◽  
Author(s):  
Richard I. Frederick ◽  
Hilliard G. Foster

2010 ◽  
Vol 13 (2) ◽  
pp. 518-524 ◽  
Author(s):  
Salvador Algarabel ◽  
Alfonso Pitarque

This experiment compares the yes-no and forced recognition tests as methods of measuring familiarity. Participants faced a phase of 3 study-test recognition trials in which they studied words using all the letters of the alphabet (overlapping condition, O), and an additional phase in which targets and lures did not share any letters (non-overlapping condition, NO). Finally, subjects performed a forced-choice task in which they had to choose one of two new words, each from one of the subsets (Parkin et al., 2001). Results in the NO condition were better than in the O condition in the yes-no recognition test, while the forced-choice rate was significantly higher than .50, showing their sensitivity to familiarity. When the letter set of the words for study in the third list of the NO condition was switched, the difference between NO and O conditions disappeared in yes-no test, while the force-choice rate was not higher than .50. We conclude that both the yes-no test and the forced-choice test are valid and equivalent measures of familiarity under the right conditions.


2010 ◽  
Vol 21 (5) ◽  
pp. 547-552 ◽  
Author(s):  
Scott McClure ◽  
Harry T. Lawless
Keyword(s):  

Sign in / Sign up

Export Citation Format

Share Document