scholarly journals autoFC: An R Package for Automatic Item Pairing in Forced-Choice Test Construction

2021 ◽  
pp. 014662162110517
Author(s):  
Mengtong Li ◽  
Tianjun Sun ◽  
Bo Zhang
Keyword(s):  
R Package ◽  
Test Construction ◽  
Forced Choice ◽  
Choice Test ◽  
Ranking Methods ◽  
Cognitive Assessments ◽  
Computational Costs ◽  
Item Characteristics ◽  
Practical Tool ◽  
Matching Rules

Recently, there has been increasing interest in adopting the forced-choice (FC) test format in non-cognitive assessments, as it demonstrates faking resistance when well-designed. However, traditional or manual pairing approaches to FC test construction are time- and effort- intensive and often involve insufficient considerations. To address these issues, we developed the new open-source autoFC R package to facilitate automated and optimized item pairing strategies. The autoFC package is intended as a practical tool for FC test constructions. Users can easily obtain automatically optimized FC tests by simply inputting the item characteristics of interest. Customizations are also available for considerations on matching rules and the behaviors of the optimization process. The autoFC package should be of interest to researchers and practitioners constructing FC scales with potentially many metrics to match on and/or many items to pair, essentially exempting users from the burden of manual item pairing and reducing the computational costs and biases induced by simple ranking methods.

scholarly journals autoFC: An R Package for Automatic Item Pairing in Forced-Choice Test Construction

2021 ◽  
Author(s):  
Mengtong Li ◽  
Tianjun Sun ◽  
Bo Zhang
Keyword(s):  
R Package ◽  
Test Construction ◽  
Forced Choice ◽  
Choice Test ◽  
Test Format ◽  
Ranking Methods ◽  
Cognitive Assessments ◽  
Computational Costs ◽  
Item Characteristics ◽  
Practical Tool

Recently, there has been increasing interest in adopting the forced-choice (FC) test format in non-cognitive assessments, as it demonstrates faking resistance when well-designed. However, traditional or manual pairing approaches to FC test construction are time- and effort- intensive, and often involves insufficient considerations. To address these issues, we developed the new open-source autoFC R package to facilitate automated and optimized item pairing strategies. The autoFC package is intended as a practical tool for FC test constructions. Users can easily obtain automatically optimized FC tests by simply inputting the item characteristics of interest. Customizations are also available for considerations on matching indices and the behaviors of the optimization process. The autoFC package should be of interest to researchers and practitioners constructing FC scales with potentially many metrics to match on and/or many items to pair, essentially exempting users from the burden of manual item pairing and reducing the computational costs and biases induced by simple ranking methods.

scholarly journals Item desirability matching in forced-choice test construction

2021 ◽  
Vol 183 ◽  
pp. 111114
Author(s):  
Goran Pavlov ◽  
Dexin Shi ◽  
Alberto Maydeu-Olivares ◽  
Amanda Fairchild
Keyword(s):  
Test Construction ◽  
Forced Choice ◽  
Choice Test

scholarly journals Item Desirability Matching in Forced-choice Test Construction

2021 ◽  
Author(s):  
Goran Pavlov ◽  
Dexin Shi ◽  
ALBERTO MAYDEU-OLIVARES
Keyword(s):  
Test Construction ◽  
Forced Choice ◽  
Mean Difference ◽  
Self Report ◽  
Choice Test ◽  
Absolute Difference ◽  
Future Research ◽  
Future Research Directions ◽  
The Mean ◽  
Perceived Desirability

The forced-choice method has been proposed as a viable strategy to prevent socially desirable responding (SDR) on self-report non-cognitive measures. The ability of the method to eliminate SDR may largely depend on how closely items comprising forced-choice item-blocks are matched in terms of perceived desirability. The gold standard in quantifying similarity between items in terms of desirability has been the mean difference index, that is, the absolute difference between items’ mean desirability ratings. The mean difference index relies on the assumption that items have one “true” desirability value, as represented by their means, and may fail if this assumption does not hold. Instead, we propose indexing within-rater agreement with several robust agreement indices to appropriately quantify similarity between items in terms of desirability (i.e., inter-item agreement). On a set of empirically derived desirability ratings, we show that relying on the mean difference index may lead to suboptimal forced-choice item assembly. Implications of our findings and future research directions are discussed. R code for computing the proposed indices on a set of desirability ratings is provided.

The Forced-Choice Test as a Structured Q-Sort

1968 ◽  
Vol 28 (4) ◽  
pp. 1103-1110 ◽  
Author(s):  
Leonard V. Gordon ◽  
Richard J. Hofmann
Keyword(s):  
Forced Choice ◽  
Choice Test ◽  
Q Sort

Validation of a detector of response bias on a forced-choice test of nonverbal ability.

1994 ◽  
Vol 8 (1) ◽  
pp. 118-125 ◽  
Author(s):  
Richard I. Frederick ◽  
Stephen D. Sarfaty ◽  
J. Dennis Johnston ◽  
Jeffrey Powel
Keyword(s):  
Response Bias ◽  
Forced Choice ◽  
Choice Test ◽  
Nonverbal Ability

Multiple measures of malingering on a forced-choice test of cognitive ability.

1991 ◽  
Vol 3 (4) ◽  
pp. 596-602 ◽  
Author(s):  
Richard I. Frederick ◽  
Hilliard G. Foster
Keyword(s):  
Cognitive Ability ◽  
Forced Choice ◽  
Choice Test ◽  
Multiple Measures

scholarly journals Familiarity Changes as a Function of Perceptual Shifts

2010 ◽  
Vol 13 (2) ◽  
pp. 518-524 ◽  
Author(s):  
Salvador Algarabel ◽  
Alfonso Pitarque
Keyword(s):  
Forced Choice ◽  
Choice Test ◽  
New Words ◽  
The Third ◽  
Study Test ◽  
Forced Choice Task ◽  
The Difference ◽  
The Right ◽  
Better Than ◽  
Additional Phase

This experiment compares the yes-no and forced recognition tests as methods of measuring familiarity. Participants faced a phase of 3 study-test recognition trials in which they studied words using all the letters of the alphabet (overlapping condition, O), and an additional phase in which targets and lures did not share any letters (non-overlapping condition, NO). Finally, subjects performed a forced-choice task in which they had to choose one of two new words, each from one of the subsets (Parkin et al., 2001). Results in the NO condition were better than in the O condition in the yes-no recognition test, while the forced-choice rate was significantly higher than .50, showing their sensitivity to familiarity. When the letter set of the words for study in the third list of the NO condition was switched, the difference between NO and O conditions disappeared in yes-no test, while the force-choice rate was not higher than .50. We conclude that both the yes-no test and the forced-choice test are valid and equivalent measures of familiarity under the right conditions.

Comparison of the triangle and a self-defined two alternative forced choice test

2010 ◽  
Vol 21 (5) ◽  
pp. 547-552 ◽  
Author(s):  
Scott McClure ◽  
Harry T. Lawless
Keyword(s):  
Forced Choice ◽  
Choice Test

scholarly journals The Fast-Mapping Abilities of Adults With Developmental Language Disorder

2020 ◽  
Vol 63 (9) ◽  
pp. 3117-3129
Author(s):  
Karla K. McGregor ◽  
Nichole Eden ◽  
Timothy Arbisi-Kelm ◽  
Jacob Oleson
Keyword(s):  
Language Disorder ◽  
Recognition Performance ◽  
Semantic Category ◽  
Forced Choice ◽  
Spoken Word ◽  
Choice Test ◽  
Developmental Language Disorder ◽  
Fast Mapping ◽  
Semantic Categories ◽  
Mapping Condition

Purpose The aim of the study was to determine the integrity of fast mapping among adults with developmental language disorder (DLD). Method Forty-eight adults with DLD or typical language development (TD) were presented with 24 novel words and photos of their unfamiliar referents from the semantic categories of mammal, bird, fruit, or insect in two conditions. In the fast-mapping condition, 12 of the 24 unfamiliar referents were presented, one at a time alongside a familiar referent (e.g., a dog) and a question (e.g., Is the tail of the torato up? ). In the explicit-encoding condition, the other 12 unfamiliar referents were presented alone, one at a time, with a label (e.g., This is a spimer ). Immediately after exposure (T1) and again after a 1-day interval (T2), memory for the word-to-exemplar link was measured with a three-alternative forced-choice test, requiring the participant to match a spoken word to one of three pictured referents from the training set. At T2, memory for semantic category information was measured with a four-alternative forced-choice test, requiring the participant to match a spoken word to one of four prototypical silhouettes representing each of the semantic categories. Results Performance on word-to-exemplar link recognition was stronger for words learned in the explicit-encoding than the fast-mapping condition and stronger for the TD group than the DLD group. Time was not a significant factor as both groups maintained posttraining levels of performance after a 1-day retention interval. Performance on semantic category recognition was stronger for words learned in the explicit-encoding than the fast-mapping condition and stronger for the TD group than the DLD group. The lower category recognition performance of the DLD group was related to their lower nonverbal IQ scores. Conclusion Contexts that allow for explicit encoding yield better learning of word-to-referent links than contexts that allow for fast mapping in both stronger and weaker learners. Adults with DLD have difficulty learning the link between words and referents, whether trained via fast mapping or explicit encoding and whether tested with exemplar or category referents. Retention is a relative strength for adults with DLD. Supplemental Material https://doi.org/10.23641/asha.12765551

Sign in / Sign up

Export Citation Format

Share Document