Investigating the Viability of Alternating Model Tree As An Item Selection Algorithm for Constructing Computerized Adaptive Psychological Testing

2021 ◽  
Vol 40 (4) ◽  
pp. 539-566
Author(s):  
Jeong-Han Youn ◽  
Taehun Lee
Author(s):  
Kyung (Chris) Tyek Han

Computerized adaptive testing (CAT) greatly improves measurement efficiency in high-stakes testing operations through the selection and administration of test items with the difficulty level that is most relevant to each individual test taker. This paper explains the 3 components of a conventional CAT item selection algorithm: test content balancing, the item selection criterion, and item exposure control. Several noteworthy methodologies underlie each component. The test script method and constrained CAT method are used for test content balancing. Item selection criteria include the maximized Fisher information criterion, the b-matching method, the astratification method, the weighted likelihood information criterion, the efficiency balanced information criterion, and the KullbackLeibler information criterion. The randomesque method, the Sympson-Hetter method, the unconditional and conditional multinomial methods, and the fade-away method are used for item exposure control. Several holistic approaches to CAT use automated test assembly methods, such as the shadow test approach and the weighted deviation model. Item usage and exposure count vary depending on the item selection criterion and exposure control method. Finally, other important factors to consider when determining an appropriate CAT design are the computer resources requirement, the size of item pools, and the test length. The logic of CAT is now being adopted in the field of adaptive learning, which integrates the learning aspect and the (formative) assessment aspect of education into a continuous, individualized learning experience. Therefore, the algorithms and technologies described in this review may be able to help medical health educators and high-stakes test developers to adopt CAT more actively and efficiently.


1972 ◽  
Vol 41 (2) ◽  
pp. 87-91
Author(s):  
M. W. Wahlstrom ◽  
S. M. Hunka

2017 ◽  
Vol 33 (6) ◽  
pp. 409-421 ◽  
Author(s):  
Anne B. Janssen ◽  
Martin Schultze ◽  
Adrian Grötsch

Abstract. Employees’ innovative work is a facet of proactive work behavior that is of increasing interest to industrial and organizational psychologists. As proactive personality and supervisor support are key predictors of innovative work behavior, reliable, and valid employee ratings of these two constructs are crucial for organizations’ planning of personnel development measures. However, the time for assessments is often limited. The present study therefore aimed at constructing reliable short scales of two measures of proactive personality and supervisor support. For this purpose, we compared an innovative approach of item selection, namely Ant Colony Optimization (ACO; Leite, Huang, & Marcoulides, 2008 ) and classical item selection procedures. For proactive personality, the two item selection approaches provided similar results. Both five-item short forms showed a satisfactory reliability and a small, however negligible loss of criterion validity. For a two-dimensional supervisor support scale, ACO found a reliable and valid short form. Psychometric properties of the short version were in accordance with those of the parent form. A manual supervisor support short form revealed a rather poor model fit and a serious loss of validity. We discuss benefits and shortcomings of ACO compared to classical item selection approaches and recommendations for the application of ACO.


Methodology ◽  
2018 ◽  
Vol 14 (4) ◽  
pp. 177-188 ◽  
Author(s):  
Martin Schultze ◽  
Michael Eid

Abstract. In the construction of scales intended for the use in cross-cultural studies, the selection of items needs to be guided not only by traditional criteria of item quality, but has to take information about the measurement invariance of the scale into account. We present an approach to automated item selection which depicts the process as a combinatorial optimization problem and aims at finding a scale which fulfils predefined target criteria – such as measurement invariance across cultures. The search for an optimal solution is performed using an adaptation of the [Formula: see text] Ant System algorithm. The approach is illustrated using an application to item selection for a personality scale assuming measurement invariance across multiple countries.


1997 ◽  
Vol 42 (8) ◽  
pp. 748-749
Author(s):  
Russell D. Allison ◽  
Rodney K. Goodyear

2003 ◽  
Vol 48 (4) ◽  
pp. 410-412
Author(s):  
F. Barton Evans

1996 ◽  
Vol 41 (11) ◽  
pp. 1147-1147
Author(s):  
Sharon Sackett

PsycCRITIQUES ◽  
2005 ◽  
Vol 50 (49) ◽  
Author(s):  
Todd L. Chmielewski

Sign in / Sign up

Export Citation Format

Share Document