Item Selection and Ability Estimation Procedures for a Mixed-Format Adaptive Test

2012 ◽  
Vol 25 (4) ◽  
pp. 305-326
Author(s):  
Tsung-Han Ho ◽  
Barbara G. Dodd
Keyword(s):  
Item Selection ◽  
Ability Estimation ◽  
Adaptive Test ◽  
Estimation Procedures ◽  
Mixed Format

Using a Response Time–Based Expected A Posteriori Estimator to Control for Differential Speededness in Computerized Adaptive Test

2021 ◽  
pp. 014662162110146
Author(s):  
Justin L. Kern ◽  
Edison Choe
Keyword(s):  
Response Times ◽  
Real Data ◽  
Joint Estimation ◽  
Item Selection ◽  
A Posteriori ◽  
Computerized Adaptive Test ◽  
Maximum Information ◽  
Adaptive Test ◽  
Estimation And Control ◽  
And Control

This study investigates using response times (RTs) with item responses in a computerized adaptive test (CAT) setting to enhance item selection and ability estimation and control for differential speededness. Using van der Linden’s hierarchical framework, an extended procedure for joint estimation of ability and speed parameters for use in CAT is developed following van der Linden; this is called the joint expected a posteriori estimator (J-EAP). It is shown that the J-EAP estimate of ability and speededness outperforms the standard maximum likelihood estimator (MLE) of ability and speededness in terms of correlation, root mean square error, and bias. It is further shown that under the maximum information per time unit item selection method (MICT)—a method which uses estimates for ability and speededness directly—using the J-EAP further reduces average examinee time spent and variability in test times between examinees above the resulting gains of this selection algorithm with the MLE while maintaining estimation efficiency. Simulated test results are further corroborated with test parameters derived from a real data example.

Adaptive Weight Estimation of Latent Ability: Application to Computerized Adaptive Testing With Response Revision

2020 ◽  
pp. 107699862097280
Author(s):  
Shiyu Wang ◽  
Houping Xiao ◽  
Allan Cohen
Keyword(s):  
Large Scale ◽  
Computerized Adaptive Testing ◽  
Estimation Method ◽  
Adaptive Testing ◽  
Ability Estimation ◽  
Weight Estimation ◽  
Data Set ◽  
Adaptive Weight ◽  
Latent Ability ◽  
Estimation Procedures

An adaptive weight estimation approach is proposed to provide robust latent ability estimation in computerized adaptive testing (CAT) with response revision. This approach assigns different weights to each distinct response to the same item when response revision is allowed in CAT. Two types of weight estimation procedures, nonfunctional and functional weight, are proposed to determine the weight adaptively based on the compatibility of each revised response with the assumed statistical model in relation to remaining observations. The application of this estimation approach to a data set collected from a large-scale multistage adaptive testing demonstrates the capability of this method to reveal more information regarding the test taker’s latent ability by using the valid response path compared with only using the very last response. Limited simulation studies were concluded to evaluate the proposed ability estimation method and to compare it with several other estimation procedures in literature. Results indicate that the proposed ability estimation approach is able to provide robust estimation results in two test-taking scenarios.

scholarly journals Measuring language ability of students with compensatory multidimensional CAT: A post-hoc simulation study

2022 ◽  
Author(s):  
Burhanettin Ozdemir ◽  
Selahattin Gelbal
Keyword(s):  
Error Variance ◽  
Stopping Rule ◽  
Language Ability ◽  
Test Reliability ◽  
Item Selection ◽  
Ability Estimation ◽  
Proficiency Tests ◽  
Test Length ◽  
Post Hoc ◽  
Multidimensional Cat

AbstractThe computerized adaptive tests (CAT) apply an adaptive process in which the items are tailored to individuals' ability scores. The multidimensional CAT (MCAT) designs differ in terms of different item selection, ability estimation, and termination methods being used. This study aims at investigating the performance of the MCAT designs used to measure the language ability of students and to compare the results of MCAT designs with the outcomes of corresponding paper–pencil tests. For this purpose, items in the English Proficiency Tests (EPT) were used to create a multi-dimensional item pool that consists of 599 items. The performance of the MCAT designs was evaluated and compared based on the reliability coefficients, root means square error (RMSE), test-length, and root means squared difference (RMSD) statistics, respectively. Therefore, 36 different conditions were investigated in total. The results of the post-hoc simulation designs indicate that the MCAT designs with the A-optimality item selection method outperformed MCAT designs with other item selection methods by decreasing the test length and RMSD values without any sacrifice in test reliability. Additionally, the best error variance stopping rule for each MCAT algorithm with A-optimality item selection could be considered as 0.25 with 27.9 average test length and 30 items for the fixed test-length stopping rule for the Bayesian MAP method. Overall, MCAT designs tend to decrease the test length by 60 to 65 percent and provide ability estimations with higher precision compared to the traditional paper–pencil tests with 65 to 75 items. Therefore, it is suggested to use the A-optimality method for item selection and the Bayesian MAP method for ability estimation for the MCAT designs since the MCAT algorithm with these specifications shows better performance than others.

scholarly journals Adaptive Testing With a Hierarchical Item Response Theory Model

2018 ◽  
Vol 43 (1) ◽  
pp. 51-67 ◽  
Author(s):  
Wenhao Wang ◽  
Neal Kingston
Keyword(s):  
Item Response Theory ◽  
Item Response ◽  
Computation Time ◽  
Item Selection ◽  
Item Pool ◽  
Scoring Methods ◽  
Computational Burden ◽  
Irt Model ◽  
Adaptive Test ◽  
Simulated Sample

The hierarchical item response theory (H-IRT) model is very flexible and allows a general factor and subfactors within an overall structure of two or more levels. When an H-IRT model with a large number of dimensions is used for an adaptive test, the computational burden associated with interim scoring and selection of subsequent items is heavy. An alternative approach for any high-dimension adaptive test is to reduce dimensionality for interim scoring and item selection and then revert to full dimensionality for final score reporting, thereby significantly reducing the computational burden. This study compared the accuracy and efficiency of final scoring for multidimensional, local multidimensional, and unidimensional item selection and interim scoring methods, using both simulated and real item pools. The simulation study was conducted under 10 conditions (i.e., five test lengths and two H-IRT models) with a simulated sample of 10,000 students. The study with the real item pool was conducted using item parameters from an actual 45-item adaptive test with a simulated sample of 10,000 students. Results indicate that the theta estimations provided by the local multidimensional and unidimensional item selection and interim scoring methods were relatively as accurate as the theta estimation provided by the multidimensional item selection and interim scoring method, especially during the real item pool study. In addition, the multidimensional method required the longest computation time and the unidimensional method required the shortest computation time.

scholarly journals Item Selection on The Moodle-Based Computerized Adaptive Test

2021 ◽  
Vol 2111 (1) ◽  
pp. 012033
Author(s):  
Haryanto ◽  
Y Neng-Shu ◽  
S Hadi ◽  
M Ali ◽  
AF Husna ◽  
...  
Keyword(s):  
Object Oriented ◽  
Item Selection ◽  
Dynamic Learning ◽  
Adaptive Tests ◽  
Computerized Adaptive Test ◽  
Test Items ◽  
Adaptive Test ◽  
Online Tests ◽  
Test Process ◽  
Selection Of

Abstract In this industrial era, all areas of life have been entered. There are five central issues that support performance, namely numerical physical devices, production tools, programs, interfaces, and networks. The Modular Object-Oriented Dynamic Learning Environment (Moodle) in order to support Industrial technology has been equipped with adaptive test facilities. Adaptive Moodle can be used to organize a communicative and interactive test process because of the communication features (chat, messaging, or forum). In addition, Adaptive Moodle can be used to administer online tests. Adaptive tests are tests whose quiz presentations will adjust to the user’s abilities. The results of the research that has been carried out regarding the selection of items on the Moodle-based computerized adaptive test (CAT) were obtained: (1) The Moodle adaptive test worked successfully in accordance with the research objectives, (2) Based on user responses, the selection of items on the Moodle adaptive test items had worked well for the exam, (3) The Moodle adaptive test can run according to its function, namely adaptive to the user’s ability.

Sign in / Sign up

Export Citation Format

Share Document