Incorporating randomness in the Fisher information for improving item-exposure control in CATs

2008 ◽  
Vol 61 (2) ◽  
pp. 493-513 ◽  
Author(s):  
Juan Ramón Barrada ◽  
Julio Olea ◽  
Vicente Ponsoda ◽  
Francisco José Abad
Keyword(s):  
Fisher Information ◽  
Exposure Control ◽  
Item Exposure ◽  
Item Exposure Control

Computerized Classification Testing Under the One-Parameter Logistic Response Model With Ability-Based Guessing

2011 ◽  
Vol 71 (6) ◽  
pp. 925-941 ◽  
Author(s):  
Wen-Chung Wang ◽  
Sheng-Yun Huang
Keyword(s):  
Fisher Information ◽  
Item Selection ◽  
Ratio Test ◽  
Exposure Rate ◽  
Exposure Control ◽  
Progressive Method ◽  
Selection Methods ◽  
Item Exposure ◽  
Item Exposure Control ◽  
The One

The one-parameter logistic model with ability-based guessing (1PL-AG) has been recently developed to account for effect of ability on guessing behavior in multiple-choice items. In this study, the authors developed algorithms for computerized classification testing under the 1PL-AG and conducted a series of simulations to evaluate their performances. Four item selection methods (the Fisher information, the Fisher information with a posterior distribution, the progressive method, and the adjusted progressive method) and two termination criteria (the ability confidence interval [ACI] method and the sequential probability ratio test [SPRT]) were developed. In addition, the Sympson–Hetter online method with freeze (SHOF) was implemented for item exposure control. Major results include the following: (a) when no item exposure control was made, all the four item selection methods yielded very similar correct classification rates, but the Fisher information method had the worst item bank usage and the highest item exposure rate; (b) SHOF can successfully maintain the item exposure rate at a prespecified level, without compromising substantial accuracy and efficiency in classification; (c) once SHOF was implemented, all the four methods performed almost identically; (d) ACI appeared to be slightly more efficient than SPRT; and (e) in general, a higher weight of ability in guessing led to a slightly higher accuracy and efficiency, and a lower forced classification rate.

scholarly journals Some Alternatives to Sympson-Hetter Item-Exposure Control in Computerized Adaptive Testing

2003 ◽  
Vol 28 (3) ◽  
pp. 249-265 ◽  
Author(s):  
Wim J. van der Linden
Keyword(s):  
Iterative Process ◽  
Computerized Adaptive Testing ◽  
Adaptive Testing ◽  
Law School ◽  
Exposure Control ◽  
Item Exposure ◽  
Item Exposure Control ◽  
Admission Test ◽  
Formal Properties ◽  
Ability Parameter

The Hetter and Sympson (1997 ; 1985 ) method is a method of probabilistic item-exposure control in computerized adaptive testing. Setting its control parameters to admissible values requires an iterative process of computer simulations that has been found to be time consuming, particularly if the parameters have to be set conditional on a realistic set of values for the examinees’ ability parameter. Formal properties of the method are identified that help us explain why this iterative process can be slow and does not guarantee admissibility. In addition, some alternatives to the SH method are introduced. The behavior of these alternatives was estimated for an adaptive test from an item pool from the Law School Admission Test (LSAT). Two of the alternatives showed attractive behavior and converged smoothly to admissibility for all items in a relatively small number of iteration steps.

scholarly journals A Dynamic Stratification Method for Improving Trait Estimation in Computerized Adaptive Testing Under Item Exposure Control

2019 ◽  
Vol 44 (3) ◽  
pp. 182-196
Author(s):  
Jyun-Hong Chen ◽  
Hsiu-Yi Chao ◽  
Shu-Ying Chen
Keyword(s):  
Computerized Adaptive Testing ◽  
Item Difficulty ◽  
Adaptive Testing ◽  
Item Selection ◽  
Exposure Control ◽  
Item Exposure ◽  
Stratification Method ◽  
High Discrimination ◽  
Item Exposure Control ◽  
Trait Estimation

When computerized adaptive testing (CAT) is under stringent item exposure control, the precision of trait estimation will substantially decrease. A new item selection method, the dynamic Stratification method based on Dominance Curves (SDC), which is aimed at improving trait estimation, is proposed to mitigate this problem. The objective function of the SDC in item selection is to maximize the sum of test information for all examinees rather than maximizing item information for individual examinees at a single-item administration, as in conventional CAT. To achieve this objective, the SDC uses dominance curves to stratify an item pool into strata with the number being equal to the test length to precisely and accurately increase the quality of the administered items as the test progresses, reducing the likelihood that a high-discrimination item will be administered to an examinee whose ability is not close to the item difficulty. Furthermore, the SDC incorporates a dynamic process for on-the-fly item–stratum adjustment to optimize the use of quality items. Simulation studies were conducted to investigate the performance of the SDC in CAT under item exposure control at different levels of severity. According to the results, the SDC can efficiently improve trait estimation in CAT through greater precision and more accurate trait estimation than those generated by other methods (e.g., the maximum Fisher information method) in most conditions.

scholarly journals Constraining Item Exposure in Computerized Adaptive Testing With Shadow Tests

2004 ◽  
Vol 29 (3) ◽  
pp. 273-291 ◽  
Author(s):  
Wim J van der Linden ◽  
Bernard P Veldkamp
Keyword(s):  
Computerized Adaptive Testing ◽  
Mean Squared Error ◽  
Adaptive Testing ◽  
Law School ◽  
Exposure Control ◽  
Require Time ◽  
Item Exposure ◽  
Squared Error ◽  
Item Exposure Control ◽  
Admission Test

Item-exposure control in computerized adaptive testing is implemented by imposing item-ineligibility constraints on the assembly process of the shadow tests. The method resembles Sympson and Hetter’s (1985) method of item-exposure control in that the decisions to impose the constraints are probabilistic. The method does not, however, require time-consuming simulation studies to set values for control parameters before the operational use of the test. Instead, it can set the probabilities of item ineligibility adaptively during the test using the actual item-exposure rates. An empirical study using an item pool from the Law School Admission Test showed that application of the method yielded perfect control of the item-exposure rates and had negligible impact on the bias and mean-squared error functions of the ability estimator.

Sign in / Sign up

Export Citation Format

Share Document