scholarly journals An Extension of Testlet-Based Equating to the Polytomous Testlet Response Theory Model

2022 ◽  
Vol 12 ◽  
Author(s):  
Feifei Huang ◽  
Zhe Li ◽  
Ying Liu ◽  
Jingan Su ◽  
Li Yin ◽  
...  
Keyword(s):  
Item Response Theory ◽  
Sample Size ◽  
Item Response ◽  
Theory Model ◽  
Response Theory ◽  
Polytomous Items ◽  
Testlet Response Theory ◽  
Accurate Performance ◽  
The Impact ◽  
Item Response Theory Models

Educational assessments tests are often constructed using testlets because of the flexibility to test various aspects of the cognitive activities and broad content sampling. However, the violation of the local item independence assumption is inevitable when tests are built using testlet items. In this study, simulations are conducted to evaluate the performance of item response theory models and testlet response theory models for both the dichotomous and polytomous items in the context of equating tests composed of testlets. We also examine the impact of testlet effect, length of testlet items, and sample size on estimating item and person parameters. The results show that more accurate performance of testlet response theory models over item response theory models was consistently observed across the studies, which supports the benefits of using the testlet response theory models in equating for tests composed of testlets. Further, results of the study indicate that when sample size is large, item response theory models performed similarly to testlet response theory models across all studies.

scholarly journals Examination of Different Item Response Theory Models on Tests Composed of Testlets

2017 ◽  
Vol 6 (4) ◽  
pp. 113
Author(s):  
Esin Yilmaz Kogar ◽  
Hülya Kelecioglu
Keyword(s):  
Item Response Theory ◽  
Sample Size ◽  
Item Response ◽  
Data Sets ◽  
Meaningful Difference ◽  
Response Theory ◽  
Size Change ◽  
Data Set ◽  
Testlet Response Theory ◽  
Item Response Theory Models

The purpose of this research is to first estimate the item and ability parameters and the standard error values related to those parameters obtained from Unidimensional Item Response Theory (UIRT), bifactor (BIF) and Testlet Response Theory models (TRT) in the tests including testlets, when the number of testlets, number of independent items, and sample size change, and then to compare the obtained results. Mathematic test in PISA 2012 was employed as the data collection tool, and 36 items were used to constitute six different data sets containing different numbers of testlets and independent items. Subsequently, from these constituted data sets, three different sample sizes of 250, 500 and 1000 persons were selected randomly. When the findings of the research were examined, it was determined that, generally the lowest mean error values were those obtained from UIRT, and TRT yielded a mean of error estimation lower than that of BIF. It was found that, under all conditions, models which take into consideration the local dependency have provided a better model-data compatibility than UIRT, generally there is no meaningful difference between BIF and TRT, and both models can be used for those data sets. It can be said that when there is a meaningful difference between those two models, generally BIF yields a better result. In addition, it has been determined that, in each sample size and data set, item and ability parameters and correlations of errors of the parameters are generally high.

Towards power and sample size calculations for the comparison of two groups of patients with item response theory models

2011 ◽  
Vol 31 (11-12) ◽  
pp. 1277-1290 ◽  
Author(s):  
Jean-Benoit Hardouin ◽  
Sarah Amri ◽  
Mohand-Larbi Feddag ◽  
Véronique Sébille
Keyword(s):  
Item Response Theory ◽  
Sample Size ◽  
Item Response ◽  
Response Theory ◽  
Sample Size Calculations ◽  
Item Response Theory Models

scholarly journals What makes experts reliable? Expert reliability and the estimation of latent traits

2019 ◽  
Vol 6 (4) ◽  
pp. 205316801987956 ◽  
Author(s):  
Kyle L. Marquardt ◽  
Daniel Pemstein ◽  
Brigitte Seim ◽  
Yi-ting Wang
Keyword(s):  
Item Response Theory ◽  
Item Response ◽  
Theory Model ◽  
Contextual Knowledge ◽  
Response Theory ◽  
Factors Affecting ◽  
Reliability Estimates ◽  
Level Data ◽  
Latent Traits ◽  
Item Response Theory Models

Experts code latent quantities for many influential political science datasets. Although scholars are aware of the importance of accounting for variation in expert reliability when aggregating such data, they have not systematically explored either the factors affecting expert reliability or the degree to which these factors influence estimates of latent concepts. Here we provide a template for examining potential correlates of expert reliability, using coder-level data for six randomly selected variables from a cross-national panel dataset. We aggregate these data with an ordinal item response theory model that parameterizes expert reliability, and regress the resulting reliability estimates on both expert demographic characteristics and measures of their coding behavior. We find little evidence of a consistent substantial relationship between most expert characteristics and reliability, and these null results extend to potentially problematic sources of bias in estimates, such as gender. The exceptions to these results are intuitive, and provide baseline guidance for expert recruitment and retention in future expert coding projects: attentive and confident experts who have contextual knowledge tend to be more reliable. Taken as a whole, these findings reinforce arguments that item response theory models are a relatively safe method for aggregating expert-coded data.

Sign in / Sign up

Export Citation Format

Share Document