A Comparison of Different Item Response Theory Models for Scaling Speeded C-Tests

As measures of general language proficiency, C-tests are ubiquitous in language testing. Speeded C-tests are quite recent developments in the field and are deemed to be more discriminatory and provide more accurate diagnostic information than power C-tests especially with high-ability participants. Item response theory modeling of speeded C-tests has not been discussed in the literature, and current approaches for power C-tests based on ordinal models either violate the model assumptions or are relatively complex to be reliably fitted with small samples. Count data models are viable alternatives with less restrictive assumptions and lower complexity. In the current study, we compare count data models with commonly applied ordinal models for modeling a speeded C-test. It was found that a flexible count data model fits equally well in absolute and relative terms as compared with ordinal models. Implications and feasibility of count data models for the psychometric modeling of C-tests are discussed.

Download Full-text

Recent Developments in Item Response Theory with Implications for Teacher Certification

Review of Research in Education ◽

10.2307/1167313 ◽

1987 ◽

Vol 14 ◽

pp. 239 ◽

Cited By ~ 1

Author(s):

Robert J. Mislevy

Keyword(s):

Item Response Theory ◽

Item Response ◽

Teacher Certification ◽

Response Theory ◽

Recent Developments

Download Full-text

A comparison of reliability and precision of subscore reporting methods for a state English language proficiency assessment

Language Testing ◽

10.1177/0265532217689949 ◽

2017 ◽

Vol 35 (2) ◽

pp. 297-317 ◽

Cited By ~ 3

Author(s):

Tanya Longabach ◽

Vicki Peyton

Keyword(s):

Item Response Theory ◽

Item Response ◽

Language Proficiency ◽

English Language ◽

English Language Proficiency ◽

Test Theory ◽

Policy Implications ◽

Proficiency Tests ◽

Response Theory ◽

K 12

K–12 English language proficiency tests that assess multiple content domains (e.g., listening, speaking, reading, writing) often have subsections based on these content domains; scores assigned to these subsections are commonly known as subscores. Testing programs face increasing customer demands for the reporting of subscores in addition to the total test scores in today’s accountability-oriented educational environment. Although reporting subscores can provide much-needed information for teachers, administrators, and students about proficiency in the test domains, one of the major drawbacks of subscore reporting includes their lower reliability as compared to the test as a whole. In addition, viewing language domains as if they were not interrelated, and reporting subscores without considering this relationship between domains, may be contradictory to the theory of language acquisition. This study explored several methods of assigning subscores to the four domains of a state English language proficiency test, including classical test theory (CTT)-based number correct, unidimensional item response theory (UIRT), augmented item response theory (A-IRT), and multidimensional item response theory (MIRT), and compared the reliability and precision of these different methods across language domains and grade bands. The first two methods assessed proficiency in the domains separately, without considering the relationship between domains; the last two methods took into consideration relationships between domains. The reliability and precision of the CTT and UIRT methods were similar and lower than those of A-IRT and MIRT for most domains and grade bands; MIRT was found to be the most reliable method. Policy implications and limitations of this study, as well as directions for further research, are discussed.

Download Full-text

The Science and Practice of Item Response Theory in Organizations

Annual Review of Organizational Psychology and Organizational Behavior ◽

10.1146/annurev-orgpsych-012420-061705 ◽

2020 ◽

Vol 8 (1) ◽

Author(s):

Jonas W.B. Lang ◽

Louis Tay

Keyword(s):

Item Response Theory ◽

Organizational Behavior ◽

Item Response ◽

Ideal Point ◽

Annual Review ◽

Publication Date ◽

Response Theory ◽

Test Items ◽

Starting Point ◽

Recent Developments

Item response theory (IRT) is a modeling approach that links responses to test items with underlying latent constructs through formalized statistical models. This article focuses on how IRT can be used to advance science and practice in organizations. We describe established applications of IRT as a scale development tool and new applications of IRT as a research and theory testing tool that enables organizational researchers to improve their understanding of workers and organizations. We focus on IRT models and their application in four key research and practice areas: testing, questionnaire responding, construct validation, and measurement equivalence of scores. In so doing, we highlight how novel developments in IRT such as explanatory IRT, multidimensional IRT, random item models, and more complex models of response processes such as ideal point models and tree models can potentially advance existing science and practice in these areas. As a starting point for readers interested in learning IRT and applying recent developments in IRT in their research, we provide concrete examples with data and R code. Expected final online publication date for the Annual Review of Organizational Psychology and Organizational Behavior, Volume 8 is January 21, 2021. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Download Full-text