Evaluation of Response Probabilities along Studied Latent Dimensions: A Polytomous Item Extension

2021 ◽  
Vol 19 (3) ◽  
pp. 179-185
Author(s):  
Tenko Raykov ◽  
Chuck Huber ◽  
George A. Marcoulides ◽  
Martin Pusic ◽  
Natalja Menold
Keyword(s):  
Polytomous Item

Introduction to bifactor polytomous item response theory analysis

2017 ◽  
Vol 60 ◽  
pp. 41-63 ◽  
Author(s):  
Michael D. Toland ◽  
Isabella Sulis ◽  
Francesca Giambona ◽  
Mariano Porcu ◽  
Jonathan M. Campbell
Keyword(s):  
Item Response Theory ◽  
Item Response ◽  
Response Theory ◽  
Theory Analysis ◽  
Polytomous Item ◽  
Item Response Theory Analysis ◽  
Polytomous Item Response Theory ◽  
Polytomous Item Response

scholarly journals Using Differential Item Functioning to Test for Interrater Reliability in Constructed Response Items

2020 ◽  
Vol 80 (4) ◽  
pp. 808-820
Author(s):  
Cindy M. Walker ◽  
Sakine Göçer Şahin
Keyword(s):  
Differential Item Functioning ◽  
Interrater Reliability ◽  
Rating Scales ◽  
Rating Scale ◽  
Intraclass Correlation ◽  
Kappa Statistic ◽  
Promising Alternative ◽  
Constructed Response ◽  
Polytomous Item ◽  
Item Functioning

The purpose of this study was to investigate a new way of evaluating interrater reliability that can allow one to determine if two raters differ with respect to their rating on a polytomous rating scale or constructed response item. Specifically, differential item functioning (DIF) analyses were used to assess interrater reliability and compared with traditional interrater reliability measures. Three different procedures that can be used as measures of interrater reliability were compared: (1) intraclass correlation coefficient (ICC), (2) Cohen’s kappa statistic, and (3) DIF statistic obtained from Poly-SIBTEST. The results of this investigation indicated that DIF procedures appear to be a promising alternative to assess the interrater reliability of constructed response items, or other polytomous types of items, such as rating scales. Furthermore, using DIF to assess interrater reliability does not require a fully crossed design and allows one to determine if a rater is either more severe, or more lenient, in their scoring of each individual polytomous item on a test or rating scale.

Sign in / Sign up

Export Citation Format

Share Document