scholarly journals Observed score reliability indices in diagnostic classification models

2021 ◽  
Author(s):  
Kazuhiro Yamaguchi ◽  
Jonathan Templin
Keyword(s):  
Diagnostic Classification ◽  
Classification Models ◽  
Diagnostic Classification Models ◽  
Reliability Indices ◽  
Score Reliability ◽  
Observed Score

scholarly journals Observed Score Reliability Indices in Diagnostic Classification Models

2020 ◽  
Author(s):  
Kazuhiro Yamaguchi ◽  
Jonathan Templin
Keyword(s):  
Latent Variable ◽  
Empirical Test ◽  
Classical Test Theory ◽  
Test Theory ◽  
Diagnostic Classification ◽  
Classification Models ◽  
Diagnostic Classification Models ◽  
Reliability Indices ◽  
Score Reliability ◽  
Observed Score

Quantifying the reliability of latent variable estimates in diagnostic classification models has been a difficult topic, complicated by the classification-based nature of these models. In this study, we derive observed score reliability indices based on diagnostic classification models as an extension of classical test theory-based reliability. Additionally, we derive conditional observed sum- and sub-score distributions. In this manner, various conditional expectations and conditional standard error of measurement estimates can be calculated for both total- and sub-scores of a test. The proposed methods provide a variety of expectations and standard errors for attribute estimates, which we demonstrate in an analysis of an empirical test.

scholarly journals The Reliability of the Posterior Probability of Skill Attainment in Diagnostic Classification Models

2019 ◽  
Vol 45 (1) ◽  
pp. 5-31
Author(s):  
Matthew S. Johnson ◽  
Sandip Sinharay
Keyword(s):  
Posterior Probability ◽  
Real Data ◽  
Exact Calculation ◽  
Diagnostic Classification ◽  
Classification Models ◽  
Diagnostic Classification Models ◽  
Reliability Measure ◽  
Skill Mastery

One common score reported from diagnostic classification assessments is the vector of posterior means of the skill mastery indicators. As with any assessment, it is important to derive and report estimates of the reliability of the reported scores. After reviewing a reliability measure suggested by Templin and Bradshaw, this article suggests three new measures of reliability of the posterior means of skill mastery indicators and methods for estimating the measures when the number of items on the assessment and the number of skills being assessed render exact calculation computationally burdensome. The utility of the new measures is demonstrated using simulated and real data examples. Two of the suggested measures are recommended for future use.

Sign in / Sign up

Export Citation Format

Share Document