Reliably Assessing Growth with Longitudinal Diagnostic Classification Models

2019 ◽  
Vol 38 (2) ◽  
pp. 68-78
Author(s):  
Matthew J. Madison
Keyword(s):  
Diagnostic Classification ◽  
Classification Models ◽  
Diagnostic Classification Models

scholarly journals The Reliability of the Posterior Probability of Skill Attainment in Diagnostic Classification Models

2019 ◽  
Vol 45 (1) ◽  
pp. 5-31
Author(s):  
Matthew S. Johnson ◽  
Sandip Sinharay
Keyword(s):  
Posterior Probability ◽  
Real Data ◽  
Exact Calculation ◽  
Diagnostic Classification ◽  
Classification Models ◽  
Diagnostic Classification Models ◽  
Reliability Measure ◽  
Skill Mastery

One common score reported from diagnostic classification assessments is the vector of posterior means of the skill mastery indicators. As with any assessment, it is important to derive and report estimates of the reliability of the reported scores. After reviewing a reliability measure suggested by Templin and Bradshaw, this article suggests three new measures of reliability of the posterior means of skill mastery indicators and methods for estimating the measures when the number of items on the assessment and the number of skills being assessed render exact calculation computationally burdensome. The utility of the new measures is demonstrated using simulated and real data examples. Two of the suggested measures are recommended for future use.

scholarly journals Retrofitting Diagnostic Classification Models to Responses From IRT-Based Assessment Forms

2017 ◽  
Vol 78 (3) ◽  
pp. 357-383 ◽  
Author(s):  
Ren Liu ◽  
Anne Corinne Huggins-Manley ◽  
Okan Bulut
Keyword(s):  
Classical Test Theory ◽  
Test Theory ◽  
Diagnostic Classification ◽  
Classification Models ◽  
Diagnostic Classification Models ◽  
Measurement Framework ◽  
Ideal Approach ◽  
Item Responses ◽  
Optimal Approach ◽  
Diagnostic Measurement

Developing a diagnostic tool within the diagnostic measurement framework is the optimal approach to obtain multidimensional and classification-based feedback on examinees. However, end users may seek to obtain diagnostic feedback from existing item responses to assessments that have been designed under either the classical test theory or item response theory frameworks. Retrofitting diagnostic classification models to existing assessments designed under other psychometric frameworks could be a plausible approach to obtain more actionable scores or understand more about the constructs themselves. This study (a) discusses the possibility and problems of retrofitting, (b) proposes a step-by-step retrofitting framework, and (c) explores the information one can gain from retrofitting through an empirical application example. While retrofitting may not always be an ideal approach to diagnostic measurement, this article aims to invite discussions through presenting the possibility, challenges, process, and product of retrofitting.

Sign in / Sign up

Export Citation Format

Share Document