Development and Application of an Exploratory Reduced Reparameterized Unified Model

2018 ◽  
Vol 44 (1) ◽  
pp. 3-24 ◽  
Author(s):  
Steven Andrew Culpepper ◽  
Yinghan Chen
Keyword(s):  
Bayesian Methods ◽  
Unified Model ◽  
Cognitive Diagnosis ◽  
Simulation Studies ◽  
Binary Matrix ◽  
Data Set ◽  
Cognitive Diagnosis Models ◽  
Item Parameters ◽  
Attribute Hierarchy ◽  
Q Matrix

Exploratory cognitive diagnosis models (CDMs) estimate the Q matrix, which is a binary matrix that indicates the attributes needed for affirmative responses to each item. Estimation of Q is an important next step for improving classifications and broadening application of CDMs. Prior research primarily focused on an exploratory version of the restrictive deterministic-input, noisy-and-gate model, and research is needed to develop exploratory methods for more flexible CDMs. We consider Bayesian methods for estimating an exploratory version of the more flexible reduced reparameterized unified model (rRUM). We show that estimating the rRUM Q matrix is complicated by a confound between elements of Q and the rRUM item parameters. A Bayesian framework is presented that accurately recovers Q using a spike–slab prior for item parameters to select the required attributes for each item. We present Monte Carlo simulation studies, demonstrating the developed algorithm improves upon prior Bayesian methods for estimating the rRUM Q matrix. We apply the developed method to the Examination for the Certificate of Proficiency in English data set. The results provide evidence of five attributes with a partially ordered attribute hierarchy.

scholarly journals A Residual-Based Approach to Validate Q-Matrix Specifications

2017 ◽  
Vol 41 (4) ◽  
pp. 277-293 ◽  
Author(s):  
Jinsong Chen
Keyword(s):  
Real Life ◽  
Attribute Level ◽  
Simulation Studies ◽  
Cognitive Diagnosis Models ◽  
Modeling Process ◽  
Response Formats ◽  
Item Level ◽  
Q Matrix ◽  
Matrix Construction ◽  
Test Level

Q-matrix validation is of increasing concern due to the significance and subjective tendency of Q-matrix construction in the modeling process. This research proposes a residual-based approach to empirically validate Q-matrix specification based on a combination of fit measures. The approach separates Q-matrix validation into four logical steps, including the test-level evaluation, possible distinction between attribute-level and item-level misspecifications, identification of the hit item, and fit information to aid in item adjustment. Through simulation studies and real-life examples, it is shown that the misspecified items can be detected as the hit item and adjusted sequentially when the misspecification occurs at the item level or at random. Adjustment can be based on the maximum reduction of the test-level measures. When adjustment of individual items tends to be useless, attribute-level misspecification is of concern. The approach can accommodate a variety of cognitive diagnosis models (CDMs) and be extended to cover other response formats.

scholarly journals Improving Robustness in Q-Matrix Validation Using an Iterative and Dynamic Procedure

2020 ◽  
Vol 44 (6) ◽  
pp. 431-446 ◽  
Author(s):  
Pablo Nájera ◽  
Miguel A. Sorrel ◽  
Jimmy de la Torre ◽  
Francisco José Abad
Keyword(s):  
Parameter Estimates ◽  
Construction Process ◽  
Data Set ◽  
Cognitive Diagnosis Models ◽  
Item Level ◽  
Attribute Classification ◽  
Q Matrix ◽  
Latent Group ◽  
The Times ◽  
Matrix Construction

In the context of cognitive diagnosis models (CDMs), a Q-matrix reflects the correspondence between attributes and items. The Q-matrix construction process is typically subjective in nature, which may lead to misspecifications. All this can negatively affect the attribute classification accuracy. In response, several methods of empirical Q-matrix validation have been developed. The general discrimination index (GDI) method has some relevant advantages such as the possibility of being applied to several CDMs. However, the estimation of the GDI relies on the estimation of the latent group sizes and success probabilities, which is made with the original (possibly misspecified) Q-matrix. This can be a problem, especially in those situations in which there is a great uncertainty about the Q-matrix specification. To address this, the present study investigates the iterative application of the GDI method, where only one item is modified at each step of the iterative procedure, and the required cutoff is updated considering the new parameter estimates. A simulation study was conducted to test the performance of the new procedure. Results showed that the performance of the GDI method improved when the application was iterative at the item level and an appropriate cutoff point was used. This was most notable when the original Q-matrix misspecification rate was high, where the proposed procedure performed better 96.5% of the times. The results are illustrated using Tatsuoka’s fraction-subtraction data set.

scholarly journals Using JAGS for Bayesian Cognitive Diagnosis Modeling: A Tutorial

2019 ◽  
Vol 44 (4) ◽  
pp. 473-503 ◽  
Author(s):  
Peida Zhan ◽  
Hong Jiao ◽  
Kaiwen Man ◽  
Lijun Wang
Keyword(s):  
Gibbs Sampler ◽  
Logistic Model ◽  
Structural Model ◽  
Higher Order ◽  
Unified Model ◽  
Cognitive Diagnosis ◽  
Cognitive Diagnosis Models ◽  
Log Linear ◽  
Linear Logistic Model ◽  
Or Gate

In this article, we systematically introduce the just another Gibbs sampler (JAGS) software program to fit common Bayesian cognitive diagnosis models (CDMs) including the deterministic inputs, noisy “and” gate model; the deterministic inputs, noisy “or” gate model; the linear logistic model; the reduced reparameterized unified model; and the log-linear CDM (LCDM). Further, we introduce the unstructured latent structural model and the higher order latent structural model. We also show how to extend these models to consider polytomous attributes, the testlet effect, and longitudinal diagnosis. Finally, we present an empirical example as a tutorial to illustrate how to use JAGS codes in R.

scholarly journals Regularized Estimation of the Four-Parameter Logistic Model

Psych ◽  
2020 ◽  
Vol 2 (4) ◽  
pp. 269-278
Author(s):  
Michela Battauz
Keyword(s):  
Logistic Model ◽  
Likelihood Function ◽  
Latent Trait ◽  
Real Data ◽  
Theory Model ◽  
Simulation Studies ◽  
Data Set ◽  
Penalty Term ◽  
Regularized Estimation ◽  
Item Parameters

The four-parameter logistic model is an Item Response Theory model for dichotomous items that limit the probability of giving a positive response to an item into a restricted range, so that even people at the extremes of a latent trait do not have a probability close to zero or one. Despite the literature acknowledging the usefulness of this model in certain contexts, the difficulty of estimating the item parameters has limited its use in practice. In this paper we propose a regularized estimation approach for the estimation of the item parameters based on the inclusion of a penalty term in the log-likelihood function. Simulation studies show the good performance of the proposal, which is further illustrated through an application to a real-data set.

scholarly journals Reconsidering Cutoff Points in the General Method of Empirical Q-Matrix Validation

2019 ◽  
Vol 79 (4) ◽  
pp. 727-753 ◽  
Author(s):  
Pablo Nájera ◽  
Miguel A. Sorrel ◽  
Francisco José Abad
Keyword(s):  
Latent Variables ◽  
Latent Class ◽  
Simulation Studies ◽  
Construction Process ◽  
Domain Experts ◽  
Cognitive Diagnosis Models ◽  
Overall Performance ◽  
Q Matrix ◽  
Matrix Construction ◽  
General Method

Cognitive diagnosis models (CDMs) are latent class multidimensional statistical models that help classify people accurately by using a set of discrete latent variables, commonly referred to as attributes. These models require a Q-matrix that indicates the attributes involved in each item. A potential problem is that the Q-matrix construction process, typically performed by domain experts, is subjective in nature. This might lead to the existence of Q-matrix misspecifications that can lead to inaccurate classifications. For this reason, several empirical Q-matrix validation methods have been developed in the recent years. de la Torre and Chiu proposed one of the most popular methods, based on a discrimination index. However, some questions related to the usefulness of the method with empirical data remained open due the restricted number of conditions examined, and the use of a unique cutoff point ( EPS) regardless of the data conditions. This article includes two simulation studies to test this validation method under a wider range of conditions, with the purpose of providing it with a higher generalization, and to empirically determine the most suitable EPS considering the data conditions. Results show a good overall performance of the method, the relevance of the different studied factors, and that using a single indiscriminate EPS is not acceptable. Specific guidelines for selecting an appropriate EPS are provided in the discussion.

A Class of Cognitive Diagnosis Models for Polytomous Data

2020 ◽  
pp. 107699862095198
Author(s):  
Xuliang Gao ◽  
Wenchao Ma ◽  
Daxun Wang ◽  
Yan Cai ◽  
Dongbo Tu
Keyword(s):  
Real Data ◽  
Information Criterion ◽  
Information Criteria ◽  
Cognitive Diagnosis ◽  
Simulation Studies ◽  
Akaike's Information Criterion ◽  
Akaike’S Information Criterion ◽  
Cognitive Diagnosis Models ◽  
Link Functions ◽  
Special Cases

This article proposes a class of cognitive diagnosis models (CDMs) for polytomously scored items with different link functions. Many existing polytomous CDMs can be considered as special cases of the proposed class of polytomous CDMs. Simulation studies were carried out to investigate the feasibility of the proposed CDMs and the performance of several information criteria (Akaike’s information criterion [AIC], consistent Akaike’s information criterion [CAIC], and Bayesian information criterion [BIC]) in model selection. The results showed that the parameters of the proposed CDMs could be recovered adequately under varied conditions. In addition, CAIC and BIC had better performance in selecting the most appropriate model than AIC. Finally, a set of real data was analyzed to illustrate the application of the proposed CDMs.

scholarly journals A Deterministic Learning Algorithm Estimating the Q-Matrix for Cognitive Diagnosis Models

Mathematics ◽  
2021 ◽  
Vol 9 (23) ◽  
pp. 3062
Author(s):  
Meng-Ta Chung ◽  
Shui-Lien Chen
Keyword(s):  
Learning Algorithm ◽  
Practical Importance ◽  
Cognitive Diagnosis ◽  
Diagnostic Assessment ◽  
Deterministic Learning ◽  
Domain Experts ◽  
The Core ◽  
Cognitive Diagnosis Models ◽  
Automated Method ◽  
Q Matrix

The goal of an exam in cognitive diagnostic assessment is to uncover whether an examinee has mastered certain attributes. Different cognitive diagnosis models (CDMs) have been developed for this purpose. The core of these CDMs is the Q-matrix, which is an item-to-attribute mapping, traditionally designed by domain experts. An expert designed Q-matrix is not without issues. For example, domain experts might neglect some attributes or have different opinions about the inclusion of some entries in the Q-matrix. It is therefore of practical importance to develop an automated method to estimate the Q-matrix. This research proposes a deterministic learning algorithm for estimating the Q-matrix. To obtain a sensible binary Q-matrix, a dichotomizing method is also devised. Results from the simulation study shows that the proposed method for estimating the Q-matrix is useful. The empirical study analyzes the ECPE data. The estimated Q-matrix is compared with the expert-designed one. All analyses in this research are carried out in R.

scholarly journals Cognitively Diagnostic Analysis Using the G-DINA Model in R

Psych ◽  
2021 ◽  
Vol 3 (4) ◽  
pp. 812-835
Author(s):  
Qingzhou Shi ◽  
Wenchao Ma ◽  
Alexander Robitzsch ◽  
Miguel A. Sorrel ◽  
Kaiwen Man
Keyword(s):  
Cognitive Diagnosis ◽  
Model Fit ◽  
Calibration Model ◽  
Cognitive Diagnosis Models ◽  
Dina Model ◽  
Hands On ◽  
R Packages ◽  
Q Matrix ◽  
Cognitively Diagnostic ◽  
Classification Reliability

Cognitive diagnosis models (CDMs) have increasingly been applied in education and other fields. This article provides an overview of a widely used CDM, namely, the G-DINA model, and demonstrates a hands-on example of using multiple R packages for a series of CDM analyses. This overview involves a step-by-step illustration and explanation of performing Q-matrix evaluation, CDM calibration, model fit evaluation, item diagnosticity investigation, classification reliability examination, and the result presentation and visualization. Some limitations of conducting CDM analysis in R are also discussed.

Sign in / Sign up

Export Citation Format

Share Document