Cognitively Diagnostic Analysis Using the G-DINA Model in R

Cognitive diagnosis models (CDMs) have increasingly been applied in education and other fields. This article provides an overview of a widely used CDM, namely, the G-DINA model, and demonstrates a hands-on example of using multiple R packages for a series of CDM analyses. This overview involves a step-by-step illustration and explanation of performing Q-matrix evaluation, CDM calibration, model fit evaluation, item diagnosticity investigation, classification reliability examination, and the result presentation and visualization. Some limitations of conducting CDM analysis in R are also discussed.

A Deterministic Learning Algorithm Estimating the Q-Matrix for Cognitive Diagnosis Models

The goal of an exam in cognitive diagnostic assessment is to uncover whether an examinee has mastered certain attributes. Different cognitive diagnosis models (CDMs) have been developed for this purpose. The core of these CDMs is the Q-matrix, which is an item-to-attribute mapping, traditionally designed by domain experts. An expert designed Q-matrix is not without issues. For example, domain experts might neglect some attributes or have different opinions about the inclusion of some entries in the Q-matrix. It is therefore of practical importance to develop an automated method to estimate the Q-matrix. This research proposes a deterministic learning algorithm for estimating the Q-matrix. To obtain a sensible binary Q-matrix, a dichotomizing method is also devised. Results from the simulation study shows that the proposed method for estimating the Q-matrix is useful. The empirical study analyzes the ECPE data. The estimated Q-matrix is compared with the expert-designed one. All analyses in this research are carried out in R.

Item Response Ranking for Cognitive Diagnosis

Cognitive diagnosis, a fundamental task in education area, aims at providing an approach to reveal the proficiency level of students on knowledge concepts. Actually, monotonicity is one of the basic conditions in cognitive diagnosis theory, which assumes that student's proficiency is monotonic with the probability of giving the right response to a test item. However, few of previous methods consider the monotonicity during optimization. To this end, we propose Item Response Ranking framework (IRR), aiming at introducing pairwise learning into cognitive diagnosis to well model the monotonicity between item responses. Specifically, we first use an item specific sampling method to sample item responses and construct response pairs based on their partial order, where we propose the two-branch sampling methods to handle the unobserved responses. After that, we use a pairwise objective function to exploit the monotonicity in the pair formulation. In fact, IRR is a general framework which can be applied to most of contemporary cognitive diagnosis models. Extensive experiments demonstrate the effectiveness and interpretability of our method.

Towards a New Generation of Cognitive Diagnosis

Cognitive diagnosis is a type of assessment for automatically measuring individuals' proficiency profiles from their observed behaviors, e.g. quantifying the mastery level of examinees on specific knowledge concepts/skills. As one of the fundamental research tasks in domains like intelligent education, a number of Cognitive Diagnosis Models (CDMs) have been developed in the past decades. Though these solutions are usually well designed based on psychometric theories, they still suffer from the limited ability of the handcrafted diagnosis functions, especially when dealing with heterogeneous data. In this paper, I will share my personal understanding of cognitive diagnosis and review our recent developments of CDMs mostly from a machine learning perspective. Meanwhile, I will show the wide applications of cognitive diagnosis.

A Systematic Review of Cognitive Diagnostic Assessment and Modeling through Concept Mapping

Traditional assessments are typically constructed on logical taxonomies and content specifications but lack explicit cognitive models of the processes and problem-solving strategies that underlie student performance. Cognitive Diagnostic Assessment (CDA) fills this gap by combining cognitive science and psychometrics. CDA is in its infancy, but over 1,000 relevant studies have been conducted in this area during the last 20 years. Facing these complicated studies, many beginners struggle to understand the whole picture of CDA. This paper systematically reviews the literature on CDA and relevant cognitive diagnosis models (CDMs) with the application of a concept mapping technology. Concept mapping is graphical representation of concepts and their relationships. Its use in this study allows researchers and students to gain in-depth knowledge about CDA and CDM and identify areas of future research.