scholarly journals Multilevel Modeling of Cognitive Diagnostic Assessment: The Multilevel DINA Example

2018 ◽  
Vol 43 (1) ◽  
pp. 34-50 ◽  
Author(s):  
Wen-Chung Wang ◽  
Xue-Lan Qiu
Keyword(s):  
Continuous Variable ◽  
Parameter Recovery ◽  
Dina Model ◽  
Variable Approach ◽  
Item Level ◽  
Multivariate Bernoulli ◽  
Level Model ◽  
Log Linear ◽  
Latent Attribute ◽  
Mathematics And Science

Many multilevel linear and item response theory models have been developed to account for multilevel data structures. However, most existing cognitive diagnostic models (CDMs) are unilevel in nature and become inapplicable when data have a multilevel structure. In this study, using the log-linear CDM as the item-level model, multilevel CDMs were developed based on the latent continuous variable approach and the multivariate Bernoulli distribution approach. In a series of simulations, the newly developed multilevel deterministic input, noisy, and gate (DINA) model was used as an example to evaluate the parameter recovery and consequences of ignoring the multilevel structures. The results indicated that all parameters in the new multilevel DINA were recovered fairly well by using the freeware Just Another Gibbs Sampler (JAGS) and that ignoring multilevel structures by fitting the standard unilevel DINA model resulted in poor estimates for the student-level covariates and underestimated standard errors, as well as led to poor recovery for the latent attribute profiles for individuals. An empirical example using the 2003 Trends in International Mathematics and Science Study eighth-grade mathematical test was provided.

scholarly journals Test-level and Item-level Model Fit Comparison of General vs. Specific Diagnostic Classification Models: A Case of True DCM

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Mahdieh Shafipoor ◽  
Hamdollah Ravand ◽  
Parviz Maftoon
Keyword(s):  
Model Fit ◽  
Diagnostic Classification ◽  
Fit Indices ◽  
Classification Models ◽  
Diagnostic Classification Models ◽  
Item Level ◽  
Model Fit Indices ◽  
Skill Mastery ◽  
Level Model ◽  
Test Level

AbstractThe current study compared the model fit indices, skill mastery probabilities, and classification accuracy of six Diagnostic Classification Models (DCMs): a general model (G-DINA) against five specific models (LLM, RRUM, ACDM, DINA, and DINO). To do so, the response data to the grammar and vocabulary sections of a General English Achievement Test, designed specifically for cognitive diagnostic purposes from scratch, was analyzed. The results of the test-level-model fit values obtained strong evidence in supporting the G-DINA and LLM models possessing the best model fit. In addition, the ACDM and RRUM were almost very identical to that of the G-DINA. The value indices of the DINO and DINA models were very close to each other but larger than those of the G-DINA and LLM. The model fit was also investigated at the item level, and the results revealed that model selection should be performed at the item level rather than the test level, and most of the specific models might perform well for the test. The findings of this study suggested that the relationships among the attributes of grammar and vocabulary are not ‘either-or’ compensatory or non-compensatory but a combination of both.

scholarly journals A Response-Time-Based Latent Response Mixture Model for Identifying and Modeling Careless and Insufficient Effort Responding in Survey Data

Psychometrika ◽  
2021 ◽  
Author(s):  
Esther Ulitzsch ◽  
Steffi Pohl ◽  
Lale Khorramdel ◽  
Ulf Kroehne ◽  
Matthias von Davier
Keyword(s):  
Response Time ◽  
Response Patterns ◽  
Response Behavior ◽  
Data Set ◽  
Parameter Recovery ◽  
Model Based ◽  
Proposed Model ◽  
Item Level ◽  
Different Types ◽  
Theoretical Considerations

AbstractCareless and insufficient effort responding (C/IER) can pose a major threat to data quality and, as such, to validity of inferences drawn from questionnaire data. A rich body of methods aiming at its detection has been developed. Most of these methods can detect only specific types of C/IER patterns. However, typically different types of C/IER patterns occur within one data set and need to be accounted for. We present a model-based approach for detecting manifold manifestations of C/IER at once. This is achieved by leveraging response time (RT) information available from computer-administered questionnaires and integrating theoretical considerations on C/IER with recent psychometric modeling approaches. The approach a) takes the specifics of attentive response behavior on questionnaires into account by incorporating the distance–difficulty hypothesis, b) allows for attentiveness to vary on the screen-by-respondent level, c) allows for respondents with different trait and speed levels to differ in their attentiveness, and d) at once deals with various response patterns arising from C/IER. The approach makes use of item-level RTs. An adapted version for aggregated RTs is presented that supports screening for C/IER behavior on the respondent level. Parameter recovery is investigated in a simulation study. The approach is illustrated in an empirical example, comparing different RT measures and contrasting the proposed model-based procedure against indicator-based multiple-hurdle approaches.

scholarly journals Cognitive Diagnosis Modeling Incorporating Item-Level Missing Data Mechanism

2020 ◽  
Vol 11 ◽  
Author(s):  
Na Shan ◽  
Xiaofei Wang
Keyword(s):  
Missing Data ◽  
International Student ◽  
Student Assessment ◽  
Cognitive Diagnosis ◽  
Parameter Recovery ◽  
International Student Assessment ◽  
Item Level ◽  
Item Responses ◽  
Computer Based ◽  
Cognitive Diagnosis Modeling

The aim of cognitive diagnosis is to classify respondents' mastery status of latent attributes from their responses on multiple items. Since respondents may answer some but not all items, item-level missing data often occur. Even if the primary interest is to provide diagnostic classification of respondents, misspecification of missing data mechanism may lead to biased conclusions. This paper proposes a joint cognitive diagnosis modeling of item responses and item-level missing data mechanism. A Bayesian Markov chain Monte Carlo (MCMC) method is developed for model parameter estimation. Our simulation studies examine the parameter recovery under different missing data mechanisms. The parameters could be recovered well with correct use of missing data mechanism for model fit, and missing that is not at random is less sensitive to incorrect use. The Program for International Student Assessment (PISA) 2015 computer-based mathematics data are applied to demonstrate the practical value of the proposed method.

scholarly journals An Item Response Theory–Informed Strategy to Model Total Score Data from Composite Scales

2021 ◽  
Vol 23 (3) ◽  
Author(s):  
Gustaf J. Wellhagen ◽  
Sebastian Ueckert ◽  
Maria C. Kjellsson ◽  
Mats O. Karlsson
Keyword(s):  
Item Response Theory ◽  
Item Response ◽  
Latent Variable ◽  
Continuous Variable ◽  
Response Theory ◽  
Irt Models ◽  
Composite Scale ◽  
Level Data ◽  
Item Level ◽  
Scale Data

AbstractComposite scale data is widely used in many therapeutic areas and consists of several categorical questions/items that are usually summarized into a total score (TS). Such data is discrete and bounded by nature. The gold standard to analyse composite scale data is item response theory (IRT) models. However, IRT models require item-level data while sometimes only TS is available. This work investigates models for TS. When an IRT model exists, it can be used to derive the information as well as expected mean and variability of TS at any point, which can inform TS-analyses. We propose a new method: IRT-informed functions of expected values and standard deviation in TS-analyses. The most common models for TS-analyses are continuous variable (CV) models, while bounded integer (BI) models offer an alternative that respects scale boundaries and the nature of TS data. We investigate the method in CV and BI models on both simulated and real data. Both CV and BI models were improved in fit by IRT-informed disease progression, which allows modellers to precisely and accurately find the corresponding latent variable parameters, and IRT-informed SD, which allows deviations from homoscedasticity. The methodology provides a formal way to link IRT models and TS models, and to compare the relative information of different model types. Also, joint analyses of item-level data and TS data are made possible. Thus, IRT-informed functions can facilitate total score analysis and allow a quantitative analysis of relative merits of different analysis methods.

scholarly journals Improved Wald Statistics for Item-Level Model Comparison in Diagnostic Classification Models

2018 ◽  
Vol 43 (5) ◽  
pp. 402-414 ◽  
Author(s):  
Yanlou Liu ◽  
Björn Andersson ◽  
Tao Xin ◽  
Haiyan Zhang ◽  
Lingling Wang
Keyword(s):  
Model Comparison ◽  
Diagnostic Classification ◽  
Classification Models ◽  
Diagnostic Classification Models ◽  
Item Level ◽  
Level Model

scholarly journals Kelly Betting with Quantum Payoff: a continuous variable approach

Quantum ◽  
2021 ◽  
Vol 5 ◽  
pp. 545
Author(s):  
Salvatore Tirone ◽  
Maddalena Ghio ◽  
Giulia Livieri ◽  
Vittorio Giovannetti ◽  
Stefano Marmi
Keyword(s):  
Degrees Of Freedom ◽  
Classical Model ◽  
Single Mode ◽  
Continuous Variable ◽  
Quantum Memory ◽  
Memory Element ◽  
Gaussian Channels ◽  
Variable Approach ◽  
Gaussian Density ◽  
Quantum Analog

The main purpose of this study is to introduce a semi-classical model describing betting scenarios in which, at variance with conventional approaches, the payoff of the gambler is encoded into the internal degrees of freedom of a quantum memory element. In our scheme, we assume that the invested capital is explicitly associated with the quantum analog of the free-energy (i.e. ergotropy functional by Allahverdyan, Balian, and Nieuwenhuizen) of a single mode of the electromagnetic radiation which, depending on the outcome of the betting, experiences attenuation or amplification processes which model losses and winning events. The resulting stochastic evolution of the quantum memory resembles the dynamics of random lasing which we characterize within the theoretical setting of Bosonic Gaussian channels. As in the classical Kelly Criterion for optimal betting, we define the asymptotic doubling rate of the model and identify the optimal gambling strategy for fixed odds and probabilities of winning. The performance of the model are hence studied as a function of the input capital state under the assumption that the latter belongs to the set of Gaussian density matrices (i.e. displaced, squeezed thermal Gibbs states) revealing that the best option for the gambler is to devote all their initial resources into coherent state amplitude.

scholarly journals Comparison of Precision and Accuracy of Five Methods to Analyse Total Score Data

2020 ◽  
Vol 23 (1) ◽  
Author(s):  
Gustaf J. Wellhagen ◽  
Mats O. Karlsson ◽  
Maria C. Kjellsson
Keyword(s):  
Repeated Measures ◽  
Rating Scale ◽  
Continuous Variable ◽  
Parameter Estimates ◽  
Irt Model ◽  
External Data ◽  
Irt Models ◽  
Level Data ◽  
Item Level ◽  
Disease Rating

AbstractTotal score (TS) data is generated from composite scales consisting of several questions/items, such as the Movement Disorder Society-Unified Parkinson’s Disease Rating Scale (MDS-UPDRS). The analysis method that most fully uses the information gathered is item response theory (IRT) models, but these are complex and require item-level data which may not be available. Therefore, the TS is commonly analysed with standard continuous variable (CV) models, which do not respect the bounded nature of data. Bounded integer (BI) models do respect the data nature but are not as extensively researched. Mixed models for repeated measures (MMRM) are an alternative that requires few assumptions and handles dropout without bias. If an IRT model exists, the expected mean and standard deviation of TS can be computed through IRT-informed functions—which allows CV and BI models to estimate parameters on the IRT scale. The fit, performance on external data and parameter precision (when applicable) of CV, BI and MMRM to analyse simulated TS data from the MDS-UPDRS motor subscale are investigated in this work. All models provided accurate predictions and residuals without trends, but the fit of CV and BI models was improved by IRT-informed functions. The IRT-informed BI model had more precise parameter estimates than the IRT-informed CV model. The IRT-informed models also had the best performance on external data, while the MMRM model was worst. In conclusion, (1) IRT-informed functions improve TS analyses and (2) IRT-informed BI models had more precise IRT parameter estimates than IRT-informed CV models.

scholarly journals Assessing Item-Level Fit for the DINA Model

2015 ◽  
Vol 39 (7) ◽  
pp. 525-538 ◽  
Author(s):  
Chun Wang ◽  
Zhan Shu ◽  
Zhuoran Shang ◽  
Gongjun Xu
Keyword(s):  
Dina Model ◽  
Item Level
Sign in / Sign up

Export Citation Format

Share Document