Reporting Subscore Profiles Using Diagnostic Classification Models in Health Professions Education

2019 ◽  
Vol 43 (3) ◽  
pp. 149-158
Author(s):  
Yoon Soo Park ◽  
Amy Morales ◽  
Linette Ross ◽  
Miguel Paniagua
Keyword(s):  
Large Scale ◽  
Latent Class ◽  
Health Professions ◽  
Health Professions Education ◽  
Diagnostic Classification ◽  
Parameter Estimates ◽  
Classification Models ◽  
Diagnostic Classification Models ◽  
Fine Grained

Learners and educators in the health professions have called for more fine-grained information (subscores) from assessments, beyond a single overall test score. However, due to concerns over reliability, there have been limited uses of subscores in practice. Recent advances in latent class analysis have made contributions in subscore reporting by using diagnostic classification models (DCMs), which allow reliable classification of examinees into fine-grained proficiency levels (subscore profiles). This study examines the innovative and practical application of DCM framework to health professions educational assessments using retrospective large-scale assessment data from the basic and clinical sciences: National Board of Medical Examiners Subject Examinations in pathology ( n = 2,006) and medicine ( n = 2,351). DCMs were fit and analyzed to generate subscores and subscore profiles of examinees. Model fit indices, classification (reliability), and parameter estimates indicated that DCMs had good psychometric properties including consistent classification of examinees into subscore profiles. Results showed a range of useful information including varying levels of subscore distributions. The DCM framework can be a promising approach to report subscores in health professions education. Consistency of classification was high, demonstrating reliable results at fine-grained subscore levels, allowing for targeted and specific feedback to learners.

scholarly journals A Variational Bayesian Inference Posterior Approximation Algorithm for Hidden Markov Diagnostic Classification Models

2021 ◽  
Author(s):  
Kazuhiro Yamaguchi ◽  
Alfonso J. Martinez
Keyword(s):  
Hidden Markov ◽  
Cognitive Learning ◽  
Real Data ◽  
Diagnostic Classification ◽  
Mcmc Methods ◽  
Parameter Estimates ◽  
Classification Models ◽  
Diagnostic Classification Models ◽  
Variational Bayesian ◽  
Posterior Approximation

General diagnostic classification models (DCMs) can be used to capture individual students’ cognitive learning status. Moreover, DCMs for longitudinal data are appropriate to track students transition of cognitive elements. This study developed an effective Bayesian posterior approximation method called variational Bayesian (VB) inference method for hidden Markov type longitudinal general DCMs. Simulation study indicated the proposed algorithm could satisfactorily recover true parameters. Comparative study of the VB and previously developed Markov chain Monte Carlo (MCMC) methods was conducted in real data example. The result revealed that the VB method provided similar parameter estimates to the MCMC with faster estimation time.

scholarly journals Evaluating model estimation processes for diagnostic classification models

2020 ◽  
Author(s):  
W. Jake Thompson
Keyword(s):  
Model Reduction ◽  
Latent Variables ◽  
Reduction Process ◽  
Diagnostic Classification ◽  
Parameter Estimates ◽  
Model Estimation ◽  
Classification Models ◽  
Data Set ◽  
Diagnostic Classification Models ◽  
Reduction Processes

Diagnostic classification models (DCMs) are a class of models that define respondent ability on a set of predefined categorical latent variables. In recent years, the popularity of these models has begun to increase. As the community of researchers of practitioners of DCMs grow, it is important to examine the implementation of these models, including the process of model estimation. A key aspect of the estimation process that remains unexplored in the DCM literature is model reduction, or the removal of parameters from the model in order to create a simpler, more parsimonious model. The current study fills this gap in the literature by first applying several model reduction processes on a real data set, the Diagnosing Teachers’ Multiplicative Reasoning assessment (Bradshaw et al., 2014). Results from this analysis indicate that the selection of model reduction process can have large implications for the resulting parameter estimates and respondent classifications. A simulation study is then conducted to evaluate the relative performance of these various model reduction processes. The results of the simulation suggest that all model reduction processes are able to provide quality estimates of the item parameters and respondent masteries, if the model is able to converge. The findings also show that if the full model does not converge, then reducing the structural model provides the best opportunities for achieving a converged solution. Implications of this study and directions for future research are discussed.

ORGANIC MATTER IN OIL SHALES

1980 ◽  
Vol 20 (1) ◽  
pp. 44 ◽  
Author(s):  
A.C. Hutton ◽  
A.J. Kantsler ◽  
A.C. Cook ◽  
D.M. McKirdy
Keyword(s):  
Organic Matter ◽  
Oil Shale ◽  
Large Scale ◽  
Mineral Matter ◽  
Green River ◽  
Fine Grained ◽  
Recovery Cost ◽  
Oil Shales

The Tertiary oil-shale deposits at Rundle in Queensland and of the Green River Formation in the western USA, together with Mesozoic deposits such as those at Julia Creek in Queensland, offer prospects of competitive recovery cost through the use of large-scale mining methods or the use of in situ processing.A framework for the classification of oil shales is proposed, based on the origin and properties of the organic matter. The organic matter in most Palaeozoic oil shales is dominantly large, discretely occurring algal bodies, referred to as alginite A. However, Tertiary oil shales of northeastern Australia are chiefly composed of numerous very thin laminae of organic matter cryptically-interbedded with mineral matter. Because the present maceral nomenclature does not adequately encompass the morphological and optical properties of most organic matter in oil shales, it is proposed to use the term alginite B for finely lamellar alginite, and the term lamosites (laminated oil shales) for oil shales which contain alginite B as their dominant organic constituent. In the Julia Creek oil shale the organic matter is very fine-grained and contains some alginite B but has a higher content of alginite A and accordingly is assigned to a suite of oil shales of mixed origin.Petrological and chemical techniques are both useful in identifying the nature and diversity of organic matter in oil shales and in assessing the environments in which they were formed. Such an understanding is necessary to develop exploration concepts for oil shales.

scholarly journals The Reliability of the Posterior Probability of Skill Attainment in Diagnostic Classification Models

2019 ◽  
Vol 45 (1) ◽  
pp. 5-31
Author(s):  
Matthew S. Johnson ◽  
Sandip Sinharay
Keyword(s):  
Posterior Probability ◽  
Real Data ◽  
Exact Calculation ◽  
Diagnostic Classification ◽  
Classification Models ◽  
Diagnostic Classification Models ◽  
Reliability Measure ◽  
Skill Mastery

One common score reported from diagnostic classification assessments is the vector of posterior means of the skill mastery indicators. As with any assessment, it is important to derive and report estimates of the reliability of the reported scores. After reviewing a reliability measure suggested by Templin and Bradshaw, this article suggests three new measures of reliability of the posterior means of skill mastery indicators and methods for estimating the measures when the number of items on the assessment and the number of skills being assessed render exact calculation computationally burdensome. The utility of the new measures is demonstrated using simulated and real data examples. Two of the suggested measures are recommended for future use.

Sign in / Sign up

Export Citation Format

Share Document