scholarly journals The Crit Value as an Effect Size Measure for Violations of Model Assumptions in Mokken Scale Analysis for Binary Data

2019 ◽  
Author(s):  
Daniela Ramona Crișan ◽  
Jorge Tendeiro ◽  
Rob Meijer
Keyword(s):  
False Positive ◽  
Binary Data ◽  
Simulation Studies ◽  
Mokken Scaling ◽  
Assumption Violations ◽  
Size Measure ◽  
Item Responses ◽  
Size Type ◽  
Nominal Rate ◽  
Questionnaire Construction

In empirical use of Mokken scaling, the Crit index is used as evidence (or lack thereof) of violations of some common model assumptions. The main goal of our study was two-fold: To make the formulation of the Crit index explicit and accessible, and to investigate its distribution under various measurement conditions. We conducted two simulation studies in the context of dichotomously-scored item responses. False positive rates and power to detect assumption violations were considered. We found that the false positive rates of Crit were close to the nominal rate in most conditions, and that power to detect misfit depended on the sample size, type of violation, and number of assumption-violating items. Our findings are relevant to all practitioners who use Mokken scaling for scale and questionnaire construction and revision.

scholarly journals The Crit coefficient in Mokken scale analysis: a simulation study and an application in quality-of-life research

2021 ◽  
Author(s):  
Daniela R. Crișan ◽  
Jorge N. Tendeiro ◽  
Rob R. Meijer
Keyword(s):  
Sample Size ◽  
False Positive ◽  
General Health Questionnaire ◽  
Small Samples ◽  
Scale Analysis ◽  
Mokken Scale Analysis ◽  
Large Samples ◽  
Quality Of Life Research ◽  
Mokken Scaling ◽  
Assumption Violations

Abstract Purpose In Mokken scaling, the Crit index was proposed and is sometimes used as evidence (or lack thereof) of violations of some common model assumptions. The main goal of our study was twofold: To make the formulation of the Crit index explicit and accessible, and to investigate its distribution under various measurement conditions. Methods We conducted two simulation studies in the context of dichotomously scored item responses. We manipulated the type of assumption violation, the proportion of violating items, sample size, and quality. False positive rates and power to detect assumption violations were our main outcome variables. Furthermore, we used the Crit coefficient in a Mokken scale analysis to a set of responses to the General Health Questionnaire (GHQ-12), a self-administered questionnaire for assessing current mental health. Results We found that the false positive rates of Crit were close to the nominal rate in most conditions, and that power to detect misfit depended on the sample size, type of violation, and number of assumption-violating items. Overall, in small samples Crit lacked the power to detect misfit, and in larger samples power differed considerably depending on the type of violation and proportion of misfitting items. Furthermore, we also found in our empirical example that even in large samples the Crit index may fail to detect assumption violations. Discussion Even in large samples, the Crit coefficient showed limited usefulness for detecting moderate and severe violations of monotonicity. Our findings are relevant to researchers and practitioners who use Mokken scaling for scale and questionnaire construction and revision.

Psychological networks in clinical populations: investigating the consequences of Berkson's bias

2019 ◽  
pp. 1-9 ◽  
Author(s):  
Jill de Ron ◽  
Eiko I. Fried ◽  
Sacha Epskamp
Keyword(s):  
Network Structure ◽  
Binary Data ◽  
Ordinal Data ◽  
Graphical Model ◽  
Rating Scale ◽  
Network Models ◽  
Simulation Studies ◽  
Potential Threat ◽  
Sum Score ◽  
Research Populations

Abstract Background In clinical research, populations are often selected on the sum-score of diagnostic criteria such as symptoms. Estimating statistical models where a subset of the data is selected based on a function of the analyzed variables introduces Berkson's bias, which presents a potential threat to the validity of findings in the clinical literature. The aim of the present paper is to investigate the effect of Berkson's bias on the performance of the two most commonly used psychological network models: the Gaussian Graphical Model (GGM) for continuous and ordinal data, and the Ising Model for binary data. Methods In two simulation studies, we test how well the two models recover a true network structure when estimation is based on a subset of the data typically seen in clinical studies. The network is based on a dataset of 2807 patients diagnosed with major depression, and nodes in the network are items from the Hamilton Rating Scale for Depression (HRSD). The simulation studies test different scenarios by varying (1) sample size and (2) the cut-off value of the sum-score which governs the selection of participants. Results The results of both studies indicate that higher cut-off values are associated with worse recovery of the network structure. As expected from the Berkson's bias literature, selection reduced recovery rates by inducing negative connections between the items. Conclusion Our findings provide evidence that Berkson's bias is a considerable and underappreciated problem in the clinical network literature. Furthermore, we discuss potential solutions to circumvent Berkson's bias and their pitfalls.

scholarly journals Psychological Networks in Clinical Populations: A tutorial on the consequences of Berkson's Bias

2019 ◽  
Author(s):  
Jill de Ron ◽  
Eiko I Fried ◽  
Sacha Epskamp
Keyword(s):  
Network Structure ◽  
Binary Data ◽  
Ordinal Data ◽  
Graphical Model ◽  
Rating Scale ◽  
Network Models ◽  
Simulation Studies ◽  
Potential Threat ◽  
Sum Score ◽  
Research Populations

In clinical research, populations are often selected on the sum-score of diagnostic criteria such as symptoms. Estimating statistical models where a subset of the data is selected based on a function of the analyzed variables introduces Berkson’s bias, which presents a potential threat to the validity of findings in the clinical literature. The aim of the present paper is to investigate the effect of Berkson’s bias on the performance of the two most commonly used psychological network models: the Gaussian Graphical Model (GGM) for continuous and ordinal data, and the Ising Model for binary data. In two simulation studies, we test how well the two models recover a true network structure when estimation is based on a subset of the data typically seen in clinical studies. The network is based on a dataset of 2,807 patients diagnosed with major depression, and nodes in the network are items from the Hamilton Rating Scale for Depression (HRSD). The simulation studies test different scenarios by varying (1) sample size and (2) the cut-off value of the sum-score which governs the selection of participants. The results of both studies indicate that higher cut-off values are associated with worse recovery of the network structure. As expected from the Berkson’s bias literature, selection reduced recovery rates by inducing negative connections between the items. Our findings provide evidence that Berkson’s bias is a considerable and underappreciated problem in the clinical network literature. Furthermore, we discuss potential solutions to circumvent Berkson’s bias and their pitfalls.

scholarly journals Approaches for Structural Investigations of Binary Data Using Confirmatory Factor Models

2018 ◽  
Vol 7 (6) ◽  
pp. 68
Author(s):  
Karl Schweizer ◽  
Siegbert Reiß ◽  
Stefan Troche
Keyword(s):  
Sample Size ◽  
Binary Data ◽  
Factor Models ◽  
The Other ◽  
Simulation Studies ◽  
Sample Sizes ◽  
Structural Investigations ◽  
Ml Estimation ◽  
Confirmatory Factor ◽  
The Relationship

An investigation of the suitability of threshold-based and threshold-free approaches for structural investigations of binary data is reported. Both approaches implicitly establish a relationship between binary data following the binomial distribution on one hand and continuous random variables assuming a normal distribution on the other hand. In two simulation studies we investigated: whether the fit results confirm the establishment of such a relationship, whether the differences between correct and incorrect models are retained and to what degree the sample size influences the results. Both approaches proved to establish the relationship. Using the threshold-free approach it was achieved by customary ML estimation whereas robust ML estimation was necessary in the threshold-based approach. Discrimination between correct and incorrect models was observed for both approaches. Larger CFI differences were found for the threshold-free approach than for the threshold-based approach. Dependency on sample size characterized the threshold-based approach but not the threshold-free approach. The threshold-based approach tended to perform better in large sample sizes, while the threshold-free approach performed better in smaller sample sizes.

scholarly journals Bayesian Sequential Monitoring of Single-Arm Trials: A Comparison of Futility Rules Based on Binary Data

2021 ◽  
Vol 18 (16) ◽  
pp. 8816
Author(s):  
Valeria Sambucini
Keyword(s):  
Binary Data ◽  
Standard Treatment ◽  
Decision Rules ◽  
Experimental Treatment ◽  
Slight Modification ◽  
Binary Response ◽  
Simulation Studies ◽  
Operating Characteristics ◽  
Binary Response Variable ◽  
Predictive Probabilities

In clinical trials, futility rules are widely used to monitor the study while it is in progress, with the aim of ensuring early termination if the experimental treatment is unlikely to provide the desired level of efficacy. In this paper, we focus on Bayesian strategies to perform interim analyses in single-arm trials based on a binary response variable. Designs that exploit both posterior and predictive probabilities are described and a slight modification of the futility rules is introduced when a fixed historical response rate is used, in order to add uncertainty in the efficacy probability of the standard treatment through the use of prior distributions. The stopping boundaries of the designs are compared under the same trial settings and simulation studies are performed to evaluate the operating characteristics when analogous procedures are used to calibrate the probability cut-offs of the different decision rules.

scholarly journals Guessing and Nature of Multidimensionality Matter: A Cautionary Note on the Use of Fit Indices to Assess Unidimensionality of Binary Data

2019 ◽  
Vol 1 (1) ◽  
Author(s):  
Yong Luo
Keyword(s):  
Binary Data ◽  
Statistical Power ◽  
Model Fit ◽  
Simulation Studies ◽  
Fit Indices ◽  
Dramatic Decrease ◽  
Cutoff Values ◽  
Model Fit Indices ◽  
Multiple Choice Items ◽  
Analysis Models

Use of cutoff values for model fit indices to assess dimensionality of binary data representing scores on multiple-choice items is a popular approach among researchers and practitioners, and the commonly used cutoff values are based on simulation studies that used as the generating model factor analysis models, which are compensatory models without modeling guessing. Consequently, it remains unknown how those cutoff values for model fit indices would perform when (a) guessing exists in data, and (b) data follow a noncompensatory multidimensional structure. In this paper, we conducted a comprehensive simulation study to investigate how guessing affected the statistical power of commonly used cutoff values for RMSEA, CFA, and TLI (RMSEA > 0.05; CFA < 0.95; TLI < 0.95) to detect violation of unidimensionality of binary data with both compensatory and noncompensatory models. The results indicated that when data were generated with compensatory models, increase of guessing values resulted in the systematic decrease of the power of RMSEA, CFA, and TLI to detect multidimensionality and in some conditions, a small increase of guessing value can result in dramatic decrease of their statistical power. It was also found that when data were generated with noncompensatory models, use of cutoff values of RMSEA, CFA, and TLI for unidimensionality assessment had unacceptably low statistical power, and while change of guessing magnitude could considerably change their statistical power, such changes were not systematic as in the compensatory models.  

scholarly journals Deterministic-Inputs, Noisy Mixed Modeling for Identifying Coexisting Condensation Rules

2020 ◽  
Author(s):  
Peida Zhan
Keyword(s):  
Psychometric Properties ◽  
Item Response ◽  
Response Probability ◽  
Cognitive Diagnosis ◽  
Model Parameters ◽  
Simulation Studies ◽  
Correct Item ◽  
Cognitive Diagnosis Models ◽  
Proposed Model ◽  
Item Responses

In cognitive diagnosis models (CDMs), the condensation rule reflects how latent attributes influence individuals’ observed item responses. In practice, multiple condensation rules may be involved in an item simultaneously, which indicates that the contribution of required attributes to the correct item response probability follows multiple condensation rules with different proportions. To consider the coexisting condensation rules while keeping the interpretability of model parameters, this study proposed the deterministic-inputs, noisy mixed (DINMix) model. Two simulation studies were conducted to evaluate the psychometric properties of the proposed model. The results indicate that the model parameters for the DINMix model can be well recovered, and the DINMix model can accurately identify coexisting condensation rules. An empirical example was also analyzed to illustrate the applicability and advantages of the proposed model.

Sign in / Sign up

Export Citation Format

Share Document