Revisiting the Effect of Varying the Number of Response Alternatives in Clinical Assessment: Evidence From Measuring ADHD Symptoms

This study illustrated the effect of varying the number of response alternatives in clinical assessment using a within-participant, repeated-measures approach. Participants reported the presence of current attention-deficit/hyperactivity disorder symptoms using both a binary and a polytomous (4-point) rating scale across two counterbalanced administrations of the Current Symptoms Scale (CSS). Psychometric properties of the CSS were examined using (a) self-reported binary, (b) self-reported 4-point ratings obtained from each administration of the CSS, and (c) artificially dichotomized responses derived from observed 4-point ratings. Under the same ordinal factor analysis model, results indicated that the number of response alternatives affected item parameter estimates, standard errors, goodness of fit indices, individuals’ test scores, and reliability of the test scores. With fewer response alternatives, the precision of the measurement decreased, and the power of using the goodness-of-fit indices to detect model misfit decreased. These findings add to recent research advocating for the inclusion of a large number of response alternatives in the development of clinical assessments and further suggest that researchers should be cautious about reducing the number of response categories in data analysis.

Download Full-text

Interkultureller Vergleich der Conners-Skalen: Lässt sich die US-amerikanische Faktorenstruktur an einer deutschen Klinikstichprobe replizieren?

Zeitschrift für Kinder- und Jugendpsychiatrie und Psychotherapie ◽

10.1024//1422-4917.29.1.16 ◽

2001 ◽

Vol 29 (1) ◽

pp. 16-24 ◽

Cited By ~ 9

Author(s):

M. Huss ◽

A. Iseler ◽

U. Lehmkuhl

Keyword(s):

Goodness Of Fit ◽

Rating Scale ◽

Fit Indices ◽

Parent Rating ◽

Parent Rating Scale

Zusammenfassung Fragestellung: Im Zuge der internationalen Vernetzung kinderpsychiatrischer Forschung kommt der Frage der interkulturellen Vergleichbarkeit von Faktorenstrukturen gängiger Fragebogenverfahren eine zentrale Bedeutung zu. Die vorliegende Studie prüft, ob die US-amerikanische Faktorenstruktur (US-Modell) der Conners Parent Rating Scale (CPRS) an einer deutschen kinderpsychiatrischen Inanspruchnahmepopulation replizierbar ist. Methodik: Die Stichprobe von 1394 Kindern und Jugendlichen wird randomisiert halbiert. An der einen Teilstichprobe wird mittels explorativer Faktorenanalyse ein deutsches Faktorenmodell (D-Modell) entwickelt. Dieses wird im Vergleich mit dem nach Conners (1989) erstellten US-Modell an der zweiten Teilstichprobe mittels konfirmatorischer Faktorenanalyse (LISREL 8) überprüft. Ergebnisse: Das D-Modell stimmt mit dem US-Modell in 87% der Pfadbeziehungen überein. Beide Modelle weisen Einschränkungen der Vorhersagegüte auf. Das D-Modell hat erwartungskonform etwas bessere Vorhersagewerte als das US-Modell (GFI = .81; AGFI = .75 versus GFI = .76; AGFI = .71). Schlussfolgerungen: Verglichen mit anderen Studien zur interkulturellen Generalisierbarkeit von Faktorenstrukturen dimensionaler Verfahren (z.B. De Groot et al., 1994 ) haben die “goodness of fit” Indices für die CPRS insgesamt schlechtere Werte. Dies ist jedoch größtenteils auf Restriktionen im Modell (keine Mehrfachladungen) zurückzuführen. Interkulturelle Abweichungen ergeben sich bei der Skala “Impulsivität/Hyperaktivität”. Die übrigen Skalen lassen sich gut replizieren.

Download Full-text

Fitting Large Factor Analysis Models With Ordinal Data

Educational and Psychological Measurement ◽

10.1177/0013164418818242 ◽

2018 ◽

Vol 79 (3) ◽

pp. 417-436 ◽

Cited By ~ 1

Author(s):

Christine DiStefano ◽

Heather L. McDaniel ◽

Liyun Zhang ◽

Dexin Shi ◽

Zhehan Jiang

Keyword(s):

Factor Analysis ◽

Ordinal Data ◽

Goodness Of Fit ◽

Weighted Least Squares ◽

Standard Errors ◽

Parameter Estimates ◽

Fit Indices ◽

Ordered Categories ◽

Size Number ◽

Analysis Models

A simulation study was conducted to investigate the model size effect when confirmatory factor analysis (CFA) models include many ordinal items. CFA models including between 15 and 120 ordinal items were analyzed with mean- and variance-adjusted weighted least squares to determine how varying sample size, number of ordered categories, and misspecification affect parameter estimates, standard errors of parameter estimates, and selected fit indices. As the number of items increased, the number of admissible solutions and accuracy of parameter estimates improved, even when models were misspecified. Also, standard errors of parameter estimates were closer to empirical standard deviation values as the number of items increased. When evaluating goodness-of-fit for ordinal CFA with many observed indicators, researchers should be cautious in interpreting the root mean square error of approximation, as this value appeared overly optimistic under misspecified conditions.

Download Full-text

The Impact of the Learning Effectiveness on Alumni Employability: A Case Study of a T/I Department in Taiwan

Studies in English Language Teaching ◽

10.22158/selt.v4n4p586 ◽

2016 ◽

Vol 4 (4) ◽

pp. 586

Author(s):

Pin-Shan Hsiung

Keyword(s):

Structural Equation ◽

Goodness Of Fit ◽

Core Competency ◽

Equation Modeling ◽

Parameter Estimates ◽

Learning Effectiveness ◽

Fit Indices ◽

Academic Advancement ◽

Current Employment Status ◽

The Impact

In recent years, the number of translation and interpretation courses offered in Taiwan has increased rapidly, but few studies has looked at the employability of their graduates. This paper is aimed to investigate the direct effects of curriculum on the professional careers of alumni as reflected in their current employment status and level of academic advancement. A questionnaire survey was carried out to evaluate multiple aspects of teaching, including learning effectiveness, core competency, curriculum design and repay the society. Through an analysis of 150 named and 300 anonymous questionnaires, this study analyzed the learning effectiveness as the mediator for the careers of alumni, using the Amos statistical package for Structural Equation Modeling (SEM), along with other related techniques, such as Confirmatory Factor Analysis (CFA). The analyses have produced parameter estimates and goodness-of-fit indices, which could be useful for many purposes, such as examining longitudinal data and comparing groups. It is hoped that this brief study may provide a better understanding and a basis for future studies.

Download Full-text

Comparison of Precision and Accuracy of Five Methods to Analyse Total Score Data

The AAPS Journal ◽

10.1208/s12248-020-00546-w ◽

2020 ◽

Vol 23 (1) ◽

Author(s):

Gustaf J. Wellhagen ◽

Mats O. Karlsson ◽

Maria C. Kjellsson

Keyword(s):

Repeated Measures ◽

Rating Scale ◽

Continuous Variable ◽

Parameter Estimates ◽

Irt Model ◽

External Data ◽

Irt Models ◽

Level Data ◽

Item Level ◽

Disease Rating

AbstractTotal score (TS) data is generated from composite scales consisting of several questions/items, such as the Movement Disorder Society-Unified Parkinson’s Disease Rating Scale (MDS-UPDRS). The analysis method that most fully uses the information gathered is item response theory (IRT) models, but these are complex and require item-level data which may not be available. Therefore, the TS is commonly analysed with standard continuous variable (CV) models, which do not respect the bounded nature of data. Bounded integer (BI) models do respect the data nature but are not as extensively researched. Mixed models for repeated measures (MMRM) are an alternative that requires few assumptions and handles dropout without bias. If an IRT model exists, the expected mean and standard deviation of TS can be computed through IRT-informed functions—which allows CV and BI models to estimate parameters on the IRT scale. The fit, performance on external data and parameter precision (when applicable) of CV, BI and MMRM to analyse simulated TS data from the MDS-UPDRS motor subscale are investigated in this work. All models provided accurate predictions and residuals without trends, but the fit of CV and BI models was improved by IRT-informed functions. The IRT-informed BI model had more precise parameter estimates than the IRT-informed CV model. The IRT-informed models also had the best performance on external data, while the MMRM model was worst. In conclusion, (1) IRT-informed functions improve TS analyses and (2) IRT-informed BI models had more precise IRT parameter estimates than IRT-informed CV models.

Download Full-text

Confirmatory Factor Analysis of the Long and Short Forms of the Worry Domains Questionnaire

European Journal of Psychological Assessment ◽

10.1027/1015-5759.25.4.239 ◽

2009 ◽

Vol 25 (4) ◽

pp. 239-243

Author(s):

Roberto Nuevo ◽

Andrés Losada ◽

María Márquez-González ◽

Cecilia Peñacoba

Keyword(s):

Factor Analysis ◽

Confirmatory Factor Analysis ◽

Factor Structure ◽

Goodness Of Fit ◽

Factorial Validity ◽

Fit Indices ◽

Short Forms ◽

Confirmatory Factor ◽

Positive Correlations ◽

Future Work

The Worry Domains Questionnaire was proposed as a measure of both pathological and nonpathological worry, and assesses the frequency of worrying about five different domains: relationships, lack of confidence, aimless future, work, and financial. The present study analyzed the factor structure of the long and short forms of the WDQ (WDQ and WDQ-SF, respectively) through confirmatory factor analysis in a sample of 262 students (M age = 21.8; SD = 2.6; 86.3% females). While the goodness-of-fit indices did not provide support for the WDQ, good fit indices were found for the WDQ-SF. Furthermore, no source of misspecification was identified, thus, supporting the factorial validity of the WDQ-SF scale. Significant positive correlations between the WDQ-SF and its subscales with worry (PSWQ), anxiety (STAI-T), and depression (BDI) were found. The internal consistency was good for the total scale and for the subscales. This work provides support for the use of the WDQ-SF, and potential uses for research and clinical purposes are discussed.

Download Full-text

A Cautionary Note on Incremental Fit Indices Reported by LISREL

Methodology ◽

10.1027/1614-1881.1.2.81 ◽

2005 ◽

Vol 1 (2) ◽

pp. 81-85 ◽

Cited By ~ 5

Author(s):

Stefan C. Schmukle ◽

Jochen Hardt

Keyword(s):

Factor Analysis ◽

Maximum Likelihood ◽

Structural Equation ◽

Structural Equation Models ◽

Null Model ◽

Likelihood Estimation ◽

Parameter Estimates ◽

Fit Indices ◽

Cautionary Note ◽

Confirmatory Factor

Abstract. Incremental fit indices (IFIs) are regularly used when assessing the fit of structural equation models. IFIs are based on the comparison of the fit of a target model with that of a null model. For maximum-likelihood estimation, IFIs are usually computed by using the χ2 statistics of the maximum-likelihood fitting function (ML-χ2). However, LISREL recently changed the computation of IFIs. Since version 8.52, IFIs reported by LISREL are based on the χ2 statistics of the reweighted least squares fitting function (RLS-χ2). Although both functions lead to the same maximum-likelihood parameter estimates, the two χ2 statistics reach different values. Because these differences are especially large for null models, IFIs are affected in particular. Consequently, RLS-χ2 based IFIs in combination with conventional cut-off values explored for ML-χ2 based IFIs may lead to a wrong acceptance of models. We demonstrate this point by a confirmatory factor analysis in a sample of 2449 subjects.

Download Full-text

Is That Measure Really One-Dimensional?

Methodology ◽

10.1027/1614-2241/a000158 ◽

2018 ◽

Vol 14 (4) ◽

pp. 188-196 ◽

Cited By ~ 5

Author(s):

Esther T. Beierl ◽

Markus Bühner ◽

Moritz Heene

Keyword(s):

Model Fit ◽

Nuisance Parameters ◽

Factorial Validity ◽

Fit Indices ◽

Analysis Model ◽

Factor Analysis Model ◽

One Dimensional ◽

Confirmatory Factor Analysis Model ◽

Cutoff Values ◽

Confirmatory Factor

Abstract. Factorial validity is often assessed using confirmatory factor analysis. Model fit is commonly evaluated using the cutoff values for the fit indices proposed by Hu and Bentler (1999) . There is a body of research showing that those cutoff values cannot be generalized. Model fit does not only depend on the severity of misspecification, but also on nuisance parameters, which are independent of the misspecification. Using a simulation study, we demonstrate their influence on measures of model fit. We specified a severe misspecification, omitting a second factor, which signifies factorial invalidity. Measures of model fit showed only small misfit because nuisance parameters, magnitude of factor loadings and a balanced/imbalanced number of indicators per factor, also influenced the degree of misfit. Drawing from our results, we discuss challenges in the assessment of factorial validity.

Download Full-text

Boxing vs Sensory Exercise for Parkinson’s Disease: A Double-Blinded Randomized Controlled Trial

Neurorehabilitation and Neural Repair ◽

10.1177/15459683211023197 ◽

2021 ◽

pp. 154596832110231

Author(s):

Kishoree Sangarapillai ◽

Benjamin M. Norman ◽

Quincy J. Almeida

Keyword(s):

Parkinson’S Disease ◽

Parkinson's Disease ◽

Randomized Controlled Trial ◽

Repeated Measures ◽

Rating Scale ◽

Controlled Trial ◽

Motor Symptoms ◽

Post Intervention ◽

Randomized Controlled ◽

Double Blinded

Background. Exercise is increasingly becoming recognized as an important adjunct to medications in the clinical management of Parkinson’s disease (PD). Boxing and sensory exercise have shown immediate benefits, but whether they continue beyond program completion is unknown. This study aimed to investigate the effects of boxing and sensory training on motor symptoms of PD, and whether these benefits remain upon completion of the intervention. Methods. In this 20-week double-blinded randomized controlled trial, 40 participants with idiopathic PD were randomized into 2 treatment groups, (n = 20) boxing or (n = 20) sensory exercise. Participants completed 10 weeks of intervention. Motor symptoms were assessed at (week 0, 10, and 20) using the Unified Parkinson’s Disease Rating Scale (UPDRS-III). Data were analyzed using SPSS, and repeated-measures ANOVA was conducted. Results. A significant interaction effect between groups and time were observed F(1, 39) = 4.566, P = .036, where the sensory group improved in comparison to the boxing group. Post hoc analysis revealed that in comparison to boxing, the effects of exercise did not wear off at washout (week 20) P < .006. Conclusion. Future rehabilitation research should incorporate similar measures to explore whether effects of exercise wear off post intervention.

Download Full-text

Metacognitive Instruction for Sustainable Learning: Learners’ Perceptions of Task Difficulty and Use of Metacognitive Strategies in Completing Integrated Speaking Tasks

Sustainability ◽

10.3390/su13116275 ◽

2021 ◽

Vol 13 (11) ◽

pp. 6275

Author(s):

Weiwei Zhang ◽

Donglan Zhang ◽

Lawrence Jun Zhang

Keyword(s):

Foreign Language ◽

Repeated Measures ◽

Task Difficulty ◽

Rating Scale ◽

Empirical Support ◽

Metacognitive Strategies ◽

Production Model ◽

High Stakes ◽

Efl Learners ◽

Metacognitive Instruction

This mixed-methods study investigated English-as-a-foreign-language (EFL) learners’ perceptions of task difficulty and their use of metacognitive strategies in completing integrated speaking tasks as empirical evidence for the effects of metacognitive instruction. A total of 130 university students were invited to complete four integrated speaking tasks and answer a metacognitive strategy inventory and a self-rating scale. A sub-sample of eight students participated in the subsequent interviews. One-way repeated measures MANOVA and structure coding with content analysis led to two main findings: (a) EFL learners’ use of metacognitive strategies, in particular, problem-solving, was considerably affected by their perceptions of task difficulty in completing the integrated speaking tasks; (b) EFL learners were not active users of metacognitive strategies in performing these tasks. These findings not only support the necessity of taking into account learners’ perceptions of task difficulty in designing lesson plans for metacognitive instruction, but also support a metacognitive instruction model. In addition, the findings provide empirical support for the utility of Kormos’ Bilingual Speech Production Model. As the integrated speaking tasks came from a high-stakes test, these findings also offer validity evidence for test development in language assessment to ascertain sustainable EFL learning for nurturing learner autonomy as an ultimate goal.

Download Full-text

Examining the Impact of and Sensitivity of Fit Indices to Omitting Covariates Interaction Effect in Multilevel Multiple-Indicator Multiple-Cause Models

Educational and Psychological Measurement ◽

10.1177/0013164421992407 ◽

2021 ◽

pp. 001316442199240

Author(s):

Chunhua Cao ◽

Eun Sook Kim ◽

Yi-Hsin Chen ◽

John Ferron

Keyword(s):

Root Mean Square ◽

Interaction Effect ◽

Intraclass Correlation ◽

Information Criteria ◽

Parameter Estimates ◽

Mean Square ◽

Fit Indices ◽

Cluster Number ◽

Multiple Indicator ◽

The Impact

This study examined the impact of omitting covariates interaction effect on parameter estimates in multilevel multiple-indicator multiple-cause models as well as the sensitivity of fit indices to model misspecification when the between-level, within-level, or cross-level interaction effect was left out in the models. The parameter estimates produced in the correct and the misspecified models were compared under varying conditions of cluster number, cluster size, intraclass correlation, and the magnitude of the interaction effect in the population model. Results showed that the two main effects were overestimated by approximately half of the size of the interaction effect, and the between-level factor mean was underestimated. None of comparative fit index, Tucker–Lewis index, root mean square error of approximation, and standardized root mean square residual was sensitive to the omission of the interaction effect. The sensitivity of information criteria varied depending majorly on the magnitude of the omitted interaction, as well as the location of the interaction (i.e., at the between level, within level, or cross level). Implications and recommendations based on the findings were discussed.

Download Full-text