scholarly journals The Poor Fit of Model Fit for Selecting Number of Factors in Exploratory Factor Analysis for Scale Evaluation

2020 ◽  
pp. 001316442094289
Author(s):  
Amanda K. Montoya ◽  
Michael C. Edwards

Model fit indices are being increasingly recommended and used to select the number of factors in an exploratory factor analysis. Growing evidence suggests that the recommended cutoff values for common model fit indices are not appropriate for use in an exploratory factor analysis context. A particularly prominent problem in scale evaluation is the ubiquity of correlated residuals and imperfect model specification. Our research focuses on a scale evaluation context and the performance of four standard model fit indices: root mean square error of approximate (RMSEA), standardized root mean square residual (SRMR), comparative fit index (CFI), and Tucker–Lewis index (TLI), and two equivalence test-based model fit indices: RMSEAt and CFIt. We use Monte Carlo simulation to generate and analyze data based on a substantive example using the positive and negative affective schedule ( N = 1,000). We systematically vary the number and magnitude of correlated residuals as well as nonspecific misspecification, to evaluate the impact on model fit indices in fitting a two-factor exploratory factor analysis. Our results show that all fit indices, except SRMR, are overly sensitive to correlated residuals and nonspecific error, resulting in solutions that are overfactored. SRMR performed well, consistently selecting the correct number of factors; however, previous research suggests it does not perform well with categorical data. In general, we do not recommend using model fit indices to select number of factors in a scale evaluation framework.

2020 ◽  
Author(s):  
Amanda Kay Montoya ◽  
Michael C. Edwards

Model fit indices are being increasingly recommended and used to select the number of factors in an exploratory factor analysis. Growing evidence suggests that the recommended cutoff values for common model fit indices are not appropriate for use in an exploratory factor analysis context. A particularly prominent problem in scale evaluation is the ubiquity of correlated residuals and imperfect model specification. Our research focuses on a scale evaluation context and the performance of four standard model fit indices: root mean squared error of approximate (RMSEA), standardized root mean squared residual (SRMR), comparative fit index (CFI), and Tucker-Lewis index (TLI), and two equivalence test-based model fit indices: RMSEAt and CFIt. We use Monte Carlo simulation to generate and analyze data based on a substantive example using the positive and negative affective schedule (N = 1000). We systematically vary the number and magnitude of correlated residuals as well as nonspecific misspecification, to evaluate the impact on model fit indices in fitting a two-factor EFA. Our results show that all fit indices, except SRMR, are overly sensitive to correlated residuals and nonspecific error, resulting in solutions which are over-factored. SRMR performed well, consistently selecting the correct number of factors; however, previous research suggests it does not perform well with categorical data. In general, we do not recommend using model fit indices to select number of factors in a scale evaluation framework.


2016 ◽  
Vol 2016 ◽  
pp. 1-8 ◽  
Author(s):  
Hyunho Kim ◽  
Boncho Ku ◽  
Jong Yeol Kim ◽  
Young-Jae Park ◽  
Young-Bae Park

Background. Phlegm pattern questionnaire (PPQ) was developed to evaluate and diagnose phlegm pattern in Korean Medicine and Traditional Chinese Medicine, but it was based on a dataset from patients who visited the hospital to consult with a clinician regarding their health without any strict exclusion or inclusion. In this study, we reinvestigated the construct validity of PPQ with a new dataset and confirmed the feasibility of applying it to a healthy population.Methods. 286 healthy subjects were finally included and their responses to PPQ were acquired. Confirmatory factor analysis (CFA) was conducted and the model fit was discussed. We extracted a new factor structure by exploratory factor analysis (EFA) and compared the two factor structures.Results. In CFA results, the model fit indices are acceptable (RMSEA = 0.074) or slightly less than the good fit values (CFI = 0.839, TLI = 0.860). Many average variances extracted were smaller than the correlation coefficients of the factors, which shows the somewhat insufficient discriminant validity.Conclusions. Through the results from CFA and EFA, this study shows clinically acceptable model fits and suggests the feasibility of applying PPQ to a healthy population with relatively good construct validity and internal consistency.


2019 ◽  
Vol 80 (2) ◽  
pp. 217-241 ◽  
Author(s):  
W. Holmes Finch

Exploratory factor analysis (EFA) is widely used by researchers in the social sciences to characterize the latent structure underlying a set of observed indicator variables. One of the primary issues that must be resolved when conducting an EFA is determination of the number of factors to retain. There exist a large number of statistical tools designed to address this question, with none being universally optimal across applications. Recently, researchers have investigated the use of model fit indices that are commonly used in the conduct of confirmatory factor analysis to determine the number of factors to retain in EFA. These results have yielded mixed results, appearing to be effective when used in conjunction with normally distributed indicators, but not being as effective for categorical indicators. The purpose of this simulation study was to compare the performance of difference values for several fit indices as a method for identifying the optimal number of factors to retain in an EFA, with parallel analysis, which is one of the most reliable such extant methods. Results of the simulation demonstrated that the use of fit index difference values outperformed parallel analysis for categorical indicators, and for normally distributed indicators when factor loadings were small. Implications of these findings are discussed.


2021 ◽  
pp. 001316442199240
Author(s):  
Chunhua Cao ◽  
Eun Sook Kim ◽  
Yi-Hsin Chen ◽  
John Ferron

This study examined the impact of omitting covariates interaction effect on parameter estimates in multilevel multiple-indicator multiple-cause models as well as the sensitivity of fit indices to model misspecification when the between-level, within-level, or cross-level interaction effect was left out in the models. The parameter estimates produced in the correct and the misspecified models were compared under varying conditions of cluster number, cluster size, intraclass correlation, and the magnitude of the interaction effect in the population model. Results showed that the two main effects were overestimated by approximately half of the size of the interaction effect, and the between-level factor mean was underestimated. None of comparative fit index, Tucker–Lewis index, root mean square error of approximation, and standardized root mean square residual was sensitive to the omission of the interaction effect. The sensitivity of information criteria varied depending majorly on the magnitude of the omitted interaction, as well as the location of the interaction (i.e., at the between level, within level, or cross level). Implications and recommendations based on the findings were discussed.


2015 ◽  
Vol 1 (311) ◽  
Author(s):  
Piotr Tarka

Abstract: The objective article is the comparative analysis of Likert rating scale based on the following range of response categories, i.e. 5, 7, 9 and 11 in context of the appropriate process of factors extraction in exploratory factor analysis (EFA). The problem which is being addressed in article relates primarily to the methodological aspects, both in selection of the optimal number of response categories of the measured items (constituting the Likert scale) and identification of possible changes, differences or similarities associated (as a result of the impact of four types of scales) with extraction and determination the appropriate number of factors in EFA model.Keywords: Exploratory factor analysis, Likert scale, experiment research, marketing


2020 ◽  
Vol 46 (Supplement_1) ◽  
pp. S179-S179
Author(s):  
Mei San Ang ◽  
Gurpreet Rekhi ◽  
Jimmy Lee

Abstract Background The conceptualization of negative symptoms has been refined in the past decades. Two-factor model comprising Motivation and Pleasure (MAP) and Emotional Expressivity (EE), five-factor model representing five domains of negative symptoms and second-order five-factor model incorporating the two-factor and five-factor models (Anhedonia, Asociality and Avolition regressed on MAP; Blunted Affect and Alogia regressed on EE) have been suggested as latent structure of negative symptoms. In most studies, the item “Lack of Normal Distress” in the Brief Negative Symptom Scale (BNSS) did not fit well in factor models. Nevertheless, the reported correlation and item-total correlation of Distress with other negative symptom domains and BNSS items were not negligible. Emotion deficit was also discussed as a part of negative symptoms conceptualization. As a single item may not be sufficient to represent an underlying construct that is potentially abstract and complex, the Schedule for the Deficit Syndrome (SDS) which comprises “Diminished Emotional Range” that is conceptually relevant to the BNSS Distress was employed. The study aimed to reexamine the conceptualization of negative symptoms by examining the model fit of several models when BNSS Distress and SDS Emotion (EMO) were included in the models using confirmatory factor analyses (CFA). Methods Two-hundred and seventy-four schizophrenia outpatients aged 21–65 were assessed on the BNSS and SDS. In the two-factor models, Restricted Affect, Diminished Emotional Range and Poverty of Speech in SDS and all items in BNSS Blunted Affect and Alogia subscales were regressed on EE, Curbing of Interests, Diminished Sense of Purpose and Diminished Social Drive in SDS and all items in BNSS Anhedonia, Asociality and Avolition subscales were regressed on MAP, without EMO, or with EMO regressed on either EE or MAP. Five-factor models and second-order five-factor models were examined, with or without EMO. Lastly, a six-factor model with EMO manifested by the sixth factor and second-order six-factor models in which EMO was regressed on either EE or MAP were tested. Root mean square error of approximation (RMSEA) <0.08, comparative fit index (CFI) >0.95, the Tucker-Lewis Index (TLI) >0.95, and weighted root-mean-square residual (WRMR) <1.0 indicate good model fit. CFAs were conducted using Mplus version 7.4. Results The two-factor models did not yield adequate fit indices. Five-factor model and second-order five-factor model without EMO had good model fit; five-factor model: RMSEA=0.056 (0.044–0.068), CFI=0.996, TFI=0.995, WRMR=0.718; second-order five-factor model: RMSEA=0.049 (0.036–0.061), CFI=0.997, TFI=0.996, WRMR=0.758. When EMO was included as indicator in one of the factors in the five-factor models, only the model in which EMO was regressed on Alogia yielded adequate fit. Similarly, in the second-order five-factor models, adequate fit indices were observed only when EMO was regressed on Alogia and Blunted Affect. The six-factor model fitted the data well, RMSEA=0.053 (0.042–0.064), CFI=0.996, TFI=0.995, WRMR=0.711. Second-order six-factor model with EMO regressed on EE yielded better model fit than MAP, RMSEA=0.050 (0.039–0.061), CFI=0.996, TFI=0.995, WRMR=0.849. Discussion In line with previous studies, five-factor and second-order five-factor models without EMO fitted the data well. When EMO was included, a six-factor model and a second-order six-factor model in which the sixth factor was regressed on EE showed good model fit. Emotion, motivation and behavior are intertwined. Our results showed that diminished emotion may also be one of the components of negative symptoms, which had higher association with EE than MAP.


2017 ◽  
Vol 34 (2) ◽  
pp. 211-218 ◽  
Author(s):  
Acácia Aparecida Angeli dos SANTOS ◽  
Thatiana Helena de LIMA

Abstract The objective of this study is to investigate the evidence of construct validity of a phonological awareness instrument. Exploratory factor analysis was carried out on data collected from 510 elementary and middle school students in 2nd and 6th grades attending two different public schools in the city of São Paulo, Brazil; most were males with mean age of 8.4 years. Confirmatory factor analysis was carried out on data collected from 427 students from other four Brazilian states in the same grades; most were females with mean age of 9.3 years. The instrument used was the Roteiro de Avaliação da Consciência Fonológica, a phonological awareness test. The exploratory factor analysis showed a three-factor solution. As for the confirmatory factor analysis, of the two models tested, the one that indicated better model fit indices was composed of three factors. The model found is adequate for the task carried out in this study. However, more studies should be carried out to further refine the instrument.


2019 ◽  
Vol 50 (1) ◽  
pp. 222-242 ◽  
Author(s):  
Jane McPherson ◽  
Neil Abell

Abstract This article introduces a measurable framework for rights-based social work practice and an accompanying set of instruments, the Human Rights Methods in Social Work (HRMSW) scales: (i) ‘participation’, (ii) ‘non-discrimination’, (iii) ‘strengths perspective’, (iv) ‘micro/macro integration’, (v) ‘capacity-building’, (vi) ‘community and interdisciplinary collaboration’, (vii) ‘activism’ and (viii) ‘accountability’. These scales, designed for use by researchers, educators and practitioners, are the first to measure social workers’ use of rights-based methods. An electronic survey was used to collect data from a convenience sample of 1,014 licensed US social workers, and a confirmatory factor analysis was used to validate the scales’ psychometric properties. A respecified model using eight error covariances fit the data (χ2/df ratio = 2.9; comparative fit index (CFI) = 0.91; tucker lewis index (TLI) = 0.90; root mean square error of approximation (RMSEA) = 0.04; standardized root mean square residual (SRMR) = 0.07). Thus, factor analysis yielded a set of eight related scales—collectively called the HRMSW—each measuring a different human rights practice method that social workers can use to promote human dignity and the rights-based principles of participation, accountability and non-discrimination. Scholars argue that ‘human rights’ are a more appropriate yardstick for measuring the impact of social work intervention rather than our traditional aim of social justice; the HRMSW scales can help us begin to test this proposition.


Sign in / Sign up

Export Citation Format

Share Document