Examining the Impact of and Sensitivity of Fit Indices to Omitting Covariates Interaction Effect in Multilevel Multiple-Indicator Multiple-Cause Models

2021 ◽  
pp. 001316442199240
Author(s):  
Chunhua Cao ◽  
Eun Sook Kim ◽  
Yi-Hsin Chen ◽  
John Ferron

This study examined the impact of omitting covariates interaction effect on parameter estimates in multilevel multiple-indicator multiple-cause models as well as the sensitivity of fit indices to model misspecification when the between-level, within-level, or cross-level interaction effect was left out in the models. The parameter estimates produced in the correct and the misspecified models were compared under varying conditions of cluster number, cluster size, intraclass correlation, and the magnitude of the interaction effect in the population model. Results showed that the two main effects were overestimated by approximately half of the size of the interaction effect, and the between-level factor mean was underestimated. None of comparative fit index, Tucker–Lewis index, root mean square error of approximation, and standardized root mean square residual was sensitive to the omission of the interaction effect. The sensitivity of information criteria varied depending majorly on the magnitude of the omitted interaction, as well as the location of the interaction (i.e., at the between level, within level, or cross level). Implications and recommendations based on the findings were discussed.

2018 ◽  
Vol 79 (3) ◽  
pp. 512-544
Author(s):  
Chunhua Cao ◽  
Eun Sook Kim ◽  
Yi-Hsin Chen ◽  
John Ferron ◽  
Stephen Stark

In multilevel multiple-indicator multiple-cause (MIMIC) models, covariates can interact at the within level, at the between level, or across levels. This study examines the performance of multilevel MIMIC models in estimating and detecting the interaction effect of two covariates through a simulation and provides an empirical demonstration of modeling the interaction in multilevel MIMIC models. The design factors include the location of the interaction effect (i.e., between, within, or across levels), cluster number, cluster size, intraclass correlation (ICC) level, magnitude of the interaction effect, and cross-level measurement invariance status. Type I error, power, relative bias, and root mean square of error of the interaction effects are examined. The results showed that multilevel MIMIC models performed well in detecting the interaction effect at the within or across levels. However, when the interaction effect was at the between level, the performance of multilevel MIMIC models depended on the magnitude of the interaction effect, ICC, and sample size, especially cluster number. Overall, cross-level measurement noninvariance did not make a notable impact on the estimation of interaction in the structural part of multilevel MIMIC models when factor loadings were allowed to be different across levels.


2020 ◽  
pp. 001316442094289
Author(s):  
Amanda K. Montoya ◽  
Michael C. Edwards

Model fit indices are being increasingly recommended and used to select the number of factors in an exploratory factor analysis. Growing evidence suggests that the recommended cutoff values for common model fit indices are not appropriate for use in an exploratory factor analysis context. A particularly prominent problem in scale evaluation is the ubiquity of correlated residuals and imperfect model specification. Our research focuses on a scale evaluation context and the performance of four standard model fit indices: root mean square error of approximate (RMSEA), standardized root mean square residual (SRMR), comparative fit index (CFI), and Tucker–Lewis index (TLI), and two equivalence test-based model fit indices: RMSEAt and CFIt. We use Monte Carlo simulation to generate and analyze data based on a substantive example using the positive and negative affective schedule ( N = 1,000). We systematically vary the number and magnitude of correlated residuals as well as nonspecific misspecification, to evaluate the impact on model fit indices in fitting a two-factor exploratory factor analysis. Our results show that all fit indices, except SRMR, are overly sensitive to correlated residuals and nonspecific error, resulting in solutions that are overfactored. SRMR performed well, consistently selecting the correct number of factors; however, previous research suggests it does not perform well with categorical data. In general, we do not recommend using model fit indices to select number of factors in a scale evaluation framework.


Author(s):  
Igor Junio de Oliveira Custódio ◽  
Gibson Moreira Praça ◽  
Leandro Vinhas de Paula ◽  
Sarah da Glória Teles Bredt ◽  
Fabio Yuzo Nakamura ◽  
...  

This study aimed to analyze the intersession reliability of global positioning system (GPS-based) distances and accelerometer-based (acceleration) variables in small-sided soccer games (SSG) with and without the offside rule, as well as compare variables between the tasks. Twenty-four high-level U-17 soccer athletes played 3 versus 3 (plus goalkeepers) SSG in two formats (with and without the offside rule). SSG were performed on eight consecutive weeks (4 weeks for each group), twice a week. The physical demands were recorded using a GPS with an embedded triaxial accelerometer. GPS-based variables (total distance, average speed, and distances covered at different speeds) and accelerometer-based variables (Player Load™, root mean square of the acceleration recorded in each movement axis, and the root mean square of resultant acceleration) were calculated. Results showed that the inclusion of the offside rule reduced the total distance covered (large effect) and the distances covered at moderate speed zones (7–12.9 km/h – moderate effect; 13–17.9 km/h – large effect). In both SSG formats, GPS-based variables presented good to excellent reliability (intraclass correlation coefficients – ICC > 0.62) and accelerometer-based variables presented excellent reliability (ICC values > 0.89). Based on the results of this study, the offside rule decreases the physical demand of 3 versus 3 SSG and the physical demands required in these SSG present high intersession reliability.


2021 ◽  
Vol 52 (1) ◽  
pp. 6-14
Author(s):  
Amit Tak ◽  
Sunita Dia ◽  
Mahendra Dia ◽  
Todd Wehner

Background: The forecasting of Coronavirus Disease-19 (COVID-19) dynamics is a centrepiece in evidence-based disease management. Numerous approaches that use mathematical modelling have been used to predict the outcome of the pandemic, including data-driven models, empirical and hybrid models. This study was aimed at prediction of COVID-19 evolution in India using a model based on autoregressive integrated moving average (ARIMA). Material and Methods: Real-time Indian data of cumulative cases and deaths of COVID-19 was retrieved from the Johns Hopkins dashboard. The dataset from 11 March 2020 to 25 June 2020 (n = 107 time points) was used to fit the autoregressive integrated moving average model. The model with minimum Akaike Information Criteria was used for forecasting. The predicted root mean square error (PredRMSE) and base root mean square error (BaseRMSE) were used to validate the model. Results: The ARIMA (1,3,2) and ARIMA (3,3,1) model fit best for cumulative cases and deaths, respectively, with minimum Akaike Information Criteria. The prediction of cumulative cases and deaths for next 10 days from 26 June 2020 to 5 July 2020 showed a trend toward continuous increment. The PredRMSE and BaseRMSE of ARIMA (1,3,2) model were 21,137 and 166,330, respectively. Similarly, PredRMSE and BaseRMSE of ARIMA (3,3,1) model were 668.7 and 5,431, respectively. Conclusion: It is proposed that data on COVID-19 be collected continuously, and that forecasting continue in real time. The COVID-19 forecast assist government in resource optimisation and evidence-based decision making for a subsequent state of affairs.


Author(s):  
Oluyori P. Dare ◽  
Eteje S. Okiemute

<p class="abstract"><strong>Background:</strong> Orthometric height, as well as geoid modelling using the geometric method, requires centroid computation. And this can be obtained using various models, as well as methods. These methods of centroid mean computation have impacts on the accuracy of the geoid model since the basis of the development of the theory of each centroid mean type is different. This paper presents the impact of different centroid means on the accuracy of orthometric height modelling by geometric geoid method.</p><p class="abstract"><strong>Methods:</strong> DGPS observation was carried out to obtain the coordinates and ellipsoidal heights of selected points. The centroid means were computed with the coordinates using three different centroid means models (arithmetic mean, root mean square and harmonic mean). The computed centroid means were entered accordingly into a Microsoft Excel program developed using the Multiquadratic surface to obtain the model orthometric heights at various centroid means. The root means square error (RMSE) index was applied to obtain the accuracy of the model using the known and the model orthometric heights obtained at various centroid means.  </p><p class="abstract"><strong>Results:</strong> The computed accuracy shows that the arithmetic mean method is the best among the three centroid means types.</p><p class="abstract"><strong>Conclusions:</strong> It is concluded that the arithmetic mean method should be adopted for centroid computation, as well as orthometric height modelling using the geometric method.</p>


2020 ◽  
Vol 46 (Supplement_1) ◽  
pp. S179-S179
Author(s):  
Mei San Ang ◽  
Gurpreet Rekhi ◽  
Jimmy Lee

Abstract Background The conceptualization of negative symptoms has been refined in the past decades. Two-factor model comprising Motivation and Pleasure (MAP) and Emotional Expressivity (EE), five-factor model representing five domains of negative symptoms and second-order five-factor model incorporating the two-factor and five-factor models (Anhedonia, Asociality and Avolition regressed on MAP; Blunted Affect and Alogia regressed on EE) have been suggested as latent structure of negative symptoms. In most studies, the item “Lack of Normal Distress” in the Brief Negative Symptom Scale (BNSS) did not fit well in factor models. Nevertheless, the reported correlation and item-total correlation of Distress with other negative symptom domains and BNSS items were not negligible. Emotion deficit was also discussed as a part of negative symptoms conceptualization. As a single item may not be sufficient to represent an underlying construct that is potentially abstract and complex, the Schedule for the Deficit Syndrome (SDS) which comprises “Diminished Emotional Range” that is conceptually relevant to the BNSS Distress was employed. The study aimed to reexamine the conceptualization of negative symptoms by examining the model fit of several models when BNSS Distress and SDS Emotion (EMO) were included in the models using confirmatory factor analyses (CFA). Methods Two-hundred and seventy-four schizophrenia outpatients aged 21–65 were assessed on the BNSS and SDS. In the two-factor models, Restricted Affect, Diminished Emotional Range and Poverty of Speech in SDS and all items in BNSS Blunted Affect and Alogia subscales were regressed on EE, Curbing of Interests, Diminished Sense of Purpose and Diminished Social Drive in SDS and all items in BNSS Anhedonia, Asociality and Avolition subscales were regressed on MAP, without EMO, or with EMO regressed on either EE or MAP. Five-factor models and second-order five-factor models were examined, with or without EMO. Lastly, a six-factor model with EMO manifested by the sixth factor and second-order six-factor models in which EMO was regressed on either EE or MAP were tested. Root mean square error of approximation (RMSEA) &lt;0.08, comparative fit index (CFI) &gt;0.95, the Tucker-Lewis Index (TLI) &gt;0.95, and weighted root-mean-square residual (WRMR) &lt;1.0 indicate good model fit. CFAs were conducted using Mplus version 7.4. Results The two-factor models did not yield adequate fit indices. Five-factor model and second-order five-factor model without EMO had good model fit; five-factor model: RMSEA=0.056 (0.044–0.068), CFI=0.996, TFI=0.995, WRMR=0.718; second-order five-factor model: RMSEA=0.049 (0.036–0.061), CFI=0.997, TFI=0.996, WRMR=0.758. When EMO was included as indicator in one of the factors in the five-factor models, only the model in which EMO was regressed on Alogia yielded adequate fit. Similarly, in the second-order five-factor models, adequate fit indices were observed only when EMO was regressed on Alogia and Blunted Affect. The six-factor model fitted the data well, RMSEA=0.053 (0.042–0.064), CFI=0.996, TFI=0.995, WRMR=0.711. Second-order six-factor model with EMO regressed on EE yielded better model fit than MAP, RMSEA=0.050 (0.039–0.061), CFI=0.996, TFI=0.995, WRMR=0.849. Discussion In line with previous studies, five-factor and second-order five-factor models without EMO fitted the data well. When EMO was included, a six-factor model and a second-order six-factor model in which the sixth factor was regressed on EE showed good model fit. Emotion, motivation and behavior are intertwined. Our results showed that diminished emotion may also be one of the components of negative symptoms, which had higher association with EE than MAP.


2019 ◽  
Vol 50 (1) ◽  
pp. 222-242 ◽  
Author(s):  
Jane McPherson ◽  
Neil Abell

Abstract This article introduces a measurable framework for rights-based social work practice and an accompanying set of instruments, the Human Rights Methods in Social Work (HRMSW) scales: (i) ‘participation’, (ii) ‘non-discrimination’, (iii) ‘strengths perspective’, (iv) ‘micro/macro integration’, (v) ‘capacity-building’, (vi) ‘community and interdisciplinary collaboration’, (vii) ‘activism’ and (viii) ‘accountability’. These scales, designed for use by researchers, educators and practitioners, are the first to measure social workers’ use of rights-based methods. An electronic survey was used to collect data from a convenience sample of 1,014 licensed US social workers, and a confirmatory factor analysis was used to validate the scales’ psychometric properties. A respecified model using eight error covariances fit the data (χ2/df ratio = 2.9; comparative fit index (CFI) = 0.91; tucker lewis index (TLI) = 0.90; root mean square error of approximation (RMSEA) = 0.04; standardized root mean square residual (SRMR) = 0.07). Thus, factor analysis yielded a set of eight related scales—collectively called the HRMSW—each measuring a different human rights practice method that social workers can use to promote human dignity and the rights-based principles of participation, accountability and non-discrimination. Scholars argue that ‘human rights’ are a more appropriate yardstick for measuring the impact of social work intervention rather than our traditional aim of social justice; the HRMSW scales can help us begin to test this proposition.


2019 ◽  
Vol 11 (10) ◽  
pp. 1211 ◽  
Author(s):  
Fardin Seifi ◽  
Xiaoli Deng ◽  
Ole Baltazar Andersen

The latest satellite and in situ data are a fundamental source for tidal model evaluations. In this work, the satellite missions TOPEX/Poseidon, Jason-1, Jason-2 and Sentinel-3A, together with tide gauge data, were used to investigate the performance of recent regional and global tidal models over the Great Barrier Reef, Australia. Ten models, namely, TPXO8, TPXO9, EOT11a, HAMTIDE, FES2012, FES2014, OSUNA, OSU12, GOT 4.10 and DTU10, were considered. The accuracy of eight major tidal constituents (i.e., K1, O1, P1, Q1, M2, S2, N2 and K2) and one shallow water constituent (M4) were assessed based on the analysis of sea-level observations from coastal tide gauges and altimetry data (TOPEX series). The outcome was compared for four different subregions, namely, the coastline, coastal, shelf and deep ocean zones. Sea-level anomaly data from the Sentinel-3A mission were corrected using the tidal heights predicted by each model. The root mean square values of the sea level anomalies were then compared. According to the results, FES2012 compares more favorably to other models with root mean square (RMS) values of 10.9 cm and 7.7 cm over the coastal and shelf zones, respectively. In the deeper sections, the FES2014 model compares favorably at 7.5 cm. In addition, the impact of sudden fluctuations in bottom topography on model performances suggest that a combination of bathymetric variations and proximity to the coast or islands contributes to tidal height prediction accuracies of the models.


2016 ◽  
Vol 77 (1) ◽  
pp. 5-31 ◽  
Author(s):  
Hsien-Yuan Hsu ◽  
Jr-Hung Lin ◽  
Oi-Man Kwok ◽  
Sandra Acosta ◽  
Victor Willson

Several researchers have recommended that level-specific fit indices should be applied to detect the lack of model fit at any level in multilevel structural equation models. Although we concur with their view, we note that these studies did not sufficiently consider the impact of intraclass correlation (ICC) on the performance of level-specific fit indices. Our study proposed to fill this gap in the methodological literature. A Monte Carlo study was conducted to investigate the performance of (a) level-specific fit indices derived by a partially saturated model method (e.g., [Formula: see text] and [Formula: see text]) and (b) [Formula: see text] and [Formula: see text] in terms of their performance in multilevel structural equation models across varying ICCs. The design factors included intraclass correlation (ICC: ICC1 = 0.091 to ICC6 = 0.500), numbers of groups in between-level models (NG: 50, 100, 200, and 1,000), group size (GS: 30, 50, and 100), and type of misspecification (no misspecification, between-level misspecification, and within-level misspecification). Our simulation findings raise a concern regarding the performance of between-level-specific partial saturated fit indices in low ICC conditions: the performances of both [Formula: see text] and [Formula: see text] were more influenced by ICC compared with [Formula: see text] and SRMRB. However, when traditional cutoff values ( RMSEA≤ 0.06; CFI, TLI≥ 0.95; SRMR≤ 0.08) were applied, [Formula: see text] and [Formula: see text] were still able to detect misspecified between-level models even when ICC was as low as 0.091 (ICC1). On the other hand, both [Formula: see text] and [Formula: see text] were not recommended under low ICC conditions.


2018 ◽  
Vol 61 (1) ◽  
pp. 77-92
Author(s):  
Eric Rakotoasimbola ◽  
Sam Blili

Using the Monte Carlo simulation method, this study analyzes the impacts on fit indices by the degree of nonnormality of variables, the sample size, and the choice of estimation method. To address these issues, we use the causal model of consumer involvement as elaborated by Mittal and Lee. Results of this study show that adjusted goodness of fit index (AGFI) and goodness of fit index (GFI) are subject to variation in sample size, and their use requires a sample size of at least 300 observations to be reliable. Comparative fit index (CFI) and root mean square error of approximation (RSMEA) are more reliable with the generalized least squares (GLS) compared with maximum likelihood estimation (MLE) method under different settings of sample size and degree of nonnormality. Finally, for the standardized root mean square residual (SRMR), it is recommended that it is used with the MLE method. This study provides prescriptions for the choice of fit indices and the requirements of sample size and estimation method to test the causal model of consumer involvement. The method used here can be extended to any model before fitting it to real data. It helps researchers to prevent conflictual results regarding the choice of fit indices.


Sign in / Sign up

Export Citation Format

Share Document