Consequences of Not Conducting Measurement Invariance Tests in Cross-Cultural Studies: A Review of Current Research Practices and Recommendations

2019 ◽  
Vol 21 (4) ◽  
pp. 466-483
Author(s):  
Shinhee Jeong ◽  
Yunsoo Lee

The Problem Cross-cultural research has received substantial attention from both academia and practice as it contributes to expand current theory and implements culturally successful human resource strategies. Although the quantity of this type of research has increased, several researchers have raised methodological concerns that the majority of cross-cultural research has simply assumed or ignored measurement invariance. The Solution In this article, we first provide the meaning for measurement invariance, discuss why it is important, and then explain stepwise confirmatory factor analysis procedures to test measurement invariance. We also diagnose the current research practice in the field of human resource development (HRD) based on a review of cross-cultural, comparative research published in the major HRD journals. Finally, we demonstrate that the group difference test results that have been found without ensuring measurement invariance can, in fact, be false. The Stakeholders This article contributes to the HRD literature and practice in two ways. First, HRD researchers are invited to recognize the importance of sophisticated research methodology, such as measurement invariance, and to examine item bias across different groups so they can make a meaningful and valid comparison. The same attention is advisable to any practitioner who attempts to identify group differences using multinational/cultural data. This article also provides HRD scholars and practitioners with specific multigroup confirmatory factor analysis (MGCFA) procedures to facilitate empirical tests of measurement models across different groups and thus disseminate the methodological advances in the field of HRD. It is our hope that the present article raises awareness, circulates relevant knowledge, and encourages more HRD scholars to think critically about measurement.

2010 ◽  
Vol 3 (1) ◽  
pp. 111-130 ◽  
Author(s):  
Taciano L. Milfont ◽  
Ronald Fischer

Researchers often compare groups of individuals on psychological variables. When comparing groups an assumption is made that the instrument measures the same psychological construct in all groups. If this assumption holds, the comparisons are valid and differences/similarities between groups can be meaningfully interpreted. If this assumption does not hold, comparisons and interpretations are not fully meaningful. The establishment of measurement invariance is a prerequisite for meaningful comparisons across groups. This paper first reviews the importance of equivalence in psychological research, and then the main theoretical and methodological issues regarding measurement invariance within the framework of confirmatory factor analysis. A step-by-step empirical example of measurement invariance testing is provided along with syntax examples for fitting such models in LISREL.


Author(s):  
Thanh V. Tran ◽  
Tam Nguyen ◽  
Keith Chan

A cross-cultural comparison can be misleading for two reasons: (1) comparison is made using different attributes and (2) comparison is made using different scale units. This chapter illustrates multiple statistical approaches to evaluating the cross-cultural equivalence of the research instruments: data distribution of the items of the research instrument, the patterns of responses of each item, the corrected item–total correlation, exploratory factor analysis (EFA), confirmatory factor analysis (CFA), and reliability analysis using the parallel test and tau-equivalence test. Equivalence is the fundamental issue in cross-cultural research and evaluation.


2017 ◽  
Vol 48 (4) ◽  
pp. 560-576 ◽  
Author(s):  
Seulki Jang ◽  
Eun Sook Kim ◽  
Chunhua Cao ◽  
Tammy D. Allen ◽  
Cary L. Cooper ◽  
...  

The Satisfaction With Life Scale (SWLS) is a commonly used life satisfaction scale. Cross-cultural researchers use SWLS to compare mean scores of life satisfaction across countries. Despite the wide use of SWLS in cross-cultural studies, measurement invariance of SWLS has rarely been investigated, and previous studies showed inconsistent findings. Therefore, we examined the measurement invariance of SWLS with samples collected from 26 countries. To test measurement invariance, we utilized three measurement invariance techniques: (a) multigroup confirmatory factor analysis (MG-CFA), (b) multilevel confirmatory factor analysis (ML-CFA), and (c) alignment optimization methods. The three methods demonstrated that configural and metric invariances of life satisfaction held across 26 countries, whereas scalar invariance did not. With partial invariance testing, we identified that the intercepts of Items 2, 4, and 5 were noninvariant. Based on two invariant intercepts, factor means of countries were compared. Chile showed the highest factor mean; Spain and Bulgaria showed the lowest. The findings enhance our understanding of life satisfaction across countries, and they provide researchers and practitioners with practical guidance on how to conduct measurement invariance testing across countries.


Author(s):  
Marlene Mußotter

AbstractThe measurement models for both nationalism and patriotism originally developed by Blank and Schmidt are broadly established. Despite their widespread usage in Germany and beyond, concerns have been voiced about the operationalisation of these nation-related concepts. However, in previous scholarship little attention has been devoted to systematically reviewing the models’ validity. This paper’s major goal is to contribute substantially to research on operationalising national attitudes by thoroughly examining how both nationalism and patriotism are measured and how valid the predominant measurement models really are. By running a confirmatory factor analysis, three measurement models based on the ISSP data of 2003 are replicated and empirically reviewed. By conducting a single-country analysis, the models are tested for the German case, including the evaluation of measurement invariance for both Eastern and Western Germany. Although the selected measurement models yield satisfying results, the paper identifies considerable shortcomings with regard to the way both nationalism and patriotism are empirically approached. It calls for a reconceptualising of the prevailing concept of pride and thus challenges the predominant operationalisation.


2018 ◽  
Vol 34 (2) ◽  
pp. 87-100 ◽  
Author(s):  
Gino Casale ◽  
Robert J. Volpe ◽  
Brian Daniels ◽  
Thomas Hennemann ◽  
Amy M. Briesch ◽  
...  

Abstract. The current study examines the item and scalar equivalence of an abbreviated school-based universal screener that was cross-culturally translated and adapted from English into German. The instrument was designed to assess student behavior problems that impact classroom learning. Participants were 1,346 K-6 grade students from the US (n = 390, Mage = 9.23, 38.5% female) and Germany (n = 956, Mage = 8.04, 40.1% female). Measurement invariance was tested by multigroup confirmatory factor analysis (CFA) across students from the US and Germany. Results support full scalar invariance between students from the US and Germany (df = 266, χ2 = 790.141, Δχ2 = 6.9, p < .001, CFI = 0.976, ΔCFI = 0.000, RMSEA = 0.052, ΔRMSEA = −0.003) indicating that the factor structure, the factor loadings, and the item thresholds are comparable across samples. This finding implies that a full cross-cultural comparison including latent factor means and structural coefficients between the US and the German version of the abbreviated screener is possible. Therefore, the tool can be used in German schools as well as for cross-cultural research purposes between the US and Germany.


2021 ◽  
pp. 003329412110051
Author(s):  
Cecilia Brando-Garrido ◽  
Javier Montes-Hidalgo ◽  
Joaquín T. Limonero ◽  
María J. Gómez-Romero ◽  
Joaquín Tomás-Sábado

A recent line of research concerns bedtime procrastination, its effects on sleep quality and duration, and the associated repercussions for health and wellbeing. The Bedtime Procrastination Scale is a brief, self-report instrument developed by Kroese et al. with the aim of evaluating this behavior and exploring its association with insufficient sleep, and hence with health. The aim was to develop and validate a Spanish version of the Bedtime Procrastination Scale (BPS-Sp) and to examine the relationship between bedtime procrastination and both general procrastination and self-control. The original BPS was translated from English into Spanish in accordance with international guidelines on the cross-cultural adaptation of measurement instruments. The sample for the validation study comprised 177 nursing students who completed a questionnaire requesting demographic data and which included the following instruments: the newly developed BPS-Sp, the Tuckman Procrastination Scale, and the Brief Self-Control Scale. Statistical analysis involved tests of normality (Kolmogorov-Smirnov), reliability (Cronbach’s alpha, test-retest), construct validity, and confirmatory factor analysis. Scores on the BPS-Sp showed excellent internal consistency (α = .83) and temporal stability (test-retest r = .84), as well as significant correlations with general procrastination ( r = .26; p < .01) and self-control ( r = −.17; p < .05). Confirmatory factor analysis showed an adequate fit for the single-factor solution proposed by Kroese et al. The results suggest that the BPS-Sp is a valid and reliable instrument for assessing bedtime procrastination in the Spanish-speaking population.


Sign in / Sign up

Export Citation Format

Share Document