Random Permutation Testing Applied to Measurement Invariance Testing with Ordered-Categorical Indicators

Multi-item ordered categorical scales and structural equation modelling approaches are often used in panel research for the analysis of latent variables over time. The accuracy of such models depends on the assumption of longitudinal measurement invariance (LMI), which states that repeatedly measured latent variables should effectively represent the same construct in the same metric at each time point. Previous research has widely contributed to the LMI literature for continuous variables, but these findings might not be generalized to ordered categorical data. Treating ordered categorical data as continuous contradicts the assumption of multivariate normality and could potentially produce inaccuracies and distortions in both invariance testing results and structural parameter estimates. However, there is still little research that examines and compares criteria for establishing LMI with ordinal categorical data. Drawing on this lack of evidence, the present chapter offers a detailed description of the main procedures used to test for LMI with ordered categorical variables, accompanied by examples of their practical application in a two-wave longitudinal survey administered to 1,912 Italian middle school teachers. The empirical study evaluates whether different testing procedures, when applied to ordered categorical data, lead to similar conclusions about model fit, invariance, and structural parameters over time.

Download Full-text

The Factor Structure and Measurement Invariance of Positive and Negative Affect

European Journal of Psychological Assessment ◽

10.1027/1015-5759/a000252 ◽

2016 ◽

Vol 32 (4) ◽

pp. 265-272 ◽

Cited By ~ 10

Author(s):

Mohsen Joshanloo ◽

Ali Bakhshi

Keyword(s):

Negative Affect ◽

Measurement Invariance ◽

Factor Structure ◽

Factor Model ◽

Positive And Negative Affect ◽

Cultural Groups ◽

Invariance Testing ◽

The Usa ◽

Scalar Invariance ◽

Gender Groups

Abstract. This study investigated the factor structure and measurement invariance of the Mroczek and Kolarz’s scales of positive and negative affect in Iran (N = 2,391) and the USA (N = 2,154), and across gender groups. The two-factor model of affect was supported across the groups. The results of measurement invariance testing confirmed full metric and partial scalar invariance of the scales across cultural groups, and full metric and full scalar invariance across gender groups. The results of latent mean analysis revealed that Iranians scored lower on positive affect and higher on negative affect than Americans. The analyses also showed that American men scored significantly lower than American women on negative affect. The significance and implications of the results are discussed.

Download Full-text

Measurement Invariance Testing Using Random Effects model for Many groups: Multilevel Confirmatory Factor Analysis (ML CFA) and Multilevel Factor Mixture Modeling (ML FMM)

THE KOREAN JOURNAL OF PSYCHOLOGY GENERAL ◽

10.22257/kjp.2019.6.38.2.185 ◽

2019 ◽

Vol 38 (2) ◽

pp. 185-218

Author(s):

Sookyung Son ◽

Hyojin Kim ◽

Sehee Hong

Keyword(s):

Factor Analysis ◽

Confirmatory Factor Analysis ◽

Measurement Invariance ◽

Random Effects ◽

Mixture Modeling ◽

Random Effects Model ◽

Invariance Testing ◽

Multilevel Confirmatory Factor Analysis ◽

Factor Mixture Modeling ◽

Confirmatory Factor

Download Full-text

Analyzing Measurement Invariance of the Students’ Engagement Instrument Brief Version: The Cases of Denmark, Finland, and Portugal

Canadian Journal of School Psychology ◽

10.1177/0829573517699333 ◽

2017 ◽

Vol 33 (4) ◽

pp. 297-313 ◽

Cited By ~ 9

Author(s):

T. E. Virtanen ◽

P. Moreira ◽

H. Ulvseth ◽

H. Andersson ◽

S. Tetler ◽

...

Keyword(s):

Student Engagement ◽

Measurement Invariance ◽

Cross Cultural ◽

School Contexts ◽

Invariance Testing ◽

National School ◽

Cultural Research ◽

Cross Cultural Research ◽

Different Levels ◽

Cross National

The promotion of students’ engagement with school is an internationally acknowledged challenge in education. There is a need to examine the structure of the concept of student engagement and to discover the best practices for fostering it across societies. That is why the cross-cultural invariance testing of students’ engagement measures is highly needed. This study aimed, first, to find the reduced set of theoretically valid items to represent students’ affective and cognitive engagement forming the Brief-SEI (brief version of the Student Engagement Instrument; SEI). The second aim was to test the measurement invariance of the Brief-SEI across three countries (Denmark, Finland, and Portugal). A total of 4,437 seventh-grade students completed the SEI questionnaires in the three countries. The analyses revealed that of the total 33 original instrument items, 15 items indicated acceptable psychometric properties of the Brief-SEI. With these 15 items, cross-national factorial validity and invariances across genders and students with different levels of academic performance (samples from Finland and Portugal) were demonstrated. This article discusses the utility of the Brief-SEI in cross-cultural research and its applicability in different national school contexts.

Download Full-text

Measurement Invariance Testing of the Patient Health Questionnaire-9 (PHQ-9) across People with and without Diabetes Mellitus from the NHANES, EHMS and UK Biobank datasets

Journal of Affective Disorders ◽

10.1016/j.jad.2021.05.031 ◽

2021 ◽

Author(s):

Arie Nouwen ◽

Sonya S. Deschênes ◽

Zhanna Balkhiyarova ◽

Juan R. Albertorio-Díaz ◽

Inga Prokopenko ◽

...

Keyword(s):

Diabetes Mellitus ◽

Measurement Invariance ◽

Patient Health Questionnaire ◽

Health Questionnaire ◽

Uk Biobank ◽

Invariance Testing ◽

Patient Health

Download Full-text

Scale Development and Factor Analysis

Advances in Library and Information Science - Scholarly Publishing and Research Methods Across Disciplines ◽

10.4018/978-1-5225-7730-0.ch008 ◽

2018 ◽

pp. 159-183

Author(s):

Lihua Xu

Keyword(s):

Scale Development ◽

Stereotype Threat ◽

Measurement Invariance ◽

Leadership Style ◽

Analytical Methods ◽

The United States ◽

Invariance Testing ◽

Testing Procedures ◽

Confirmatory Factor ◽

The Common

Scale development is an important step in empirical research. This chapter describes the common procedures to follow in scale development with essential factor analytical methods. The concept of measurement invariance, the importance of its testing prior to group comparisons, and testing procedures are discussed. Single-group, multi-group, and hierarchical confirmatory factor analytical methods and associated decision making are described. Procedural steps in scale development and measurement invariance testing are illustrated at length using a real dataset in stereotype threat and principals' leadership style in the United States.

Download Full-text

Cross-cultural examination of the Big Five Personality Trait Short Questionnaire: Measurement invariance testing and associations with mental health

PLoS ONE ◽

10.1371/journal.pone.0226223 ◽

2019 ◽

Vol 14 (12) ◽

pp. e0226223 ◽

Cited By ~ 1

Author(s):

Laura Mezquita ◽

Adrian J. Bravo ◽

Julien Morizot ◽

Angelina Pilatti ◽

Matthew R. Pearson ◽

...

Keyword(s):

Mental Health ◽

Measurement Invariance ◽

Big Five ◽

Personality Trait ◽

Cross Cultural ◽

Big Five Personality ◽

Invariance Testing ◽

Short Questionnaire

Download Full-text

Response Shift After a Mindfulness-Based Intervention: Measurement Invariance Testing of the Comprehensive Inventory of Mindfulness Experiences

Mindfulness ◽

10.1007/s12671-017-0764-4 ◽

2017 ◽

Vol 9 (1) ◽

pp. 212-220 ◽

Cited By ~ 6

Author(s):

Christian U. Krägeloh ◽

Claudia Bergomi ◽

Richard J. Siegert ◽

Oleg N. Medvedev

Keyword(s):

Measurement Invariance ◽

Response Shift ◽

Invariance Testing

Download Full-text

Multigroup CFA and alignment approaches for testing measurement invariance and factor score estimation: Illustration with the schoolwork-related anxiety survey across countries and gender

Methodology ◽

10.5964/meth.2281 ◽

2021 ◽

Vol 17 (1) ◽

pp. 22-38

Author(s):

Jason C. Immekus

Keyword(s):

Measurement Invariance ◽

Large Scale ◽

International Student ◽

Student Assessment ◽

International Studies ◽

Invariance Testing ◽

International Student Assessment ◽

Item Parameters ◽

And Gender ◽

Gender Groups

Within large-scale international studies, the utility of survey scores to yield meaningful comparative data hinges on the degree to which their item parameters demonstrate measurement invariance (MI) across compared groups (e.g., culture). To-date, methodological challenges have restricted the ability to test the measurement invariance of item parameters of these instruments in the presence of many groups (e.g., countries). This study compares multigroup confirmatory factor analysis (MGCFA) and alignment method to investigate the MI of the schoolwork-related anxiety survey across gender groups within the 35 Organisation for Economic Co-operation and Development (OECD) countries (gender × country) of the Programme for International Student Assessment 2015 study. Subsequently, the predictive validity of MGCFA and alignment-based factor scores for subsequent mathematics achievement are examined. Considerations related to invariance testing of noncognitive instruments with many groups are discussed.

Download Full-text

Measuring Physical Intimate Partner Violence to Monitor Sustainable Development Goal 5: A Global Psychometric Assessment

10.1101/2021.07.01.21259594 ◽

2021 ◽

Author(s):

Kathryn M Yount ◽

Yuk Fai Cheong ◽

Zara Khan ◽

Irina Bergenfeld ◽

Nadine Kaslow ◽

...

Keyword(s):

Sustainable Development ◽

Measurement Invariance ◽

Demographic And Health Surveys ◽

Model Parameters ◽

Sustainable Development Goal ◽

Development Goal ◽

Invariance Testing ◽

Multiple Group ◽

Approximate Measurement ◽

Cross National

Background. One third of women experience IPV and potential sequelae. Sustainable Development Goal (SDG) 5.2, to eliminate all violence against women, including IPV, compels national governments to monitor such violence. We conducted the first global measurement-invariance assessment of standardized physical IPV items. Methods. Thirty-six Demographic and Health Surveys (DHS) from 36 Lower-/Middle-Income Countries (LMICs) administering the same 18 IPV items during 2012-2018 were included. We performed exploratory and confirmatory factor analyses (EFA/CFA) with seven physical IPV items, which are the most behaviorally specific and reliable. Datasets meeting EFA/CFA model fit criteria (loadings>.35, RMSEA<.08, CFI/TLI>.95) were included in multiple-group CFA to test strict measurement invariance, and in alignment optimization (AO) to test approximate measurement invariance. We compared national rankings based on AO-derived scores and lifetime physical IPV prevalences, and correlated AO-dervied scores with physical, sexual, and psychological IPV prevalences. Results. Estimated lifetime physical IPV varied widely (5.6%-50.5%). All loadings and fit statistics met thresholds in country-specific EFA/CFAs. A unidimensional, seven-item physical IPV construct lacked scalar invariance in multiple-group CFA but achieved approximate measurement invariance in AO analysis, as 12.3% (<25%) of model parameters were non-invariant. National rankings of AO-derived scores and estimated physical IPV prevalences were similarly distributed, but national estimates often were not significantly different, so grouped score ranges or prevalence ranges are advised. Three items (slap, twist, choke) warrant cognitive testing to improve their psychometric performance. Correlations of AO-derived scores with IPV prevalences ranged from .48 to .66. Conclusions. Seven DHS physical-IPV items were approximately invariant across 36 LMICs spanning five regions and are reasonable for cross-national, grouped comparison of physical IPV. Measurement-invariance testing over time will inform their utility to monitor SDG5.2.1; cross-national and cross-time measurement-invariance testing of other IPV item sets is warranted.

Download Full-text