Psychometric Validation of the Dutch Version of the Promotive and Prohibitive Voice Scale

The aim of this three-study report was to validate the Dutch version of the promotive and prohibitive voice scale and to further embed the constructs of promotive and prohibitive voice within their nomological network. Promotive voice refers to the expression of suggestions for improving work practices, whereas prohibitive voice refers to the expression of concerns about practices and behaviors that are detrimental. In Study 1 (N = 121), confirmatory factor analyses (CFAs) provided evidence for the two-factor structure, which was replicated in the other two studies. In Study 2 (NT1 = 209/NT2 = 107), we investigated the convergent and discriminant validity of the promotive and prohibitive voice scale, and tested measurement invariance across gender and time. Results provided validity evidence, partial scalar invariance for gender, and scalar invariance across time. In Study 3 (N = 149), we expanded the nomological network of the promotive and prohibitive voice scales through their relationship with personal initiative, approach temperament, and risk propensity. Taken together, our results provide strong evidence for the validity of the Dutch version of the promotive and prohibitive voice scale.

Are my measures comparable across groups? A unified item- and scale-score-level approach to quantifying scalar non-invariance bias

When scalar invariance does not hold, which is often the case in application scenarios, the amount of non-invariance bias may either be consequential for observed mean comparisons or not. So far, only a few attempts have been made to quantify the extent of bias due to measurement non-invariance. Building on Millsap and Olivera-Aguilar (2012), we derived a new effect size measure, called Measurement Invariance Violation Index (MIVI), from first principles. MIVI merely assumes partial scalar invariance for a set of items forming a scale and quantifies the intercept difference of one non-invariant item (at the item-score level) or several non-invariant items (at the scale-score level) as the share (i.e., proportion) of the total observed scale score difference between groups. One can inspect the cancelation effects of item bias at the scale-score level when using directional instead of absolute terms. We provide computational code and exemplify MIVI in simulated contexts.

Estimating measurement equivalence of the 12-item General Health Questionnaire across ethnic groups in the UK

Background This study investigates the extent to which the GHQ-12 exhibits configural, metric and scalar invariance across six ethnic groups in Britain and Northern Ireland, using the UK Household Longitudinal Study (N = 35 410). Methods A confirmatory factor analysis was carried out on a white British group in order to establish an adequate measurement model. Secondly, a multi-group confirmatory factor analysis was conducted in order to assess measurement invariance. A sensitivity analysis comparing summated and latent means across groups was carried out. Finally, revised estimates of scale reliability were derived using two different methods. Results A one-factor model including correlated error terms on the negatively phrased items showed superior fit in all ethnic groups. Tests for equal factor loadings and intercepts also showed adequate fit demonstrating metric and scalar invariance. Latent and summated scale estimates of mean group differences were similar for all groups. Scale reliability using McDonald's ω is lower than when using the more conventional Cronbach's α. Reliability across groups is reasonably consistent. Conclusions We find that the GHQ-12 does not display obvious bias in regard to ethnic groups in the UK and that valid comparisons across these groups can be made for the purposes of population research. Caution is needed when using as a screening tool for individuals.

Causal attributions for success and failure among athletes: Validation of the Croatian version of the revised Causal dimension scale (CDS-II)

Background and Study Aim. Researchers since the late 1970s have been interested in finding out the reasons attributed to outcomes. To facilitate attribution research in Croatia we translated and validated The Revised Causal Dimension Scale (CDS-II) and examined its invariance when attributing most and least successful competition performances. Materials and methods. To achieve our stated aim, 384 kinesiology students completed the translated CDS-II. To test the latent structure of the questionnaire, we used CFA and tested two alternative models (orthogonal solution and model with correlated latent variables). Additionally, we examined the CDS-II invariance when attributing the most and the least successful competition performance in sport using longitudinal CFA. The reliability was tested using Cronbach alpha coefficients. Lastly, we tested differences in latent means between most and least successful performance using pairwise t-test. Results. Similar to the originally published findings, CFA indicated the CDS-II structure with correlated latent variables had an adequate and better fit than the orthogonal solution in both situations. Furthermore, we confirmed configural, metric and scalar invariance, as well as partial strict invariance since one item’s residuals differed significantly from the others. Cronbach alpha coefficients were adequate across both situations. Lastly, athletes attributed their most successful performances to more internal, stable and controllable reasons than their least successful performances. Conclusions. We confirmed that the Croatian version of the CDS-II has adequate psychometric properties and is therefore suitable for research in sport situations.

Factor structure and sex invariance of the temporal experience of pleasure scale (TEPS) in Chinese university students and clinical population

Background A motivation dimension of the core psychiatric symptom anhedonia additional has been suggested. The Temporal Experience of Pleasure Scale (TEPS) has been reported to assess anticipatory and consummatory pleasure separately in multiple factor-structure models. This study explored the factor structure of a Chinese version of the 18-item TEPS and further explored the measurement invariance of the TEPS across sex and clinical status (non-clinical, psychiatric). Methods Best-fit factor structure of the TEPS was examined in a non-clinical cohort of 7410 undergraduates, randomized into sample 1 (N = 3755) for exploratory factor analysis (EFA) and sample 2 (N = 3663) for confirmatory factor analysis (CFA). Additionally, serial CFA was conducted to evaluate measurement invariance across sex and between clinical (N = 313) and non-clinical (N = 341) samples. Results EFA supported a new four-factor structure with a motivation component, based on the original two-factor model (consummatory pleasure with/without motivation drive, anticipatory pleasure with/without motivation drive). CFA confirmed the four-factor model as the best-fit structure and revealed a second-order hierarchy in non-clinical and clinical samples. Full scalar invariance was observed across clinical and non-clinical samples and across sex in the clinical sample; only partial scalar invariance was observed across sex in the non-clinical sample. Conclusions A four-factor structured TEPS can assess motivation-driving dimensions of anticipatory and consummatory pleasure, consistent with the recently advanced multidimensional structure of anhedonia. CFA and measurement invariance results support application of the TEPS for assessing motivation aspects of anhedonia.

Is Differential Noneffortful Responding Associated with Type I Error in Measurement Invariance Testing?

Low test-taking effort as a validity threat is common when examinees perceive an assessment context to have minimal personal value. Prior research has shown that in such contexts subgroups may differ in their effort, which raises two concerns when making subgroup mean comparisons. First, it is unclear how differential effort could influence evaluations of scale property equivalence. Second, if attaining full scalar invariance, the degree to which differential effort can bias subgroup mean comparisons is unknown. To address these issues, a simulation study was conducted to examine the influence of differential noneffortful responding (NER) on evaluations of measurement invariance and latent mean comparisons. Results showed that as differential rates of NER grew, increased type I errors of measurement invariance were observed only at the metric invariance level, while no negative effects were apparent for configural or scalar invariance. When full scalar invariance was correctly attained, differential NER led to bias of mean score comparisons as large as 0.18 standard deviations with a differential NER rate of 7%. These findings suggest that test users should evaluate and document potential differential NER prior to both conducting measurement quality analyses and reporting disaggregated subgroup mean performance.

Is Differential Noneffortful Responding Associated With Type I Error in Measurement Invariance Testing?

Low test-taking effort as a validity threat is common when examinees perceive an assessment context to have minimal personal value. Prior research has shown that in such contexts, subgroups may differ in their effort, which raises two concerns when making subgroup mean comparisons. First, it is unclear how differential effort could influence evaluations of scale property equivalence. Second, if attaining full scalar invariance, the degree to which differential effort can bias subgroup mean comparisons is unknown. To address these issues, a simulation study was conducted to examine the influence of differential noneffortful responding (NER) on evaluations of measurement invariance and latent mean comparisons. Results showed that as differential rates of NER grew, increased Type I errors of measurement invariance were observed only at the metric invariance level, while no negative effects were apparent for configural or scalar invariance. When full scalar invariance was correctly attained, differential NER led to bias of mean score comparisons as large as 0.18 standard deviations with a differential NER rate of 7%. These findings suggest that test users should evaluate and document potential differential NER prior to both conducting measurement quality analyses and reporting disaggregated subgroup mean performance.

Is Differential Noneffortful Responding Associated with Type I Error in Measurement Invariance Testing?

Low test-taking effort as a validity threat is common when examinees perceive an assessment context to have minimal personal value. Prior research has shown that in such contexts subgroups may differ in their effort, which raises two concerns when making subgroup mean comparisons. First, it is unclear how differential effort could influence evaluations of scale property equivalence. Second, if attaining full scalar invariance, the degree to which differential effort can bias subgroup mean comparisons is unknown. To address these issues, a simulation study was conducted to examine the influence of differential noneffortful responding (NER) on evaluations of measurement invariance and latent mean comparisons. Results showed that as differential rates of NER grew, increased type I errors of measurement invariance were observed only at the metric invariance level, while no negative effects were apparent for configural or scalar invariance. When full scalar invariance was correctly attained, differential NER led to bias of mean score comparisons as large as 0.18 standard deviations with a differential NER rate of 7%. These findings suggest that test users should evaluate and document potential differential NER prior to both conducting measurement quality analyses and reporting disaggregated subgroup mean performance.

Measurement Invariance of Psychotic-Like Symptoms as Measured With the Prodromal Questionnaire, Brief Version (PQ-B) in Adolescent and Adult Population Samples

Valid measurement of group differences in self-reported psychotic-like experiences (PLEs) requires knowing any group-specific measurement properties of the instruments. We investigated the measurement invariance of the 21-item Prodromal Questionnaire–Brief (PQ-B) questionnaire across gender, ethnic minority/majority status, and presence of depressive symptoms in two different US non-clinical undergraduate samples (N = 1,099). For each item, endorsement of the experience and the associated distress were combined for analysis. A unidimensional model of the PQ-B fit the data well. Across genders, the PQ-B showed configural and metric, but not full scalar invariance; there were statistically significant differences in eight thresholds of six items, most being higher endorsement thresholds for self-identified females. Partial scalar invariance was also found for ethnic status, with five thresholds of three items being higher for the minority participants. For depressive symptomatology, defined as the top quintile by the Beck Depression Inventory–II, partial scalar invariance required dropping one item, after which there were statistically significant differences only in two response thresholds. Overall, a wide range of PLE questionnaire items were found to be robust to gender and ethnicity effects, strengthening confidence in found group differences in PLEs. Although full scalar invariance could not be ascertained for any of the group comparisons, the few found scalar differences across groups were small, with minimal impact on group PLE estimates. However, since PLEs are easily conceptually entangled with depression symptoms, similar items should be considered for exclusion if separable constructs are the target of investigation.

Measurement Invariance of the Multidimensional Scale of Perceived Social Support Among Chinese and South Asian Ethnic Minority Adolescents in Hong Kong

Seven hundred adolescents (Chinese = 50%; South Asian ethnic minority = 50%) with mean age of 15.3 years (SD = 1.53). Multigroup confirmatory factor analysis was performed to assess measurement invariance of the MSPSS scale across Chinese and South Asian ethnic minority samples. Results show that the original three-factor structure of the MSPSS was supported in both samples. Measurement invariance was supported in terms of configural, metric, and partial scalar invariance. Given partial scalar invariance was achieved, the latent mean differences were compared across samples. Chinese adolescents had higher levels of all three types of social support when compared with their South Asian ethnic minority counterparts. The present study attempts to assess the measurement invariance of the MSPSS across ethnic groups in Asian society, which sheds light on future research that involves perceived social support among adolescents in Chinese culture contexts.