Assessing measurement equivalence of the Danish and Dutch Four-Dimensional Symptom Questionnaire using differential item and test functioning analysis

Aims: The Dutch Four-Dimensional Symptom Questionnaire (4DSQ) measures distress, depression, anxiety and somatisation, facilitating the distinction between stress-related problems and psychiatric disorder in primary and occupational health care. The aim of the study was to examine the measurement equivalence across the Danish and Dutch 4DSQ. Methods: Danish 4DSQ data were obtained from a cohort of Danish citizens on sick leave for mental-health problems. Dutch 4DSQ data were obtained from a cohort of Dutch employees on sick leave and a cohort of general practice attenders suspected of having mental-health problems. The study samples were matched on age and sex. The 4DSQ scales were assessed for essential unidimensionality using confirmatory factor analysis. Measurement equivalence of the 4DSQ across the groups was assessed using differential item and test functioning (DIF and DTF) analysis. Results: The study groups each consisted of 1363 people (63% female, Mage=42 years). The 4DSQ scales proved essentially unidimensional. DIF was detected in 20 items. In terms of Cohen’s effect size, DIF was mostly small or moderate. In terms of effect size, the mean effect on the scale score (DTF) was negligible. Nevertheless, it is recommended to adjust some of the cut-off points for two Danish 4DSQ scales to retain the meaning of these cut-off points in Dutch respondents. Conclusions: The Danish version of the 4DSQ measures the same constructs as the original Dutch questionnaire. Twenty items functioned differently in Danish respondents than in Dutch respondents, but this had only a small impact on the scale scores.

On the design and validation of a rubric for the evaluation of performance in a musical contest

Evaluation of music performance in competitive contexts often produces discrepancies between the expert judges. These discrepancies can be reduced by using appropriate rubrics that minimise the differences between judges. The objective of this study was the design and validation of an analytical evaluation rubric, which would allow the most objective evaluation possible of a musical solo performance in a regulated official competition. A panel of three experts created an analytical rubric made up of five review criteria and three scoring levels, together with their respective indicators. To validate the rubric, two independent panels of judges used it to score a sample of recordings. An examination was made of the dimensionality, sources of error, inter-rater reliability and internal consistency of the scores coming from the experts. The essential unidimensionality of the rubric was confirmed. No differential effects between raters were found, nor were significant differences seen in each rater’s internal consistency. The use of a rubric as tool for evaluating music performance in a competitive context has positive effects, improving reliability and objectivity of the results, both in terms of intra-rater consistency and agreement between raters.

Dimensionality assessment of binary response test items: A non-parametric approach of bayesian item response theory measurement

This study investigated dimensionality of Binary Response Items through a non-parametric technique of Item Response Theory measurement framework. The study used causal comparative research type of non-experimental design. The sample consisted of 5,076 public senior secondary school examinees (SSS3) between the age of 14-16 years from 45 schools, which were drawn randomly from three senatorial districts of Osun State, Nigeria. Instrument used for this study was 2018 Osun State unified multiple-choice mathematics achievement test items with empirical reliability coefficient of 0.82. Data obtained were analysed using Non-linear factor analysis, Stout’s Test of Essential Unidimensionality (STEU), Factor Analysis (FA), Full Information Factor Analysis (FIFA) and Bootstrap Modified Parallel Analysis Test (BMPAT). Results showed that both the BMPAT and STEU ascertained violation of unidimensionality assumption of the test items (the observed difference in the second eigenvalue of the observed data and that of second eigenvalue of the simulated data was statistically significant, p = 0.0099; Stout’s test rejected the assumption of essential unidimensionality, T = 10.6260, p<0.05). Non-linear factor analysis and full information factor analysis revealed that four dimensions embedded in the test items and loadings of the items showed within-item multidimensionality respectively. The authors’ concluded that modeling examinees’ performance with unidimensional model when it was actually multidimensional in nature would affect performance of examinees adversely and could lead to blur judgment. Consequently, it is recommended that unidimensional scoring method of Osun State unified mathematics achievement test implicit in Classical Test Theory should be jettisoned and an appropriate scoring model (multidimensional) should be embraced.

Searching for G: A New Evaluation of SPM-LS Dimensionality

There has been increased interest in assessing the quality and usefulness of short versions of the Raven’s Progressive Matrices. A recent proposal, composed of the last twelve matrices of the Standard Progressive Matrices (SPM-LS), has been depicted as a valid measure of g. Nonetheless, the results provided in the initial validation questioned the assumption of essential unidimensionality for SPM-LS scores. We tested this hypothesis through two different statistical techniques. Firstly, we applied exploratory graph analysis to assess SPM-LS dimensionality. Secondly, exploratory bi-factor modelling was employed to understand the extent that potential specific factors represent significant sources of variance after a general factor has been considered. Results evidenced that if modelled appropriately, SPM-LS scores are essentially unidimensional, and that constitute a reliable measure of g. However, an additional specific factor was systematically identified for the last six items of the test. The implications of such findings for future work on the SPM-LS are discussed.

Development of a Short-Form of the Medication Management Test: Evaluation of Dimensionality, Reliability, Information and Measurement Equivalence Using Latent Variable Models

Background and Purpose: The Medication Management Test (MMT) measures higher cognitive functioning. The aim of the analyses presented was to reduce assessment burden by developing a short-form version, and describe its psychometric properties. Methods: Factor analyses, item response theory (IRT), and differential item functioning (DIF) were performed to examine the dimensionality, reliability information, and measurement equivalence. Results: The ratio of the first two extracted eigenvalues from the exploratory principal component analysis was 7.62, indicating essential unidimensionality. Although one item “needs prompting for pill regime” evidenced DIF above the threshold for education and race/ethnicity, the magnitude was relatively small and the impact minimal. IRT-based reliability estimates were high (>0.80) across all subgroups. Conclusions: Because medication management is an important task associated with independent living, it is critical to assess whether medications can be self-administered safely.

Factor Structure of the Maslach Burnout Inventory–Educators Survey in a Polish-Speaking Sample

This article investigates the psychometric properties of a mainstream burnout measure dedicated to teachers: the Maslach Burnout Inventory–Educators Survey (MBI-ES). The study used data gathered from a random sample of 1,206 primary school teachers in Poland to verify the construct validity of the MBI-ES. Eight alternative measurement models suggested in the literature were tested using confirmatory factor analysis. Contrary to many previous studies, this study did not support the oblique three-factor structure of the MBI-ES. A bifactor model with one general Burnout factor and three specific orthogonal factors of personal accomplishment, depersonalization, and emotional exhaustion showed best fit to the data. Additional analyses supported the measure’s essential unidimensionality. The results yield theoretical implications for construct reconceptualization and practical guidelines for researchers and practitioners.

Validating the Polish Adaptation of the 11-Item De Jong Gierveld Loneliness Scale

The aim of this study was to translate into Polish and establish the psychometric properties of the 11-item De Jong Gierveld Loneliness Scale (DJGLS). The translation process followed the recent guidelines for the crosscultural adaptation of questionnaires and was tested through analysis of differential item functioning (DIF) by the use of the Poly-SIBTEST method and bilingual groups. The essential unidimensionality was checked by bifactor analysis. Reliability was assessed by internal consistency and homogeneity analysis, and external construct validity by correlation with several external scales. The research indicated no differences in item performance between the final Polish and English versions, and confirmed the earlier findings indicating that the DJGLS measures two dimensions of loneliness (social and emotional) which generalize into a higher-order factor of a general sense of loneliness (bifactor structure). Reliability (α = .89) and homogeneity (H = .47) proved to be high. Research showed that the instrument has a satisfactory validity criterion: Correlation with the UCLA Loneliness Scale r = .82; with Rosenberg’s Self-Esteem Scale r = –.56; and with the Beck Depression Inventory r = .46 (all p < .01). The Polish adaptation of the DJGLS thus presents a bifactor structure, with good levels of internal consistency, homogeneity, and construct validity.