Treating ordinal data: a comparison between rating scale and structural equation models

This article presents an empirical comparative assessment of the measurement quality of two instruments commonly used to measure fuzzy characteristics in computer-assisted questionnaires: a graphic scale (a line production scale using a slider bar) and an endecanary scale (a 0–10 rating scale using radio buttons). Data are analyzed by means of multitrait–multimethod models estimated as structural equation models with a mean and covariance structure. For the first time in such research, the results include bias, valid variance, method variance, and random error variance. The data are taken from a program that assesses entrepreneurial competences in undergraduate Economics and Business students by means of questionnaires administered on desktop computers. Neither of the measurement instruments was found to be biased with respect to the other, meaning that their scores are comparable. While both instruments achieve valid and reliable measurements, the reliability and validity are higher for the endecanary scale. This study contributes to the still scarce literature on fuzzy measurement instruments and on the comparability and relative merits of graphic and discrete rating scales on computer-assisted questionnaires.

Download Full-text

The Protective Role of Childcare Quality for Behavioral Adjustment in 3- to 5-Year-Old Children

Zeitschrift für Entwicklungspsychologie und Pädagogische Psychologie ◽

10.1026/0049-8637/a000162 ◽

2017 ◽

Vol 49 (1) ◽

pp. 1-10 ◽

Cited By ~ 2

Author(s):

Corina Wustmann Seiler ◽

Eva Müller ◽

Heidi Simoni

Keyword(s):

Structural Equation ◽

Structural Equation Models ◽

Rating Scale ◽

Risk Index ◽

Cumulative Risk ◽

Process Quality ◽

Family Risk Factors ◽

High Quality ◽

Family Risks

Abstract. This study examined the role of childcare process quality regarding the relation between family risks and preschoolers’ social–emotional problems. The study included 24 childcare centers with 42 groups in the German-speaking part of Switzerland. The 162 children in the sample were aged 3 – 5 years. Parents and teachers completed the Strengths and Difficulties Questionnaire (SDQ-Deu). Eight family risk factors were subsumed into a cumulative risk index. Childcare process quality was assessed by various observation instruments, for example, the Infant/Toddler Environment Rating Scale-Revised (ITERS-R) and the Early Childhood Environment Rating Scale-Revised (ECERS–R). The calculated structural equation models show that high-quality teaching and interaction, and provisions for learning, can mitigate the negative effects of family risks on children’s internalizing problems. High process quality can provide the chance of considerable attention, encouragement, and new learning opportunities for children at risk; these high-quality centers thereby contribute to protective processes.

Download Full-text

The Harmonization of Four Delirium Instruments

Innovation in Aging ◽

10.1093/geroni/igaa057.1678 ◽

2020 ◽

Vol 4 (Supplement_1) ◽

pp. 520-520

Author(s):

Benjamin Helfand ◽

Elke Detroyer ◽

Koen Milisen ◽

Dimitrios Adamis ◽

Richard Jones

Keyword(s):

Structural Equation ◽

Structural Equation Models ◽

Rating Scale ◽

Confusion Assessment Method ◽

Assessment Method ◽

Clinical Syndrome ◽

Kappa Statistics ◽

Highly Correlated ◽

Sum Scores ◽

Meta Analyses

Abstract Delirium is a clinical syndrome characterized by acute cognitive dysfunction, which is pervasive in older persons. Delirium affects over 2.6 million Americans over age 65 annually. One major problem in detection of delirium is that over 40 different instruments have been created to identify delirium in different clinical settings. There is no single agreed upon reference standard instrument. In previous work, we performed a systematic review to identify the four most commonly cited and well-validated instruments for delirium identification. The aim of this study is to harmonize these four commonly used instruments: Confusion Assessment Method (CAM), Delirium Observation Screening Scale (DOSS), Delirium Rating Scale-Revised-98 (DRS-R-98), and Memorial Delirium Assessment Scale (MDAS). We used data from three separate sources (N = 1623). Participants were rated by multiple and overlapping instruments across studies, allowing us to apply item response theory linking procedures. We fit generalized structural equation models, and found unidimensional factor models fit well. We found the instruments were highly correlated (all r > 0.9) and kappa statistics for delirium case identification were high (range: 0.89 to 0.95). We generated crosswalks to map sum scores on one instrument to another. Our results suggest the same underlying construct, propensity to delirium, is measured across the four instruments. The crosswalks facilitate comparison and combination for immediate clinical use or for future meta-analyses. In future steps, we will use our results to find the optimal cut-points for use across all instruments to identify delirium.

Download Full-text

Measuring the Big Five in Self-Report and Other Ratings

European Journal of Psychological Assessment ◽

10.1027//1015-5759.16.1.31 ◽

2000 ◽

Vol 16 (1) ◽

pp. 31-43 ◽

Cited By ~ 24

Author(s):

Claudio Barbaranelli ◽

Gian Vittorio Caprara

Keyword(s):

Big Five ◽

Structural Equation ◽

Structural Equation Models ◽

Error Variance ◽

The Self ◽

Self Report ◽

Sources Of Information ◽

Third Person ◽

Fit Indexes ◽

Response Modes

Summary: The aim of the study is to assess the construct validity of two different measures of the Big Five, matching two “response modes” (phrase-questionnaire and list of adjectives) and two sources of information or raters (self-report and other ratings). Two-hundred subjects, equally divided in males and females, were administered the self-report versions of the Big Five Questionnaire (BFQ) and the Big Five Observer (BFO), a list of bipolar pairs of adjectives ( Caprara, Barbaranelli, & Borgogni, 1993 , 1994 ). Every subject was rated by six acquaintances, then aggregated by means of the same instruments used for the self-report, but worded in a third-person format. The multitrait-multimethod matrix derived from these measures was then analyzed via Structural Equation Models according to the criteria proposed by Widaman (1985) , Marsh (1989) , and Bagozzi (1994) . In particular, four different models were compared. While the global fit indexes of the models were only moderate, convergent and discriminant validities were clearly supported, and method and error variance were moderate or low.

Download Full-text

Collective Efficacy and Organizational Commitment in an Italian City Hall

European Psychologist ◽

10.1027/1016-9040.14.4.363 ◽

2009 ◽

Vol 14 (4) ◽

pp. 363-371 ◽

Cited By ~ 14

Author(s):

Laura Borgogni ◽

Silvia Dello Russo ◽

Laura Petitta ◽

Gary P. Latham

Keyword(s):

Organizational Commitment ◽

Collective Efficacy ◽

Structural Equation ◽

Structural Equation Models ◽

Practical Significance ◽

City Hall ◽

Italian City ◽

Public Sector Managers ◽

The Individual ◽

Level Group

Employees (N = 170) of a City Hall in Italy were administered a questionnaire measuring collective efficacy (CE), perceptions of context (PoC), and organizational commitment (OC). Two facets of collective efficacy were identified, namely group and organizational. Structural equation models revealed that perceptions of top management display a stronger relationship with organizational collective efficacy, whereas employees’ perceptions of their colleagues and their direct superior are related to collective efficacy at the group level. Group collective efficacy had a stronger relationship with affective organizational commitment than did organizational collective efficacy. The theoretical significance of this study is in showing that CE is two-dimensional rather than unidimensional. The practical significance of this finding is that the PoC model provides a framework that public sector managers can use to increase the efficacy of the organization as a whole as well as the individual groups that compose it.

Download Full-text

A Cautionary Note on Incremental Fit Indices Reported by LISREL

Methodology ◽

10.1027/1614-1881.1.2.81 ◽

2005 ◽

Vol 1 (2) ◽

pp. 81-85 ◽

Cited By ~ 5

Author(s):

Stefan C. Schmukle ◽

Jochen Hardt

Keyword(s):

Factor Analysis ◽

Maximum Likelihood ◽

Structural Equation ◽

Structural Equation Models ◽

Null Model ◽

Likelihood Estimation ◽

Parameter Estimates ◽

Fit Indices ◽

Cautionary Note ◽

Confirmatory Factor

Abstract. Incremental fit indices (IFIs) are regularly used when assessing the fit of structural equation models. IFIs are based on the comparison of the fit of a target model with that of a null model. For maximum-likelihood estimation, IFIs are usually computed by using the χ2 statistics of the maximum-likelihood fitting function (ML-χ2). However, LISREL recently changed the computation of IFIs. Since version 8.52, IFIs reported by LISREL are based on the χ2 statistics of the reweighted least squares fitting function (RLS-χ2). Although both functions lead to the same maximum-likelihood parameter estimates, the two χ2 statistics reach different values. Because these differences are especially large for null models, IFIs are affected in particular. Consequently, RLS-χ2 based IFIs in combination with conventional cut-off values explored for ML-χ2 based IFIs may lead to a wrong acceptance of models. We demonstrate this point by a confirmatory factor analysis in a sample of 2449 subjects.

Download Full-text

Forced Zero Cross-Loading Misspecifications in Measurement Component of Structural Equation Models

Methodology ◽

10.1027/1614-2241/a000084 ◽

2014 ◽

Vol 10 (4) ◽

pp. 138-152 ◽

Cited By ~ 5

Author(s):

Hsien-Yuan Hsu ◽

Susan Troncoso Skidmore ◽

Yan Li ◽

Bruce Thompson

Keyword(s):

Structural Equation ◽

Structural Model ◽

Structural Equation Models ◽

Equation Model ◽

Model Fit ◽

Model Specification ◽

Simulation Design ◽

Measurement Models ◽

Fit Statistics ◽

Path Coefficients

The purpose of the present paper was to evaluate the effect of constraining near-zero parameter cross-loadings to zero in the measurement component of a structural equation model. A Monte Carlo 3 × 5 × 2 simulation design was conducted (i.e., sample sizes of 200, 600, and 1,000; parameter cross-loadings of 0.07, 0.10, 0.13, 0.16, and 0.19 misspecified to be zero; and parameter path coefficients in the structural model of either 0.50 or 0.70). Results indicated that factor pattern coefficients and factor covariances were overestimated in measurement models when near-zero parameter cross-loadings constrained to zero were higher than 0.13 in the population. Moreover, the path coefficients between factors were misestimated when the near-zero parameter cross-loadings constrained to zero were noteworthy. Our results add to the literature detailing the importance of testing individual model specification decisions, and not simply evaluating omnibus model fit statistics.

Download Full-text