Phantom effects in school composition research: consequences of failure to control biases due to measurement error in traditional multilevel models

The paper explores some issues related to endogeneity in multilevel models, focusing on the case where the random effects are correlated with a level 1 covariate in a linear random intercept model. We consider two basic specifications, without and with the sample cluster mean. It is generally acknowledged that the omission of the cluster mean may cause omitted-variable bias. However, it is often neglected that the inclusion of the sample cluster mean in place of the population cluster mean entails a measurement error that yields biased estimators for both the slopes and the variance components. In particular, the contextual effect is attenuated, while the level 2 variance is inflated. We derive explicit formulae for measurement error biases that allow us to implement simple post-estimation corrections based on the reliability of the covariate. In the first part of the paper, the issue is tackled in a standard framework where the population cluster mean is treated as a latent variable. Later we consider a different framework arising when sampling from clusters of finite size, where the latent variable methods may have a poor performance, and we show how to effectively modify the measurement error correction. The theoretical analysis is supplemented with a simulation study and a discussion of the implications for effectiveness evaluation.

Download Full-text

Measurement Error in Longitudinal Data

10.1093/oso/9780198859987.001.0001 ◽

2021 ◽

Keyword(s):

Measurement Error ◽

Longitudinal Data ◽

Survey Methodology ◽

Multilevel Models ◽

Growth Models ◽

Estimation Methods ◽

Longitudinal Data Collection ◽

Panel Conditioning ◽

Low Sensitivity ◽

Mere Measurement

Understanding change is essential in most scientific fields. This is highlighted by the importance of issues such as shifts in public health and changes in public opinion regarding politicians and policies. Nevertheless, our measurements of the world around us are often imperfect. For example, measurements of attitudes might be biased by social desirability, while estimates of health may be marred by low sensitivity and specificity. In this book we tackle the important issue of how to understand and estimate change in the context of data that are imperfect and exhibit measurement error. The book brings together the latest advances in the area of estimating change in the presence of measurement error from a number of different fields, such as survey methodology, sociology, psychology, statistics, and health. Furthermore, it covers the entire process, from the best ways of collecting longitudinal data, to statistical models to estimate change under uncertainty, to examples of researchers applying these methods in the real world. The book introduces the reader to essential issues of longitudinal data collection such as memory effects, panel conditioning (or mere measurement effects), the use of administrative data, and the collection of multi-mode longitudinal data. It also introduces the reader to some of the most important models used in this area, including quasi-simplex models, latent growth models, latent Markov chains, and equivalence/DIF testing. Further, it discusses the use of vignettes in the context of longitudinal data and estimation methods for multilevel models of change in the presence of measurement error.

Download Full-text

Benefits from Computerized Adaptive Testing as Seen in Simulation Studies

European Journal of Psychological Assessment ◽

10.1027//1015-5759.15.2.91 ◽

1999 ◽

Vol 15 (2) ◽

pp. 91-98 ◽

Cited By ~ 10

Author(s):

Lutz F. Hornke

Keyword(s):

Measurement Error ◽

Computerized Adaptive Testing ◽

Test Procedure ◽

Adaptive Testing ◽

Parameter Estimates ◽

Simulation Studies ◽

Computerized Adaptive Test ◽

Item Banks ◽

Item Parameters ◽

General Reliability

Summary: Item parameters for several hundreds of items were estimated based on empirical data from several thousands of subjects. The logistic one-parameter (1PL) and two-parameter (2PL) model estimates were evaluated. However, model fit showed that only a subset of items complied sufficiently, so that the remaining ones were assembled in well-fitting item banks. In several simulation studies 5000 simulated responses were generated in accordance with a computerized adaptive test procedure along with person parameters. A general reliability of .80 or a standard error of measurement of .44 was used as a stopping rule to end CAT testing. We also recorded how often each item was used by all simulees. Person-parameter estimates based on CAT correlated higher than .90 with true values simulated. For all 1PL fitting item banks most simulees used more than 20 items but less than 30 items to reach the pre-set level of measurement error. However, testing based on item banks that complied to the 2PL revealed that, on average, only 10 items were sufficient to end testing at the same measurement error level. Both clearly demonstrate the precision and economy of computerized adaptive testing. Empirical evaluations from everyday uses will show whether these trends will hold up in practice. If so, CAT will become possible and reasonable with some 150 well-calibrated 2PL items.

Download Full-text

Modeling Intraindividual Variability in Three-Level Multilevel Models

Methodology ◽

10.1027/1614-2241/a000150 ◽

2018 ◽

Vol 14 (3) ◽

pp. 95-108 ◽

Cited By ~ 4

Author(s):

Steffen Nestler ◽

Katharina Geukes ◽

Mitja D. Back

Keyword(s):

Multilevel Model ◽

Multilevel Models ◽

Mixed Effects ◽

Repeated Measurements ◽

Scale Model ◽

Suggested Approach ◽

Psychological Study ◽

Level Data ◽

Ecological Momentary ◽

Momentary Assessment

Abstract. The mixed-effects location scale model is an extension of a multilevel model for longitudinal data. It allows covariates to affect both the within-subject variance and the between-subject variance (i.e., the intercept variance) beyond their influence on the means. Typically, the model is applied to two-level data (e.g., the repeated measurements of persons), although researchers are often faced with three-level data (e.g., the repeated measurements of persons within specific situations). Here, we describe an extension of the two-level mixed-effects location scale model to such three-level data. Furthermore, we show how the suggested model can be estimated with Bayesian software, and we present the results of a small simulation study that was conducted to investigate the statistical properties of the suggested approach. Finally, we illustrate the approach by presenting an example from a psychological study that employed ecological momentary assessment.

Download Full-text