More Error than Attitude in Implicit Association Tests (IATs), a CFA-MTMM analysis of measurement error.
Many design characteristics of the popular Implicit Association Test (IAT) appear to make the task highly susceptible to measurement error. This study examined potential sources of measurement error for two types of IAT, the classic verbal IAT (VIAT) and a fully pictorial IAT (PIAT). A CFA-MTMM analytical approach was used to estimate the influence of both random error and method variance on the IAT scores. Four empirical IATs were employed to assess implicit bias towards Middle Eastern and European people (‘Racial’ VIAT and PIAT) and countries (‘Country’ VIAT and PIAT). They were completed by 198 student participants from an Australian University. The CFA-MTMM analysis provided clear evidence of measurement error confounding IAT scores. Specifically, IAT data was shown to be, on average, comprised of just over 50% random error variance, nearly 30% method variance and under 20% trait variance. These results demonstrate unequivocally that IAT scores are predominantly composed of measurement error not implicit attitudes. These findings have significant implications for the use of IATs in applied research. Options for minimising the impact of high error variance in future implicit attitudinal research are considered.