On the Reliability of Implicit Measures: Current Practices and Novel Perspectives
During the last two decades a new class of indirect measurement procedures has emerged that has been used widely in psychological science. These procedures were developed in order to circumvent the limitations of self-reports and crack open the hidden world of ‘implicit’ cognition. Yet despite their popularity there seems to be no general framework that constrains (or guides) the way in which we think about the reliability of implicit measures. Instead there are far too many (subjective) researcher degrees of freedom when it comes to deciding how to assess, interpret, and even report reliability. In this paper we introduce such a framework and argue that it can be used by novel and seasoned researchers alike to estimate and interpret the reliability of their implicit measures. Our approach draws on Latent Variable Modeling and the idea of parceling in order to approximate reliability (in the sense of consistency, equivalence, stability) and test the assumptions that underlie those approximations. We close by discussing the implications of our framework for the conceptualization of reliability in particular and for implicit cognition research more generally.