A Minority pulls the sample mean: on the individual prevalence of robust group-level cognitive phenomena – the instance of the SNARC effect
The aim of cognitive psychology is to obtain insights into human cognition in general. For this purpose, group-studies are typically conducted on representative samples so that the results can be generalized to the population. Using this approach, individual differences in such group-level cognitive phenomena are typically neglected and not much is known about their prevalence at the individual level. Such information is nevertheless important for claims about the universality of phenomena, as in theory, significant effects at the group-level can in principle be driven by a minority of participants. Here we used a uniform analysis of 18 existing data sets revealing a well-replicated phenomenon in numerical cognition: the SNARC (Spatial-Numerical Association of Response Codes) effect, in order to investigate the prevalence of the effect at the individual level. Three methods of analyzing the presence of the effect at the individual level were utilized: one psychometric and two bootstrapping methods. The results show that the group-level SNARC effect is driven by a minority of individuals (≤ 45%) who reveal the effect. This finding demonstrate an important theoretical issue: whether group-level effects really reflect general principles of cognition. We discuss advantages and drawbacks of the present methods and their usefulness for investigating the prevalence of other cognitive phenomena. We posit that testing the presence of robust group-level cognitive effects at the individual level as well as ensuring their reliable measurement is an important step towards integrating two traditionally separate approaches of scientific psychology proposed back in 50’ by Cronbach: experimental and correlational.