In a recent study, we empirically demonstrated limitations in traditional ways that psychologists have used factor analysis to define cognitive constructs (Delis et al., 2003). Our criticism of factor analysis was not directed at this statistical method per se, but rather at how it has often been employed by psychologists to test cognitive constructs. Specifically, we pointed out shortcomings in using this techniquewith normal or mixed clinical populations. We argued that the factor-analytic studies of memory tests with normal or mixed clinical populations often yielded solutions in which measures of immediate and delayed memory loaded on the same factor. This particular use of factor analysis can mask important distinctions between critical cognitive functions that have been demonstrated using other research methods, such as experimental manipulations or case studies. We then conducted a factor-analytic study that empirically demonstrated that, whereas immediate and delayed memory measures load on the same factor when using normal or mixed clinical samples, these measures load onseparatefactors when using a homogenous population of patients with Alzheimer's disease (AD). We drew the conclusion that factor analytic techniques can still be used asonemethod for exploring conceptual relationships, but only if these methods are used as part of a systematic, programmatic exploration involving separate confirmatory factor analyses using multiple homogenous patient populations. In the first published reply to our study, Larrabee (2003) pointed out other limitations, stating that application of factor-analytic techniques to a single test that yields multiple measures may result in global, simplistic solutions due to method variance, which is the tendency of different variables from the same test to correlate significantly.
How do neuropsychologists define cognitive constructs? Further thoughts on limitations of factor analysis used with normal or mixed clinical populations
