Test Theory and Measurement in Assessment

This chapter discusses the two major theories underlying scale development: classical test theory, which has dominated the field for the past century, and item response theory, which is more recent. It begins by summarizing the history of measurement, first of physical and physiological parameters and later of intelligence. This is followed by the steps involved in developing a scale: creating the items, determining if they fully span the construct of interest while at the same time not including irrelevant content, and assessing the usability of the items (whether they are understood correctly, whether they are free of jargon, if they avoid negatively worded phrases, etc.). The chapter then describes how to establish the reliability and validity of the scale—what are called the psychometric properties of the scale. It concludes by discussing some of the shortcomings with classical test theory, how item response theory attempts to address them, and the degree to which it has been successful in this regard. This chapter should be useful for those who need to evaluate existing scales as well as for those wanting to develop new scales.

Decision-Making in Psychological Assessment

This chapter describes a number of factors that may influence a clinician’s judgment and conclusions while conducting an assessment, and it discusses others that make interpretation of the results less than straightforward. It begins by discussing the effects of the prevalence, or base rate, of the disorder on the diagnostic accuracy of the findings. Even in the presence of seemingly unequivocal results pointing to a given diagnosis, the findings may lead to a false-positive conclusion if the prevalence is low and to a false-negative one if the prevalence is high. The chapter shows how using Bayes’ theorem can tell us the likelihood of a wrong diagnosis. It next discusses incremental validity—whether adding another test to the battery increases diagnostic accuracy. If the new test is correlated with ones already administered, then the amount of new information it provides is limited and may increase unwarranted confidence in the final diagnosis. Third, the chapter discusses various biases and heuristics that may affect diagnostic decision-making, such as anchoring, diagnostic momentum, premature closure, and the influence of patient and assessor characteristics. It concludes by presenting a number of steps that should be taken to minimize the effects of these biases.

Multimethod Assessment of Personality and Psychopathology

Multimethod assessment of personality and psychopathology plays a central role in diagnosis, treatment planning, and risk management. This chapter presents an evidence-based framework for multimethod assessment of personality and psychopathology in clinical and research settings, with an emphasis on test score integration. To implement this framework the clinician must: (a) understand the intra- and interpersonal processes that underlie different assessment methods; (b) interpret meaningful test score divergences as well as convergences; and (c) contextualize work in this area using ideas and findings from beyond clinical psychology. Following a discussion of major personality and psychopathology assessment tools and the intra- and interpersonal dynamics that influence responses to interviews, self-attribution tests (which have traditionally been identified as objective or self-report tests), and stimulus-attribution measures (traditionally labeled projective tests), conceptual, clinical, and empirical implications of current findings in this area are discussed. A template for multimethod assessment in clinical settings is presented, using personality pathology as an exemplar. A multimethod, process-focused approach to test score validation is described, which yields separate indices of outcome-focused and process-focused validity. Finally, the principles of multimethod assessment are applied to emerging models of personality and psychopathology.