Norming

Chapter 30 focuses on issues relating to norm-referenced measures, in particular the use of norms in international assessments. This chapter highlights some of the complex issues involved in norming scores. While the initial sections of the chapter review some general issues of norm construction and use, this is not a chapter on the mechanics of how to produce norms. Rather, it focuses on issues of when and how to use norms, what aggregations of samples to base them on, and how norm-referenced scores should be interpreted. In particular, it considers issues relating to the development and use of international norms. Test norms are often essential for stakeholders to understand the meaning of test scores by providing information about the standing of the test taker relative to other members of the population. Finally, the chapter notes that culturally related variance may reflect either measurement bias or effects of cultural style.

Reliability and Validity

Chapter 28 describes the evolution of the two most important concepts in psychometrics and for psychological and educational testing: reliability and validity. Between the publication of the first psychological tests and the most recent developments, the scientific, professional, and ethical requirements demanded by testing have largely evolved. Also the scientific disciplines of psychology and education and the practice based on these disciplines are no longer the same as early in their history. Psychometric models have changed, theories have changed, and the problems and requirements made by psychological and educational practice have changed. It does therefore not surprise that the notions of reliability and validity have also evolved. The aim of this chapter is to offer a historical and conceptual view of both these notions, to discuss some approaches in the investigation of reliability and validity, and to formulate some considerations on the way the two notions have evolved.

Multigroup Comparisons

One common weakness in multigroup comparison research is the pervading assumption that both the assessment scale and the construct(s) it is designed to measure are operating equivalently across the groups of interest. Structural equation modeling (SEM) has long been recognized as the premier methodological approach to test assumptions of equivalence. The primary focus of this chapter is directed toward the use of SEM in testing for the equivalence (also termed “invariance”) of assessment scales across independent groups. The purposes are threefold: (a) to explicate the basic notions underlying the concepts of measurement and structural equivalence; (b) to outline and describe the hierarchical set of steps involved in testing for measurement and structural equivalence; and (c) to provide an annotated, yet concise illustration of this equivalence-testing process as it relates to a commonly used assessment scale designed to measure teacher burnout across elementary and secondary female teachers.

Test Adaptations

Adaptations refer to the process of translating and adjusting an instrument to a new cultural context, which usually goes beyond close translations and involves various changes to the instrument in order to increase its cultural and linguistic adequacy. A brief historical overview of approaches is presented that have been used in the past 75 years, illustrating how the traditional emphasis on linguistic aspects has shifted to multidisciplinary efforts; modern test adaptations are often made by teams. A taxonomy of adaptations is then presented, distinguishing between conceptual, cultural, linguistic, and measurement reasons for adjusting an instrument. A good adaptation integrates these four types. Various guidelines for adapting instruments are described, such as the ITC Guidelines. The authors conclude that test adaptations have largely replaced the concept of test translations and that test adaptations can be used in large-scale surveys and can accommodate instruments that cover both general (universal) and local components.

Response Biases

Response biases comprise a variety of systematic tendencies of responding to questionnaire items. Response biases exert an influence on item responses in addition to any constructs that the questionnaire is designed to measure and can therefore potentially bias the corresponding trait level estimates. This chapter addresses general response biases that are independent of item content, including response styles (e.g., extreme response style, acquiescence) and rater biases (halo effect, leniency/severity bias), as well as response biases that are related to item content and depend strongly on the context (socially desirable responding). The chapter summarizes research on correlates of response biases and research on inter-individual and cross-cultural differences in engaging in response styles and rater biases. It describes different methods that can be applied at the test construction stage to prevent or minimize the occurrence of response biases. Finally, it depicts methods developed for correcting for the effects of response biases.

Advances in Rehabilitation and Health Assessments

This chapter considers the types of measures used in rehabilitation and health settings, including their alignment with the major health and disease diagnostic and classification systems. The chapter provides an overview of the needs that are served by rehabilitation and health assessments and highlights the increasing importance of measures that yield data to guide decisions to support functioning across a broad range of life domains. It makes the case for prospective rehabilitation and health assessments that predict more reliably future functioning in preferred activities taking into account personal and environment interactions. Environmental factors include the context for both achieved and aspired participation and for which rehabilitation and health supports are intended. The future of rehabilitation and health assessments is tied to their ability to provide data that translate directly into needed support services for maximal participation by people living with debilitating health conditions.

Neuropsychological Assessment

Neuropsychological assessment involves compiling information about a person’s premorbid or current behaviour and using it to make inferences about his or her brain functioning or to predict behavior. Today, neuropsychological assessment is focused on identifying cognitive deficits associated with different disorders; changes over time; efficacy of treatment; and patient strengths and weaknesses. It is also used to predict functioning, guide rehabilitation planning, and supplement forensic/legal evaluations. Key elements of neuropsychological assessments described in the chapter include history taking; behavioral observations; the testing situation; selection, administration, and scoring of tests; and test standardization and norms. Descriptions of fixed, flexible, and “flexible battery” approaches to testing are provided, as are descriptions of quantitative (norm) and qualitative (process) approaches. Several norming approaches of particular interest to neuropsychological assessment (i.e., continuous norming, equivalent scores, overlapping cell tables) are noted. An overview is provided of a number of commonly assessed areas of neuropsychological functioning.

Psychopathology

This chapter reviews the assessment of psychopathology, with a focus on psychosis and clinically related phenomena and conditions, such as prodromal phases and at-risk mental states of psychosis. The psychosis syndrome, which is characterized by a disruption of higher cognitive functions, can be found when any basic psychological process (e.g., memory, attention, etc.) is altered. It is used here as an example of psychopathological disorder. The chapter begins with an overview of the psychosis syndrome as a model of psychopathological disorder, emphasizing its core domains (i.e., positive, negative, and cognitive symptoms). It discusses the main psychological tests and procedures for psychosis assessment and provides an overall review of measurement instruments for psychosis risk assessment from both clinical and psychometric high-risk paradigms, where psychological testing plays a crucial role in terms of detecting people at risk for psychosis prior to developing serious mental disorder and need for care.

Personality Assessment

The chapter focuses on the assessment of personality in an international context. Starting from the definition of personality, the chapter discusses the way culture and personality are mixed and sets then out to explain the emic (indigenous) versus etic (universal) debate in personality assessment. The combined emic-etic approach is outlined as an interesting evolution in cross-cultural personality assessment, and two measures based on this approach are discussed, the Cross-Cultural Personality Assessment Inventory (CPAI) and the South African Personality Inventory (SAPI). Finally, the chapter discusses the currently dominant model of personality used in assessment internationally, the five-factor model, outlining some of the dilemmas still being debated related to this model, such as the broad versus narrow debate, the cross-cultural replicability issue, and the bandwidth-fidelity dilemma.

The History and Current Status of Testing Across Cultures and Countries

Chapter 2 gives a narrative account of the use of tests across countries and cultures over the course of nearly a century. Initially, differences in test score distributions were usually interpreted at face value, independent of the cultural and educational background of test takers. Findings of group differences in the domain of intelligence were seen as a confirmation of widely held beliefs, also among psychologists, about racial differences. Soon implicit assumptions to the effect that tests can be culture-free or culture-fair were challenged. The cross-cultural equivalence of tests became an issue for conceptual and, especially, for psychometric analysis. This line of work has resulted in a large array of methods and statistical procedures for identifying cultural bias in items and tests. The concluding section of the chapter envisages further improvements in the quality of instruments for cross-cultural usage, notably through the use of international teams of authors in test development.