Schools as Refractors: Comparing Summertime and School-Year Skill Inequality Trajectories

How does schooling affect inequality in students’ academic skills? Studies comparing children’s trajectories during summers and school years provide a provocative way of addressing this question, but the most persuasive seasonal studies (1) focus primarily on skill gaps between social categories (e.g., social class, race/ethnicity), which constitute only a small fraction of overall skill inequality, and (2) are restricted to early grades, making it difficult to know whether the patterns extend into later grades. In this study, we use seasonal comparisons to examine the possibilities that schooling exacerbates, reduces, or reproduces overall skill inequality in math, reading, language use, and science with recent national data on U.S. public school students spanning numerous grade levels from the Northwest Evaluation Association. Our results suggest that schooling has a compensatory effect on inequality in reading, language, and science skills but not math skills. We conclude by discussing the theoretical implications of our findings, possible reasons why the math findings differ from those of other subjects, and discrepant seasonal patterns across national data sets.

The Relationship Between Test Item Format and Gender Achievement Gaps on Math and ELA Tests in Fourth and Eighth Grades

Prior research suggests that males outperform females, on average, on multiple-choice items compared to their relative performance on constructed-response items. This paper characterizes the extent to which gender achievement gaps on state accountability tests across the United States are associated with those tests’ item formats. Using roughly 8 million fourth- and eighth-grade students’ scores on state assessments, we estimate state- and district-level math and reading male-female achievement gaps. We find that the estimated gaps are strongly associated with the proportions of the test scores based on multiple-choice and constructed-response questions on state accountability tests, even when controlling for gender achievement gaps as measured by the National Assessment of Educational Progress (NAEP) or Northwest Evaluation Association (NWEA) Measures of Academic Progress (MAP) assessments, which have the same item format across states. We find that test item format explains approximately 25% of the variation in gender achievement gaps among states.

Introducing a Computer-Adaptive Testing System to a Small School District

A computer-adaptive test (CAT) is a relatively new type of technology in which a computer program “intelligently” selects and presents questions to examinees according to an evolving estimate of achievement and a prescribed test plan. A well written CAT can be expected to efficiently produce student achievement estimates that are more accurate and more meaningful than a typical teacher-generated paper and pencil (P&P) test with a similar number of questions. Although this method of testing sounds good in theory, many schools and districts are waiting for positive examples of practical applications and observable benefits before adopting a CAT. This chapter begins by describing the essential elements of meaningful measurement in education and the features of a typical CAT. Next, we describe the Measures of Academic Progress (MAP) system of the Northwest Evaluation Association (NWEA; 2004) and observations made during the introduction of this system into a small semirural school district. Finally, as independent observers, we provide a set of recommendations to help guide other districts as they consider the potentials of implementing a CAT system to guide instruction within their schools.

Introducing a Computer-Adaptive Testing System to a Small School District

A computer-adaptive test (CAT) is a relatively new type of technology in which a computer program “intelligently” selects and presents questions to examinees according to an evolving estimate of achievement and a prescribed test plan. A well written CAT can be expected to efficiently produce student achievement estimates that are more accurate and more meaningful than a typical teacher-generated paper and pencil (P&P) test with a similar number of questions. Although this method of testing sounds good in theory, many schools and districts are waiting for positive examples of practical applications and observable benefits before adopting a CAT. This chapter begins by describing the essential elements of meaningful measurement in education and the features of a typical CAT. Next, we describe the Measures of Academic Progress (MAP) system of the Northwest Evaluation Association (NWEA; 2004) and observations made during the introduction of this system into a small semirural school district. Finally, as independent observers, we provide a set of recommendations to help guide other districts as they consider the potentials of implementing a CAT system to guide instruction within their schools.