One of the potential strengths of the No Child Left Behind (NCLB) Act enacted in 2002 is that the law requires the production of an enormous amount of data, particularly from tests, which, if used properly, might help us improve education. As an economist and as someone who served 13 years on the School Committee1 in Brookline Massachusetts, until May 2009, I have been appalled by the limited ability of districts to analyze these data; I have been equally appalled by the cavalier manner in which economists use test scores and related measures in their analyses. The summary data currently provided are very hard to interpret, and policymakers, who typically lack statistical sophistication, cannot easily use them to assess progress. In some domains, most notably the use of average test scores to evaluate teachers or schools, the education community is aware of the biases and has sought better measures. The economics and statistics communities have both responded to and created this demand by developing value-added measures that carry a scientific aura. However, economists have largely failed to recognize many of the problems with such measures. These problems are sufficiently important that they should preclude any automatic link between these measures and rewards or sanctions. They do, however, contain information and can be used as a catalyst for more careful evaluation of teachers and schools, and as a lever to induce principals and other administrators to act on their knowledge.
Testing the validity of value-added measures of educational progress with genetic data