This article analyzes the effects of federal compensatory education policy on the capacity of schools to deliver high-quality services to poor children who have benefited only marginally from schooling. A major focus is federal assessment of Chapter 1 programs and how such assessment practices shape the organizational capacity of schools. Current assessment practices focus on student outcomes, thereby ignoring the inchoate mass of organizational variables that shapes the quality of instructional practices and student achievement. This article examines the policy history of Chapter 1 evaluation, particularly as evaluation relates to school improvement. It then proposes alternatives to current practice that focus on efforts to move evaluation from instantaneous and often misleading snapshots of performance to evaluation that informs federal policymakers of their ability to shape institutional competence and organizational capacity. While, as a practical and political matter, Chapter 1 assessment will continue to focus on individual student achievement, that assessment should be supplemented by evaluation that focuses on the capacity of Chapter 1 to shape the quality of educational practice in schools. Finally, the article proposes several conceptual models of organizational assessment that foster practices aligned with institutional capacity building.