Value Added and Its Uses: Where You Stand Depends on Where You Sit

2013 ◽  
Vol 8 (3) ◽  
pp. 418-434 ◽  
Author(s):  
Sean Corcoran ◽  
Dan Goldhaber
Keyword(s):  
Performance Evaluation ◽  
Teacher Evaluation ◽  
Teacher Performance ◽  
Value Added ◽  
Evidence Base ◽  
High Stakes ◽  
Practical Utility ◽  
Evaluation Systems ◽  
Value Added Model ◽  
High Level

In this policy brief we argue that there is little debate about the statistical properties of value-added model (VAM) estimates of teacher performance, yet, despite this, there is little consensus about what the evidence about VAMs implies for their practical utility as part of high-stakes performance evaluation systems. A review of the evidence base that underlies the debate over VAM measures, followed by our subjective opinions about the value of using VAMs, illustrates how different policy conclusions can easily arise even given a high-level general agreement about an existing body of evidence. We conclude the brief by offering a few thoughts about the limits of our knowledge and what that means for those who do wish to integrate VAMs into their own teacher-evaluation strategy.

Value-Added Model (VAM) Scholars on Using VAMs for Teacher Evaluation After the Passage of the Every Student Succeeds Act

2020 ◽  
Vol 122 (7) ◽  
pp. 1-34
Author(s):  
Matthew Ryan Lavery ◽  
Audrey Amrein-Beardsley ◽  
Tray Geiger ◽  
Margarita Pivovarova
Keyword(s):  
No Child Left Behind ◽  
Teacher Evaluation ◽  
Quantitative Methods ◽  
Common Ground ◽  
Value Added ◽  
Multiple Perspectives ◽  
High Stakes ◽  
Evaluation Systems ◽  
Federal Initiatives ◽  
Every Student Succeeds Act

Background/Context The Race to the Top federal initiatives and requirements surrounding waivers of No Child Left Behind promoted expanded use of value-added models (VAMs) to evaluate teachers. Even after passage of the Every Student Succeeds Act (ESSA) relaxed these requirements, allowing more flexibility and local control, many states and districts continue to use VAMs in teacher evaluation systems, suggesting that they consider VAMs a valid measure of teacher effectiveness. Scholars in the fields of economics, education, and quantitative methods continue to debate several aspects of VAMs’ validity for this purpose, however. Purpose The purpose of this study was to directly ask the most experienced VAM scholars about validity of VAM use in teacher evaluation based on the aspects of validity described in the Standards for Educational and Psychological Testing and found in a review of high-quality peer-reviewed literature on VAMs. Participants We invited the 115 scholars listed as an author or coauthor of one or more of the 145 articles published on evaluating teachers with VAMs that have been published in prominent peer-reviewed journals between 2002 and implementation of ESSA in 2016. In this article, we analyze data from 36 respondents (12 economists, 13 educators, and 11 methodologists) who rated themselves as “experienced scholars,” “experts,” or “leading experts” on VAMs. Research Design This article reports both quantitative and qualitative analyses of a survey questionnaire completed by experienced VAM scholars. Findings Analyses of 44 Likert-scale items indicate that respondents were generally neutral or mixed toward the use of VAMs in teacher evaluation, though responses from educational researchers were more critical of VAM use than were responses from economists and quantitative methodologists. Qualitative analysis of free response comments suggests that participants oppose exclusive or high-stakes use of VAMs but are more supportive of their use as a component of evaluation systems that use multiple measures. Conclusions These findings suggest that scholars and stakeholders from different disciplines and backgrounds think about VAMs and VAM use differently. We argue that it is important to understand and address stakeholders’ multiple perspectives to find the common ground on which to build consensus.

What Explains the Race Gap in Teacher Performance Ratings? Evidence From Chicago Public Schools

2020 ◽  
pp. 016237372097020
Author(s):  
Matthew P. Steinberg ◽  
Lauren Sartain
Keyword(s):  
Public Schools ◽  
Teacher Performance ◽  
School Context ◽  
Value Added ◽  
Chicago Public Schools ◽  
Performance Ratings ◽  
Evaluation Systems ◽  
Teacher Diversity ◽  
Educator Evaluation ◽  
School Differences

Racial gaps in teacher performance ratings have emerged nationwide across newly implemented educator evaluation systems. Using Chicago Public Schools data, we quantify the magnitude of the race gap in teachers’ classroom observation scores, examine its determinants, and describe the potential implications for teacher diversity. Between-school differences explain most of the race gap and within-school classroom-level differences—poverty, incoming achievement, and prior-year misconduct of a teacher’s students—explain the remainder of the race gap. Teachers’ value-added scores explain none of the race gap. Leveraging within-teacher variation in the teacher–evaluator race match, we find that racial mismatch does not influence observation scores. Adjusting observation scores for classroom and school context will generate more equitable ratings of teacher performance and mitigate potential adverse consequences for teacher diversity.

The Sensitivity of Teacher Performance Ratings to the Design of Teacher Evaluation Systems

2017 ◽  
Vol 46 (7) ◽  
pp. 378-396 ◽  
Author(s):  
Matthew P. Steinberg ◽  
Matthew A. Kraft
Keyword(s):  
Teacher Evaluation ◽  
Teacher Performance ◽  
Critical Role ◽  
Performance Measure ◽  
Performance Ratings ◽  
Evaluation Systems ◽  
Teacher Proficiency ◽  
And Performance ◽  
Teacher Evaluation Systems ◽  
Using Data

In recent years, states and districts have responded to federal incentives and pressure to institute major reforms to their teacher evaluation systems. The passage of the Every Student Succeeds Act in 2015 now provides state policymakers with even greater autonomy to redesign existing evaluation systems. Yet, little evidence exists to inform decisions about two key system design features: teacher performance measure weights and performance ratings thresholds. Using data from the Measures of Effective Teaching study, we conduct simulation-based analyses that illustrate the critical role that performance measure weights and ratings thresholds play in determining teachers’ summative evaluation ratings and the distribution of teacher proficiency rates. These findings offer insights to policymakers and administrators as they refine and possibly remake teacher evaluation systems.

scholarly journals Declining Morale, Diminishing Autonomy, and Decreasing Value: Principal Reflections on a High-Stakes Teacher Evaluation System

2018 ◽  
Vol 13 (8) ◽  
Author(s):  
Noelle A Paufler
Keyword(s):  
Student Achievement ◽  
Teacher Evaluation ◽  
Evaluation System ◽  
Negative Impact ◽  
Value Added ◽  
Unintended Consequences ◽  
Evaluation Systems ◽  
State And Local ◽  
Teacher Evaluation Systems ◽  
District Culture

Since the adoption of teacher evaluation systems that rely, at least in part, on controversial student achievement measures, little research has been conducted that focuses on stakeholders’ perceptions of systems in practice, specifically the perceptions of school principals. This study was conducted in a large urban school district to better understand principals’ perceptions of evaluating teachers based on professional and instructional practices as well as student achievement (i.e., value-added scores). Principals in this study strongly expressed concerns regarding: (a) the negative impact of the teacher evaluation system on district culture and morale; (b) their lack of autonomy in evaluating teachers and making staffing decisions; and (c) their perceived lack of value as professionals in the district. Examining the implications of teacher evaluation systems, per the experiences of principals as practitioners, is increasingly important if state and local policymakers as well as the general public are to better understand the intended and unintended consequences of these systems in practice.

Refocusing the Debate: Assessing the Purposes and Tools of Teacher Evaluation

2012 ◽  
Vol 82 (1) ◽  
pp. 123-141 ◽  
Author(s):  
John Papay
Keyword(s):  
Professional Development ◽  
Teacher Evaluation ◽  
Teacher Development ◽  
Teacher Performance ◽  
Value Added ◽  
Evaluation Tools ◽  
Development Tools ◽  
Measurement Tools ◽  
Value Added Measures

In this article, John Papay argues that teacher evaluation tools should be assessed not only on their ability to measure teacher performance accurately, but also on how well they inform and support ongoing teacher development. He looks at two major approaches to teacher evaluation reform: value-added measures and standards-based evaluations. Papay analyzes these two approaches both as measurement tools and as professional development tools, illuminating the advantages, drawbacks, and untapped potential of each. In the process, attention is refocused towards a broader conception of the purpose of teacher evaluation.

scholarly journals Enacting the Rubric: Teacher Improvements in Windows of High-Stakes Observation

2019 ◽  
pp. 1-51
Author(s):  
Aaron R. Phipps ◽  
Emily A. Wiseman
Keyword(s):  
Teacher Evaluation ◽  
Evaluation System ◽  
Teacher Practices ◽  
Causal Link ◽  
High Stakes ◽  
Evaluation Program ◽  
Evaluation Systems ◽  
Post Evaluation ◽  
Teacher Evaluation Systems ◽  
Teacher Evaluation Program

Teacher evaluation systems that use in-class observations, particularly in high-stakes settings, are frequently understood as accountability systems intended as non-intrusive measures of teacher quality. Presumably, the evaluation system motivates teachers to improve their practice – an accountability mechanism – and provides actionable feedback for improvement – an information mechanism. No evidence exists, however, establishing the causal link between an evaluation program and daily teacher practices. Importantly, it is unknown how teachers may modify their practice in the time leading up to an unannounced in-class observation, or how they integrate feedback into their practice post-evaluation, a question that fundamentally changes the design and philosophy of teacher evaluation programs. We disentangle these two effects with a unique empirical strategy that exploits random variation in the timing of in-class observations in the Washington, D.C. teacher evaluation program IMPACT. Our key finding is that teachers work to improve during periods in which they are more likely to be observed, and they improve with subsequent evaluations. We interpret this as evidence that both mechanisms are at work, and as a result, policymakers should seriously consider both when designing teacher evaluation systems.

scholarly journals Effect of Teacher Performance Evaluation on School Effectiveness

sjesr ◽  
2021 ◽  
Vol 4 (1) ◽  
pp. 431-439
Author(s):  
Muhammad Akram ◽  
Farrukh Munir ◽  
Ahmad Bilal
Keyword(s):  
Performance Evaluation ◽  
Teacher Evaluation ◽  
School Effectiveness ◽  
Teacher Performance ◽  
Linear Regression Analysis ◽  
Assessment Instrument ◽  
School Teachers ◽  
Evaluation Score ◽  
Evaluation Scores ◽  
Teacher Performance Evaluation

This study was conducted to measure the effect of teacher performance evaluation on school effectiveness in public high schools in Pakistan. Teacher evaluation is a formal and systematic process of evaluating teacher performance that plays an important role in enhancing school effectiveness. School effectiveness is a process that ensures that a particular school has effectively maintained a safe and orderly environment, implemented an instructional framework and curriculum that focuses on enhancing student learning, where the school monitoring system is highly responding, and where a competency-based system is in practice that ensures increased student achievement. A correlational research design was used to conduct this study. Using multistage sampling techniques, data were collected from 580 secondary school teachers in district Okara. Self-Assessment Instrument for Teacher Evaluation (α=.88) and School Effectiveness Questionnaire ((α=.86) were used for data collection. The Pearson correlation coefficient showed that teacher evaluation scores and school effectiveness were significantly correlated with each other (r=.69). As teacher performance evaluation scores increased, the score on school effectiveness also increased. Multiple linear regression analysis revealed that teacher performance evaluation score significantly predicted 46% of variance in school effectiveness. Further, female teachers were better on teacher performance evaluation score and school effectiveness. Teachers in urban schools showed higher scores on teacher performance evaluation scores and school effectiveness as compared to rural school teachers.

Exploring the Intended and Unintended Consequences of High-Stakes Teacher Evaluation on Schools, Teachers, and Students

2014 ◽  
Vol 116 (1) ◽  
pp. 1-29
Author(s):  
Alyson Leah Lavigne
Keyword(s):  
Student Achievement ◽  
Teacher Evaluation ◽  
Value Added ◽  
Opportunity To Learn ◽  
Unintended Consequences ◽  
Observation Data ◽  
Effective Teacher ◽  
High Stakes ◽  
Improve Student Achievement ◽  
Teacher Workforce

Background/Context The stakes are getting higher for teachers daily as more and more states adopt hiring, firing, and tenure-granting policies based on teacher evaluations. Even more concerning is the limited discussion about whether or not high-stakes teacher evaluation can meet the intended outcome of improved student achievement, and at what cost. These high-stakes decisions are based on the rationale that firing ineffective teachers (as primarily measured by observation data and value-added scores) will improve student achievement. This premise is challenged by various variables and assumptions (e.g., reliability, validity, percentage fired, and turnover) that, if not met, could result in a number of possible unintended consequences. Focus of Study This paper examines the history of high-stakes teacher evaluation and the ways in which teacher evaluation data are being used in today's schools to make human capital decisions. The intended consequences and unintended consequences are explored in great detail. Research Design This paper is an analytic essay. Conclusions/Recommendations There is no evidence that high-stakes teacher evaluation can produce a more effective teacher workforce and improve student achievement. Even if basic requirements and assumptions are met (e.g., highly reliable and valid measures, retention of effective teachers, and highly effective hires), gains in student achievement may be short lived, insignificant, or practically meaningless. The possible unintended consequences could result in worse, rather than better, student achievement outcomes and increase the gap in opportunity to learn for students attending the most and least affluent schools.

scholarly journals The Effect of Evaluation on Teacher Performance

2012 ◽  
Vol 102 (7) ◽  
pp. 3628-3651 ◽  
Author(s):  
Eric S Taylor ◽  
John H Tyler
Keyword(s):  
Student Achievement ◽  
Performance Evaluation ◽  
School Reform ◽  
Teacher Performance ◽  
Objective Measure ◽  
Value Added ◽  
Post Evaluation ◽  
Long Run ◽  
Ex Ante ◽  
Teacher Performance Evaluation

Teacher performance evaluation has become a dominant theme in school reform efforts. Yet, whether evaluation changes the performance of teachers, the focus of this paper, is unknown. Instead, evaluation has largely been studied as an input to selective dismissal decisions. We study mid-career teachers for whom we observe an objective measure of productivity—value-added to student achievement—before, during, and after evaluation. We find teachers are more productive in post-evaluation years, with the largest improvements among teachers performing relatively poorly ex-ante. The results suggest teachers can gain information from evaluation and subsequently develop new skills, increase long-run effort, or both.

Sign in / Sign up

Export Citation Format

Share Document