Does Student Sorting Invalidate Value-Added Models of Teacher Effectiveness? An Extended Analysis of the Rothstein Critique

2011 ◽  
Vol 6 (1) ◽  
pp. 18-42 ◽  
Author(s):  
Cory Koedel ◽  
Julian R. Betts
Keyword(s):  
Teacher Effectiveness ◽  
Value Added ◽  
Classroom Teaching ◽  
First Year ◽  
Value Added Models ◽  
Value Added Model ◽  
Extended Analysis ◽  
Research Questions ◽  
Value Added Modeling ◽  
Student Sorting

Value-added modeling continues to gain traction as a tool for measuring teacher performance. However, recent research questions the validity of the value-added approach by showing that it does not mitigate student-teacher sorting bias (its presumed primary benefit). Our study explores this critique in more detail. Although we find that estimated teacher effects from some value-added models are severely biased, we also show that a sufficiently complex value-added model that evaluates teachers over multiple years reduces the sorting bias problem to statistical insignificance. One implication of our findings is that data from the first year or two of classroom teaching for novice teachers may be insufficient to make reliable judgments about quality. Overall, our results suggest that in some cases value-added modeling will continue to provide useful information about the effectiveness of educational inputs.

Rethinking Teacher Evaluation: A Conversation about Statistical Inferences and Value-Added Models

2013 ◽  
Vol 83 (2) ◽  
pp. 349-370 ◽  
Author(s):  
Kimberlee Callister Everson ◽  
Erika Feinauer ◽  
Richard Sudweeks
Keyword(s):  
Teacher Evaluation ◽  
Teacher Effectiveness ◽  
Value Added ◽  
Statistical Methodology ◽  
Current Standard ◽  
Accountability System ◽  
Statistical Inferences ◽  
Value Added Models ◽  
Value Added Modeling

In this article, the authors provide a methodological critique of the current standard of value-added modeling forwarded in educational policy contexts as a means of measuring teacher effectiveness. Conventional value-added estimates of teacher quality are attempts to determine to what degree a teacher would theoretically contribute, on average, to the test score gains of any student in the accountability population (i.e., district or state). Everson, Feinauer, and Sudweeks suggest an alternative statistical methodology, propensity score matching, which allows estimation of how well a teacher performs relative to teachers assigned comparable classes of students. This approach more closely fits the appropriate role of an accountability system: to estimate how well employees perform in the job to which they are actually assigned. It also has the benefit of requiring fewer statistical assumptions—assumptions that are frequently violated in value-added modeling. The authors conclude that this alternative method allows for more appropriate and policy-relevant inferences about the performance of teachers.

scholarly journals Investigating More Effective and Less Effective EFL Learners’Attitudes and Motivational Orientation

2018 ◽  
Vol 9 (5) ◽  
pp. 58
Author(s):  
Mohammadreza Yousefi Halvaei ◽  
Ali Akbar Ansarin
Keyword(s):  
Individual Differences ◽  
Graduate Program ◽  
Motivational Orientation ◽  
Classroom Teaching ◽  
First Year ◽  
Chi Square ◽  
Motivational Orientations ◽  
Efl Learners ◽  
Chi Square Test ◽  
Research Questions

The present study investigated the differences in attitudes and motivation between more effective and less effective learners studying at the first year of graduate university program. Based on this aim, first, the researcher developed the three-part 35-item attitude-motivation questionnaire of the present study based on the AMTB developed by Gardner (1985). Second, he selected 441 students (224 male & 217 female) studying at the first year of their graduate program at universities in East Azarbayjan province (Iran) as the participants of the study. Third, he administered the questionnaire of the study to the selected participants and asked them to complete it. Finally, he analyzed the data of the study in order to answer the mentioned research questions. The chi-square test was utilized to analyze the collected data of the study. The results of the study showed that there were significant differences between more effective and less effective learners’ attitudes and motivation. Based on the results of the study, it was argued that, the EFL syllabus designers and teachers should help the learners to become aware of their attitudes, and motivational orientations. Furthermore, they should make an effort to tailor classroom materials and classroom teaching to the EFL learners’ individual differences.

scholarly journals Estimating Private School Premium for Primary School Children in Ethiopia: Evidence from Individual-level Panel Data

2019 ◽  
Vol 20 (1) ◽  
pp. 26-44
Author(s):  
Godstime Osekhebhen Eigbiremolen
Keyword(s):  
Public Schools ◽  
Private School ◽  
Primary Schools ◽  
Value Added ◽  
Primary School Children ◽  
Individual Level ◽  
Value Added Models ◽  
Value Added Model ◽  
Private And Public ◽  
Unobserved Ability

This article presents the first value-added model of private school effect in Ethiopia, using the unique Young Lives longitudinal data. I found a substantial and statistically significant private school premium (about 0.5 standard deviation) in Maths, but not in Peabody Picture Vocabulary Test (PPVT). Private school premium works for both low and high ability children. The results are robust to sorting on unobserved ability, grouping on lag structures and transfer between private and public schools. Combined with available contextual data, empirical evidence suggests that the effectiveness of private primary schools may be due to more learning time and teacher’s attention enjoyed by students. I also attempted to contribute methodologically to the literature by directly testing the structural assumption underpinning value-added models.

scholarly journals Impact of Contextual Predictors on Value-Added Teacher Effectiveness Estimates

2020 ◽  
Vol 10 (12) ◽  
pp. 390
Author(s):  
Ismail Aslantas
Keyword(s):  
Teacher Effectiveness ◽  
Test Scores ◽  
Value Added ◽  
School Characteristics ◽  
School Life ◽  
Student Academic Performance ◽  
Value Added Models ◽  
Math Test ◽  
Math Test Scores ◽  
The Relationship

It is widely believed that the teacher is one of the most important factors influencing a student’s success at school. In many countries, teachers’ salaries and promotion prospects are determined by their students’ performance. Value-added models (VAMs) are increasingly used to measure teacher effectiveness to reward or penalize teachers. The aim of this paper is to examine the relationship between teacher effectiveness and student academic performance, controlling for other contextual factors, such as student and school characteristics. The data are based on 7543 Grade 8 students matched with 230 teachers from one province in Turkey. To test how much progress in student academic achievement can be attributed to a teacher, a series of regression analyses were run including contextual predictors at the student, school and teacher/classroom level. The results show that approximately half of the differences in students’ math test scores can be explained by their prior attainment alone (47%). Other factors, such as teacher and school characteristics explain very little the variance in students’ test scores once the prior attainment is taken into account. This suggests that teachers add little to students’ later performance. The implication, therefore, is that any intervention to improve students’ achievement should be introduced much earlier in their school life. However, this does not mean that teachers are not important. Teachers are key to schools and student learning, even if they are not differentially effective from each other in the local (or any) school system. Therefore, systems that attempt to differentiate “effective” from “ineffective” teachers may not be fair to some teachers.

Measuring Teacher Effectiveness Using Value-Added Models of High School Achievement

2016 ◽  
Vol 118 (13) ◽  
pp. 1-32
Author(s):  
Nandita G. Gawade ◽  
Robert H. Meyer
Keyword(s):  
High School ◽  
Teacher Effectiveness ◽  
School Achievement ◽  
School Context ◽  
Value Added ◽  
High School Achievement ◽  
Core Subjects ◽  
Value Added Models ◽  
Supplemental Course ◽  
Secondary Grades

This article uses empirical data to consider the consequences of particular characteristics of instruction and testing in high school for the modeling and estimation of value-added measures of school or teacher effectiveness. Unlike Mathematics and Reading for most elementary and middle school grades, there is a lack of annual testing of students in all secondary grades and subjects. The development of value-added models in high school is complicated by the resulting unavailability of direct measures of prior knowledge and readiness of the student for the relevant course. Another distinction between high school and earlier grades is the presence of greater differentiated instruction in high school caused by supplemental course requirements or by student self-selection into different courses. We show that the traditional value-added model used in NCLB grades and subjects can be generalized to the high school context. Specifically, prior-year test scores in related or core subjects can be used to control for differences in student aptitude for the course or subject being evaluated. Similarly, we can account for relevant differences in classroom characteristics—such as the average prior achievement of the students in the classroom—if they are assumed to be beyond a teacher's control.

scholarly journals Student Sorting and Bias in Value-Added Estimation: Selection on Observables and Unobservables

2009 ◽  
Vol 4 (4) ◽  
pp. 537-571 ◽  
Author(s):  
Jesse Rothstein
Keyword(s):  
Value Added ◽  
Fourth Grade ◽  
Causal Effects ◽  
Fifth Grade ◽  
Amount Of Information ◽  
Value Added Models ◽  
Sorting Process ◽  
Unobserved Variables ◽  
Student Sorting

Nonrandom assignment of students to teachers can bias value-added estimates of teachers' causal effects. Rothstein (2008, 2010) shows that typical value-added models indicate large counterfactual effects of fifth-grade teachers on students' fourth-grade learning, indicating that classroom assignments are far from random. This article quantifies the resulting biases in estimates of fifth-grade teachers' causal effects from several value-added models, under varying assumptions about the assignment process. If assignments are assumed to depend only on observables, the most commonly used specifications are subject to important bias, but other feasible specifications are nearly free of bias. I also consider the case in which assignments depend on unobserved variables. I use the across-classroom variance of observables to calibrate several models of the sorting process. Results indicate that even the best feasible value-added models may be substantially biased, with the magnitude of the bias depending on the amount of information available for use in classroom assignments.

Measuring Teacher Effectiveness in Gifted Education

2011 ◽  
Vol 22 (5) ◽  
pp. 750-770 ◽  
Author(s):  
Megan E. Welsh
Keyword(s):  
Teacher Effectiveness ◽  
Gifted Education ◽  
Evaluation System ◽  
Value Added ◽  
New Teacher ◽  
Pedagogical Content ◽  
Quality Instruction ◽  
Observation Systems ◽  
Value Added Models ◽  
New Teacher Evaluation

States and districts are under increasing pressure to evaluate the effectiveness of their teachers and to ensure that all students receive high-quality instruction. This article describes some of the challenges associated with current effectiveness approaches, including paper-and-pencil tests of pedagogical content knowledge, classroom observation systems, and value-added models. It proposes development of a new teacher evaluation system using a virtual reality environment and describes how innovations in educational measurement and technology can be used to develop an improved teacher effectiveness measure.

Sign in / Sign up

Export Citation Format

Share Document