Validating Teacher Effects on Students’ Attitudes and Behaviors: Evidence from Random Assignment of Teachers to Students

2018 ◽  
Vol 13 (3) ◽  
pp. 281-309 ◽  
Author(s):  
David Blazar

There is growing interest among researchers, policy makers, and practitioners in identifying teachers who are skilled at improving student outcomes beyond test scores. However, questions remain about the validity of these teacher effect estimates. Leveraging the random assignment of teachers to classes, I find that teachers have causal effects on their students’ self-reported behavior in class, self-efficacy in math, and happiness in class that are similar in magnitude to effects on math test scores. Weak correlations between teacher effects on different student outcomes indicate that these measures capture unique skills that teachers bring to the classroom. Teacher effects calculated in nonexperimental data are related to these same outcomes following random assignment, revealing that they contain important information content on teachers. However, for some nonexperimental teacher effect estimates, large and potentially important degrees of bias remain. These results suggest that researchers and policy makers should proceed with caution when using these measures. They likely are more appropriate for low-stakes decisions—such as matching teachers to professional development—than for high-stakes personnel decisions and accountability.

2009 ◽  
Vol 4 (4) ◽  
pp. 439-467 ◽  
Author(s):  
J. R. Lockwood ◽  
Daniel F. McCaffrey

This article develops a model for longitudinal student achievement data designed to estimate heterogeneity in teacher effects across students of different achievement levels. The model specifies interactions between teacher effects and students' predicted scores on a test, estimating both average effects of individual teachers and interaction terms indicating whether individual teachers are differentially effective with students of different predicted scores. Using various longitudinal data sources, we find evidence of these interactions that is of relatively consistent but modest magnitude across different contexts, accounting for about 10 percent of the total variation in teacher effects across all students. However, the amount that the interactions matter in practice depends on the heterogeneity of the groups of students taught by different teachers. Using empirical estimates of the heterogeneity of students across teachers, we find that the interactions account for about 3–4 percent of total variation in teacher effects on different classes, with somewhat larger values in middle school mathematics. Our findings suggest that ignoring these interactions is not likely to introduce appreciable bias in estimated teacher effects for most teachers in most settings. The results of this study should be of interest to policy makers concerned about the validity of value-added teacher effect estimates.


2011 ◽  
Vol 113 (12) ◽  
pp. 2633-2669 ◽  
Author(s):  
Julian Vasquez Heilig

Background/Context The prevailing theory of action underlying No Child Left Behind's high-stakes testing and accountability ratings is that schools and students held accountable to these measures will automatically increase educational output as educators try harder, schools will adopt more effective methods, and students will learn more. In Texas, the centerpiece of high school accountability is the pressure to improve exit test scores, a battery of minimum competency exams that students have to pass to graduate from high school. Despite the theory underlying accountability, it is unknown whether policies that reward and sanction schools and students based on high-stakes tests improve English learner (EL) student outcomes over the long term. Purpose/Objective/Research Question/Focus of Study The purpose of the research is to better understand the interaction between high-stakes testing, accountability, and ELs. This study asks the following questions: Have student outcomes for ELs improved since the inception of accountability in Texas? To what extent does social capital theory inform our understanding of the impact of high-stakes exit testing on EL exit test performance in Texas high schools? What are the perceptions of teachers, principals, and students regarding the effects of high-stakes testing and accountability on ELs? Research Design This article reviews longitudinal student outcomes (test scores, dropout, grade retention, and graduation rates) for Texas ELs from the inception of accountability in 1993. To understand the interaction between ELs and high-stakes exams, the researcher undertook qualitative field work in high schools in four Texas districts with large numbers of ELs to understand how the life contexts of ELs interact with Texas-style high-stakes testing and accountability policies. Via administrator, teacher, and student perceptions of exit testing, the article attempts to shed light on the academic challenges faced by ELs in the current accountability context. Conclusions/Recommendations This article underscores the legitimacy of the concern that ELs experience unintended consequences associated with high-stakes exit testing and accountability policy and suggests that social justice and equity are ratiocinative critiques of high-stakes testing and accountability policies. The next round of federal and state educational policy must be a mandate that provides support for ELs to meet performance standards by providing evidence-based solutions: appropriate curriculum, pedagogy, and well-trained teachers. Furthermore, policy makers, practitioners, and researchers should be cognizant of the less intrusive approach that many ELs and their families have toward schools by reconsidering whether “one size fits all” high-stakes exit testing policies are plausible for increasingly heterogeneous student populations. The use of multiple measures of EL student success in content areas, such as portfolios, is an accountability mechanism that makes sense, not just for ELs, but for all students.


2021 ◽  
Vol 8 (1) ◽  
Author(s):  
Dmitri Rozgonjuk ◽  
Karin Täht ◽  
Kristjan Vassil

Abstract Background The excessive use of Internet-based technologies has received a considerable attention over the past years. Despite this, there is relatively little research on how general Internet usage patterns at and outside of school as well as on weekends may be associated with mathematics achievement. Moreover, only a handful of studies have implemented a longitudinal or repeated-measures approach on this research question. The aim of the current study was to fill that gap. Specifically, we investigated the potential associations of Internet use at and outside of school as well as on weekends with mathematics test performance in both high- and low-stakes testing conditions over a period of 3 years in a representative sample of Estonian teenagers. Methods PISA 2015 survey data in conjunction with national educational registry data were used for the current study. Specifically, Internet use at and outside of school as well as on weekends were queried during the PISA 2015 survey. In addition, the data set included PISA mathematics test results from 4113 Estonian 9th-grade students. Furthermore, 3758 of these students also had a 9th-grade national mathematics exam score from a couple of months after the PISA survey. Finally, of these students, the results of 12th-grade mathematics national exam scores were available for 1612 and 1174 students for “wide” (comprehensive) and “narrow” (less comprehensive) mathematics exams, respectively. Results The results showed that the rather low-stakes PISA mathematics test scores correlated well with the high-stakes national mathematics exam scores obtained from the 9th (completed a couple of months after the PISA survey) and 12th grade (completed approximately 3 years after the PISA survey), with correlation values ranging from r = .438 to .557. Furthermore, socioeconomic status index was positively correlated with all mathematics scores (ranging from r = .162 to .305). Controlled for age and gender, the results also showed that students who reported using Internet the longest tended to have, on average, the lowest mathematics scores in all tests across 3 years. Although effect sizes were generally small, they seemed to be more pronounced in Internet use at school. Conclusions Based on these results, one may notice that significantly longer time spent on Internet use at and outside of school as well as on weekends may be associated with poorer mathematics performance. These results are somewhat in line with research outlining the potentially negative associations between longer time spent on digital technology use and daily life outcomes.


2021 ◽  
pp. 109442812199908
Author(s):  
Yin Lin

Forced-choice (FC) assessments of noncognitive psychological constructs (e.g., personality, behavioral tendencies) are popular in high-stakes organizational testing scenarios (e.g., informing hiring decisions) due to their enhanced resistance against response distortions (e.g., faking good, impression management). The measurement precisions of FC assessment scores used to inform personnel decisions are of paramount importance in practice. Different types of reliability estimates are reported for FC assessment scores in current publications, while consensus on best practices appears to be lacking. In order to provide understanding and structure around the reporting of FC reliability, this study systematically examined different types of reliability estimation methods for Thurstonian IRT-based FC assessment scores: their theoretical differences were discussed, and their numerical differences were illustrated through a series of simulations and empirical studies. In doing so, this study provides a practical guide for appraising different reliability estimation methods for IRT-based FC assessment scores.


2004 ◽  
Vol 26 (3) ◽  
pp. 237-257 ◽  
Author(s):  
Barbara Nye ◽  
Spyros Konstantopoulos ◽  
Larry V. Hedges

It is widely accepted that teachers differ in their effectiveness, yet the empirical evidence regarding teacher effectiveness is weak. The existing evidence is mainly drawn from econometric studies that use covariates to attempt to control for selection effects that might bias results. We use data from a four-year experiment in which teachers and students were randomly assigned to classes to estimate teacher effects on student achievement. Teacher effects are estimated as between-teacher (but within-school) variance components of achievement status and residualized achievement gains. Our estimates of teacher effects on achievement gains are similar in magnitude to those of previous econometric studies, but we find larger effects on mathematics achievement than on reading achievement. The estimated relation of teacher experience with student achievement gains is substantial, but is statistically significant only for 2nd-grade reading and 3rd-grade mathematics achievement. We also find much larger teacher effect variance in low socioeconomic status (SES) schools than in high SES schools.


2020 ◽  
pp. 027347532091231 ◽  
Author(s):  
William Humphrey ◽  
Debbie Laverie ◽  
Caroline Muñoz

Recent calls to the academy challenge marketing educators to prepare new graduates for a complex, high-stakes marketing environment driven by technology. We explore the value of utilizing readily available marketing technology (martech) learning platforms as a means to stimulate preparedness for new careers or career advancement. Specifically, we explore the potential of using Salesforce Trailhead digital badges as a learning platform. We discuss how faculty can develop a Trailmix for their classes based on their specific learning outcomes. Next, we test the efficacy and student outcomes of using a marketing technology–focused digital badge learning platform.


1997 ◽  
Vol 24 (3) ◽  
pp. 207-212 ◽  
Author(s):  
Donelson R. Forsyth ◽  
C. Ray Archer

Computer-based teaching methods can improve the transfer of information, increase instructional focus on conceptual and methodological skills, enhance motivation, and stimulate the development of expressive skills. After reviewing a number of studies of computer-based applications, we report a correlational study of psychology students' attitudes and achievement in a technologically enhanced classroom. The results indicated that (a) students rated the computer-based instructional components positively, (b) those with weak academic backgrounds who consistently used the technology achieved higher test scores than weak students who did not use the technology, and (c) students who dropped out of college the following semester tended to be low users of technology. We also discuss the implications of computer technology for teaching.


1986 ◽  
Vol 33 (9) ◽  
pp. 19-48
Author(s):  
Marilyn N. Suydam ◽  
Mark Driscoll

In any field, one way to plan for a program's excellence is to study those programs that have been recognized for their excellence and to try to isolate the influential factors. During 1983 and 1984, the Northeast Regional Exchange (NEREX) engaged in “A Study of Exemplary Mathematics Programs.” The study, funded by the National Institute of Education, sought to “identify and describe factors and conditions associated with excellence in precollege mathematics.” Excellence was defined by student outcomes—test scores, awards, consistently high course enrollments, and so on.


2019 ◽  
Vol 13 (1) ◽  
pp. 31
Author(s):  
Linyu Liao

As a high-stakes standardized test, IELTS is expected to have comparable forms of test papers so that test takers from different test administration on different dates receive comparable test scores. Therefore, this study examined the text difficulty and task characteristics of four parallel academic IELTS reading tests to reveal to what extent the four tests were comparable in terms of text difficulty, construct coverage, response format, item scope, and task scope. The Coh-Metrix-TEA software was used for the text difficulty analyses and expert judgments were used for task characteristics analyses. The results show that the four reading tests were partly comparable in text difficulty, comparable in terms of construct coverage and item scope, but not comparable in terms of response format and task scope. Based on the findings, implications were discussed on test development and future research.


Sign in / Sign up

Export Citation Format

Share Document