Teacher Merit Pay and Student Test Scores: A Meta-Analysis (with L. Pham and T. Nguyen) [pdf]
In recent years, teacher merit pay programs have garnered considerable political and financial support, spurring rapid growth in the number of research studies investigating the effects of teacher pay incentives. The burgeoning research literature on this topic presents a novel opportunity to synthesize our understanding of how teacher merit pay influences on student test scores, and this study fills that role as a meta-analysis of reported findings from 44 primary studies.Â Our meta-analysis finds that the presence of a merit pay program is associated with a modest, statistically significant, positive effect on student test scores â 0.052 standard deviations. We also find that effects sizes are highly sensitive to program design and study context, which suggests that while some merit pay programs have the potential to improve student test scores in some contexts, researchers and policy-makers should pay close attention to how the program is structured and implemented.
Do Unobserved Student Environmental Characteristics Prevent the Unbiased Estimation of Teacher Effects? Evidence from a Twin Study Design (with S. Doan)
In this paper, we address claims that unobserved student environmental characteristics confound and ultimately prevent the unbiased estimation of teacher effects by implementing a novel twin-by-year fixed effect approach that accounts for the influence of any factors that are held common within a pair of twins, within a given year. Because twins are born to the same parents, are very likely to live in the same household, and by construction in this sample, attend the same school and grade, a host of school and household related factors that are often cited as potential confounders (e.g., parental education, parental marital status, computers in the home) in the teacher effects literature are held fixed and differenced out from the estimating model.
Does Incentive Pay Impact Teacher Turnover? Evidence from Tennessee (with L.L. Taylor)
The Obama Administrationâs Teacher Incentive Fund and Race to the Top Initiative encouraged many states to experiment with teacher incentive pay. In Tennessee, some school districts developed incentive pay plans that provided teachers with bonus awards while other districts incorporated incentive pay into their salary schedules. This paper uses panel data on individual teachers and instrumental variables regression to examine the impact of those incentive programs on teacher retention. We find that Tennessee’s strategic compensation programs had a significant impact on teacher turnover in participating schools. Teachers who received large awards were significantly more likely to be retained while teachers who received no award were significantly more likely to turn over. On net, turnover rates rose in bonus program schools because there was an increase in the percentage of teachers moving to other districts, not because there was a significant increase in the percentage of teachers leaving the public school system in Tennessee. Furthermore, the teachers who are moving between districts in the wake of the strategic compensation program appear more likely to be teachers with low Tennessee Value Added Assessment System (TVAAS) scores than teachers with high TVAAS scores. This is somewhat disconcerting, as it implies that other districts are absorbing teachers who are somewhat less effective than average. The strategic compensation programs may have had unintended consequences for non-program schools and districts.
Do Evaluation Ratings Affect Teachers’ Professional Development Activities? (with C. Koedel, J. Li, and L. Tan)Â Evaluation and PD paper
We document substantial differences across teachers who receive different performance ratings from the educator evaluation system in Tennessee in terms of both the intensity of their self-reported professional development activities, and the extent to which these activities are guided by feedback from their performance evaluations. Using a regression discontinuity design, we test whether ratings per se causally influence these gaps and find that they do not. We then perform an exploratory analysis of an alternative mechanism to explain the gaps. Namely, we examine the link between the feedback that teachers receive during classroom observations and their professional development activities, and find a strong association. We conclude that future research in this area should aim to more rigorously test hypotheses that relate teachersâ experiences during classroom observations to their professional development activities.
The Effect of Children’s Health Insurance Expansion on Student Achievement (with W. Swain)
Numerous scholars, policymakers and advocates who favor a âbroader, bolderâ approach to education policy have argued for the devotion of increase resources to promote student health. The logic is simple. Poor studentsâ treatable health problems, like asthma or hearing and vision impairment pose serious obstacles to their regular attendance and full participation in the learning environment. However, empirical evidence of the academic benefits of large-scale health interventions remains sparse. In the proposed study, I plan to exploit the overlap between the administration of a nationally representative longitudinal education survey (ECLS-K) and the radical expansion of youth health coverage from the implementation of the State Childrenâs Health Insurance Program to examine the effects of a health policy intervention on studentâs academic success. The findings of the proposed dissertation would be of interest to policymakers interested both in issues of educational equity and access to healthcare. If this study could draw a clear link between interventions to improve student health and improved academic achievement, particularly among disadvantaged subgroups of students, it would highlight a potentially important tool for elevating overall academic performance and combating persistent achievement gaps. Related, if broad reaching, expensive health care expansions are causally linked to even small increases in student achievement, the economic returns of the improved achievement could significantly shift cost benefit analyses of interventions targeting student health.
Student-Teacher Race Congruence: New Evidence and Insights from Tennessee (with E. Joshi and S. Doan)
An analysis of existing research on teacher merit pay programs reveals that the highly debated practice is having a positive effect on student outcomes, according to a new Vanderbilt University report.
Teacher merit pay, also known as incentive pay, performance pay and pay-for-performance, offers financial incentives to teachers who meet certain criteria, usually involving improved student test scores.
Despite substantial opposition on several fronts, teacher merit pay programs are growing in popularity with considerable political and financial support. The federal government has awarded more than $2 billion in more than 30 states to design and implement performance pay systems. Under this spotlight, numerous research studies have been conducted over the last decade to evaluate merit pay’s effectiveness.
The new Vanderbilt report was co-led by Matthew G. Springer, assistant professor of public policy and education at Vanderbilt’s Peabody College of education and human development, and two Peabody College doctoral students, Lam D. Pham and Tuan D. Nguyen. They searched through more than 19,000 research reports before focusing on 44 primary studies.
“We found overall that the presence of a merit pay program was associated with a modest, but statistically significant, positive effect on student test scores,” Springer said. “Approximately 74 percent of the effect sizes recorded in our review were positive. The influence was relatively similar across the two subject areas, mathematics and English language arts.”
Among studies conducted in U.S. schools, the academic increase was roughly equivalent to adding three additional weeks of learning to the school year.
“These general findings continue to hold even when we restrict our analysis to those studies utilizing the most rigorous methods,” Nguyen said.
Not all merit pay programs yielded equal results, however. Program impacts varied depending on the design of the incentive pay scheme. For example, merit pay programs rewarding teams of teachers produced an effect almost twice as large those rewarding merit raises on rank-order. That finding lends support to the shared nature of teaching and learning in schools.
“We found that effect sizes were highly sensitive to program design and study context,” Pham said. “This suggests that, while some merit pay programs have the potential to improve student test scores in some contexts, the more pertinent question that researchers and policymakers should consider is how the program is structured and implemented.”
Emerging studies also suggest that merit pay can improve teacher recruitment and retention, which has been found to contribute to many positive outcomes for students, particularly those in low-income areas. Springer suggests continued investigation into teacher labor market outcomes, especially the effects of pay incentives on the mobility patterns of highly effective teachers, and the exit decisions of traditionally low-performing teachers.
Download a PDF of “Teacher Merit Pay and Student Test Scores: A Meta-Analysis.”
Follow @eduspringer and @lamdspham on Twitter.
Joan Brasher, (615) 322-NEWS