As children return to school after Labor Day in the United States, it’s a good time to think about the evaluation of workers who have such a tremendous impact on our human capital stock – teachers – and look at the current scientific evidence on measuring teacher performance (referred to hereafter as teacher effectiveness, the term used in the literature).
Pay and incentives for teachers have always been controversial subjects. Designing appropriate incentives is a particular challenge due to the lack of appropriate data and the difficulty of reliably measuring teacher effectiveness.
The No Child Left Behind program has generated lots of test score data that researchers have used to model teacher effectiveness. The data requirements to adequately measure teacher effectiveness are quite onerous. For example, you need test score data over a number of years for children for each teacher and need information on individual teachers and students so that you can isolate the effects of different factors.
The Summer 2010 edition of The Journal of Economic Perspectives has an article entitled “Searching for Effective Teachers with Imperfect Information” by Douglas Staiger and Jonah Rockoff. They summarize the evidence on five key parameters regarding teacher effectiveness and then suggest how school leaders can use the currently available but imperfect measures of teacher effectiveness to recruit, evaluate and retain teachers.
The parameters they discuss are useful data points to keep in mind when thinking about teacher effectiveness and incentives and so I’ve translated them below from the (bold-faced) research language, where necessary. Even if some statements appear to be common sense, it is important to recognize that these are research results rather than opinions. You can find all the references in the original article.
- Teacher productivity based on gains in student achievement is heterogeneous. If you look at students’ performance in different teachers’ classrooms, even controlling for student characteristics, the data show that there are good teachers and bad teachers (in the sense that the same student performs well in some teachers’ classrooms and not as well in others’). This implies that there are potentially large gains to students if school leaders can attract and retain good teachers and “manage” the employment of bad teachers – through refusing tenure or providing remedial training.
- Estimates of heterogeneous teacher effects include a substantial noise component. It is hard to reliably measure teacher effectiveness because you cannot control for everything in the measurement – e.g., the detrimental effect on scores due to a dog barking in the school parking lot during the day of the test. This means that you have to be careful when interpreting results on teacher effectiveness.
- Teachers improve substantially in their first few years on the job. Students typically perform better with experienced teachers. The gap in performance drops rapidly in the first two years and then virtually disappears after the third year. In other words, it takes just a few years of experience for teachers to ramp up to the level of experienced teachers.
- The main cost of teacher turnover is the reduction in student achievement when an experienced teacher is replaced by a novice, not direct hiring costs. Based on the gains that teachers make in their first few years of experience, every time a school district loses an experienced teacher with two or more years of experience and is forced to hire a novice teacher, the students assigned to the novice teacher lose out on student achievement gains. Using estimates of future earnings based on differences in student achievement, it turns out that the economic cost dwarfs the direct costs of teacher hiring. This means that retaining good experienced teachers is very important.
- School leaders have very little ability to select effective teachers during the initial hiring process. For a variety of reasons, school leaders are unable to effectively separate good teachers from bad during the new teacher hiring process. This is demonstrated by the fact that most of the variation in teacher effects occurs among teachers hired into the same school.
After performing some Monte Carlo simulations, the authors conclude that “the current system which focuses on credentials at the time of hire and grants tenure as a matter of course, is at odds with decades of evidence on teacher effectiveness. Instead, teacher recruitment and retention policies should focus on improving our methods of teacher evaluation and use admittedly imperfect measures of teacher effectiveness to identify and retain only the best teachers early in their teaching careers.”
What are the parallels with respect to standard employee screening, performance management and retention?