If classroom observation is to play a key role in how the success of K-12 teachers are evaluated, then the methods for that observation need to be improved, says a Dartmouth economics professor studying the issue.

Douglas Staiger, the John French Professor of Economics, recently co-authored a research paper exploring the best ways to measure classroom observation of teachers. The findings in the preliminary report released last week by the Bill & Melinda Gates Foundation state that the following criteria need to be met to properly utilize observation as a measurement tool:

• Choose an observation instrument that sets clear expectations.
• Require observers to demonstrate accuracy before they rate teacher practice.
• When high stakes decisions are being made, multiple observations are necessary.
• Track system-level reliability by double scoring some teachers with impartial observers.
• Combine observations with student achievement gains and student feedback.
• Regularly verify that those with stronger observation scores also have stronger student achievement gains on average.

The findings in the observations report are based on 7,491 videos of instruction by 1,333 teachers in grades four through eight from the following six districts: Charlotte-Mecklenburg, N.C.; Dallas, Texas; Denver, Colo.; Hillsborough Co., Fla.; New York City; and Memphis, Tenn. Staiger said teachers provided video for four to eight lessons over the 2009-10 school year and some 900 trained raters took part in the subsequent lesson scoring.

“We collaborated with 25 lead research and organizational partners and thousands of teachers. Most of these partners were responsible for development and implementation of specific measures, e.g. student surveys, student tests, videotaping teachers, scoring those videos, and so on,” Staiger said.

The report is the second in a four-part series called Measures of Effective Teaching (MET) Project of which the goal is to improve the quality of information about teaching effectiveness available to education professionals within states and districts—information that will help them build fair and reliable systems for teacher observation that can be used for a variety of purposes, including feedback, development, and continuous improvement. According to MET’s website, two-thirds of American teachers feel that current evaluations don’t accurately capture the full picture of what they do in the classroom.

Even preliminarily, the findings show commonalities between the various participating school districts.

Staiger said parents of K-12 students would want to know that there are large differences across teachers in their effectiveness, that student test scores, student surveys, and classroom observation of teachers can be used by districts to systematically gather meaningful feedback on teacher effectiveness. State test scores are also an important component of teacher effectiveness, but effective teachers have impacts on a wider range of student outcomes, including student performance on supplemental tests of conceptual understanding in mathematics, a literacy test requiring short-answer responses, and student effort and enjoyment in class.

The recommendations found in the series of reports are designed to provide guidance to both policymakers and practitioners on how evaluation methods can best be used to tell teachers more about the skills that make them most effective and to help districts identify great teaching. Since the report’s release, Staiger has been interviewed by several media outlets to discuss the meaning of the findings.

“Policymakers and practitioners at every level are intensely focused on improving teaching and learning through better evaluation of teachers,” said Staiger, “and these results from the Measures of Effective Teaching project will have an immediate impact on school districts across the country.”