Contributors
Lance T. Izumi - Contributor
[Courtesty of Pacific Research Institute]

Lance Izumi is Director of Education Studies for the Pacific Research Institute and
Senior Fellow in California Studies. He is a leading expert in education policy and the author of several major PRI studies. [go to Izumi index]

A Lesson For School Accountability
[Lance T. Izumi] 7/26/03

The New York Times recently reported that Texas state auditors have found that Houston schools have been seriously undercounting the number of dropouts. Partisan critics are using the audit results to take shots at U.S. education secretary Rod Paige, a former Houston superintendent, arguing that the city's widely acclaimed education improvement is a myth. The critics, however, are missing the larger point that accountability systems must not create incentives for schools to cheat.

In Texas, the accountability system rates schools based not only on test scores, but also on attendance and dropout rates. While it's hard to disguise a student's poor test score, it is possible for schools to hide their true attendance and dropout rates. Some Houston schools, evidently, claimed that students had transferred to different schools when they had actually dropped out. Other schools simply falsified records to show that no students had dropped out. Masking their real dropout rates allowed many schools to gain a high state ranking, with staffs receiving cash bonuses for the school's "achievement." In other words, cheating paid.

What happened in Texas should be a lesson for California. So far, California's accountability system uses only student scores on state tests to rate schools. However, the law that created the accountability system allows for attendance and dropout rates to be added to the rating calculus. And indeed, the proposed state master plan for education envisions non-test-score factors becoming part of the rating system. Yet in California the dropout rate calculations are just as unreliable as in Texas.

While more than 30 percent of California ninth-graders never graduate from high school, the state reports a high school dropout rate of less than 11 percent. Since the state figure is based on local reporting, a state education official says, "Someone's lying." If attendance and dropout rates become part of California's school rating system, the incentive to lie would only increase.

Some in Texas believe that schools actually encourage lower performing students to drop out. They argue that if such students drop out, and a school covers up the real dropout rate, then the higher achieving students left in the school would skew upward both test scores and the school's ranking. This strategy, though, could be defeated by basing rankings on how much individual scores improve from year to year, so-called "value-added" testing, rather than simply relying on a school's annual schoolwide average test score.

The National Center for Policy Analysis found that of the top 100 schools in Texas based on schoolwide average test scores, only 12 would rank so high if test scores were measured on a value-added basis. Value-added testing, therefore, exposes bad schools that rely on good students to pull up the ranking. Indeed, using value-added testing, a school that has a low schoolwide average score could rank higher than a school with a much higher average if students at the former make higher year-to-year gains than students at the latter. The key is longitudinal improvement not an uninformative one-time snapshot.

California should switch to a value-added testing system rather than adding non-academic factors such as dropout and attendance rates. School accountability systems must be safeguarded against manipulation, which means focusing on student achievement, not smoke and mirrors.

copyright 2003 Pacific Research Institute