District seeks new index to rate school performance

In an unexpected move, the Philadelphia School District announced Wednesday that it was seeking an outside firm to develop a model to replace its controversial school performance index after finding flaws in the system.

Developed by the staff in consultation with others in 2009, the index uses state test results, attendance, survey results, and other data to rate the performance of every district school and charter school in the city on a scale of one to 10, with one being the highest.

The scores are one of several factors the School Reform Commission considers when selecting district schools for academic overhaul or closing and whether charter schools should be renewed or allowed to expand. The scores also help parents compare schools.

Some charter operators, principals, parents, and teachers have complained about inaccuracies in the school performance index, which has been dubbed the "SPI." Most said their schools deserved higher scores than they had received.

Paul Kihn, deputy superintendent, said the district began a comprehensive review of the index methodology and calculations after an analyst in the district's charter school office in May reported concerns that some of the 2011 scores could be inaccurate. He declined to name the analyst, saying it was a personnel matter.

Kihn said the district released a request Wednesday that asks data analytic firms to submit proposals for recalculating the performance index. The project includes determining the school scores for 2011-12 and refiguring the scores for the 2009-10 and 2010-11 academic years.

The district, he said, expects that outside funds will pay for the project and that no district dollars will be used. Kihn said the district cannot disclose the philanthropy yet because talks are continuing.

"It is very important to note that the underlying data that feeds into the SPI is not in question and is not considered to be faulty," the district said in a statement announcing the recalculation plan. "What is in question is the calculation that was done to produce the SPI scores for traditional public and charter schools."

To ensure transparency, Kihn said, all firms seeking the contract must agree to prepare reports and make their data and calculations public.

In addition, the district plans to retain a national expert on data analysis to oversee the process. He said the expert would work with a small advisory group of four to five district and charter representatives.

Under the district's accelerated timetable, prospective firms must submit proposals by Nov. 7 so the SRC can vote on awarding the contract Nov. 15.

Kihn said that the district hopes the firm that is awarded the contract will be able to produce the 2011-12 school scores by the spring, when the SRC considers charter expansions and new operating agreements for those whose charters expire in 2013.

He said the district will rely on underlying data rather than the scores this year when it considers closing schools as part of its facilities master plan or moves to turn low-performing district schools over to charter school operators.

Kihn told charter school operators in an e-mail Wednesday that the district would hire a firm to redo the index.

Lawrence Jones, president of the Pennsylvania Coalition of Public Charter Schools, said that while charter operators have encouraged the district to make changes to the index, he was surprised by Kihn's e-mail.

"We want [the index] to be as accurate as possible," said Jones, who is also the chief executive of the Richard Allen Preparatory Charter School in Southwest Philadelphia. But he said he primarily was concerned that some past charter school decisions may have been based on faulty scores.