Philadelphia district says 13 schools need deeper study on cheating

Thirteen Philadelphia public schools' state tests bear further investigation for possible cheating, officials said Monday.

District staff made the announcement at a news conference called to address a state report analyzing 2009 Pennsylvania System of School Assessment exams. The state had earlier directed Philadelphia to offer explanations for 28 schools it had flagged.

Of that group, 13 had both considerable jumps in test scores and flags for answers being changed and should be looked at more closely, district officials said. They declined to name the 13 and emphasized that they needed more information from the state, plus assistance conducting any further investigations.

Officials also repeatedly raised questions Monday about the state's report, saying they have "many concerns" about the data analysis done by vendor Data Recognition Corp. (DRC).

Though saying DRC had been a good partner for years, Daniel Piotrowski, the district's executive director of accountability and assessment, said the company did not look at long-term trends and did not typically do this type of analysis. He said DRC's examination of erasures was flawed, focusing too narrowly on wrong-to-right answer changes.

Still, deputy chief of accountability and educational technology Fran Newberg said, "We recognize the gravity of these potential allegations. And we want to pursue any cheating. We want to deal with it seriously. We want sanctions."

If the allegations are true, she said, "people can lose jobs and should lose jobs."

Beginning this school year, the district will revise its program for monitoring state exams, Newberg said. Under the current system, monitors trained by the state show up unannounced at schools to observe how exams are given.

"We feel proud of that system," Newberg said, "but we feel that that's not enough."

Under a tiered monitoring plan, schools with suspected testing irregularities may be prevented from opening testing materials themselves; there, central office staff would control testing materials, locking them up at the end of each day.

"The idea that we're policing ourselves - that doesn't go over well, as all of you know," Newberg said. "We want some outsiders to participate in this investigation."

As The Inquirer reported in May, multiple teachers at Roosevelt Middle School in East Germantown said they witnessed many test security breaches, and they attributed a remarkable two-year increase in state test scores to cheating.

The district this spring conducted its own investigation into Roosevelt testing improprieties and found claims of cheating unfounded.

Last month, Pennsylvania Secretary of Education Ronald Tomalis ordered Philadelphia and dozens of other districts statewide to look into possible testing improprieties. The directive came after the Philadelphia Public School Notebook reported on the previously unknown forensic analysis of 2009 statewide testing data.

District officials stressed that many of the tips they had received about possible cheating had been inconclusive.

In many cases, "I think what we have is very vague allegations against schools," Piotrowski said.

"We really want anyone that knows anything to step forward and provide names," Newberg said.

Amid intense pressure to improve, Philadelphia public school students' test scores have been increasing for the last nine years. Newberg said that these potentially questionable scores would not mar that record.

"We also have to acknowledge for the record that if we were to remove the students that are part of this file, our nine-year trend in achievement would still hold," she said.

The state directed districts to investigate only those schools flagged for abnormal patterns in three or more areas. That yielded the 28 schools - from elementary to high school - that were the subject of the district's report to the state.

But according to an Inquirer analysis, 88 schools across the district - a third of all schools - were flagged for erasures, with nearly half marked for both math and reading. About 40 of those schools showed aberrant erasures in more than one grade.

Four of the district's schools were assigned the highest "threat score" of 99.9 for erasures - meaning the odds were astronomical that such an erasure pattern would occur by chance. Those schools - Olney Elementary, Tilden Middle, Barratt Middle, and Strawberry Mansion High - had a combined 278 students whose tests were considered unusual.

Roosevelt is one of 20 district schools identified as having abnormal erasure patterns involving at least 10 flagged tests in one subject and one grade. But that may be a conservative tally.

Unlike threat scores given to schools, the data analysis was far more lenient on student erasures. Only students with threat scores of 50 or greater were counted. "Using a flagging criterion of 10 resulted in too many students being flagged," the report said. "We wanted to keep the number of flagged students to a manageable number."

Philadelphia and other districts had 30 days to complete their reports. The district sent its findings Monday.

Tim Eller, a spokesman for Tomalis, said Monday that the reports were "just one step" in a larger process.

"The department is still going to do an investigatory review based on their own analysis, and see if further investigation is needed," Eller said.

Superintendent Arlene C. Ackerman did not attend Monday's news conference. Ackerman was ill and not at work, a spokeswoman said.