AISD Efficiency Study Thwarted by Math

A week after it was first delivered, district staff and members of the Austin Independent School District board of trustees were already concerned about methodologies used and conclusions reached in the $455,000 districtwide efficiency study commissioned from MGT of America (see "AISD 'Efficiency' Study: Not Exactly What the Doctor Ordered?," June 12). Now additional questions have been raised about the numbers in the report – and whether they even add up.

MGT submitted what was supposed to be the final version of the report to AISD, dated May 29. The district in turn supplied a printed copy of this document to The Austin Chronicle on June 2, the day after it was presented to the AISD board. Credit goes to the Chronicle's eagle-eyed proofers for noticing that the hand-delivered version didn't match the version on the AISD website. Upon further review, the discrepancies do not appear to be incidental, and they call into question the accuracy of the entire report.

The hard copy was the original version, submitted by MGT, and it contained several numerical errors, including some in one of the most important parts of the document: Exhibit ES-1, the executive summary chart showing how much each cost-saving option could produce. MGT added them all up to equal $392,283,538 over five years. Special assistant to the superintendent Milan Sevak said, "When I first got the final draft, I did add the numbers up, just to do a check, and that's when I noticed the difference." AISD staff notified MGT about the problem: Using MGT's numbers, the grand total should have been $441,503,622. This meant MGT's original headline number was off by about $50 million – or possibly $40 million. If the five-year subtotals for each chapter are added together, rather than using the subtotals for each year, the total actually comes to $431,369,271. (A total based on the annual grand subtotals in the released version yields yet another figure: $401.5 million – see chart. Depending on which variable totals you choose, it's a margin of error of roughly 12.5%.)

Prior to its formal release, the report had already gone through feedback and corrections, as MGT had submitted a draft to AISD on April 16 and a second on May 7. Sevak said, "You'd expect, to some degree, that there would be mistakes or things for us to correct," but he described the number as "a lot," and several survived the proofing process. For example, the "original" final report claimed the district has 13,322 middle school students taking 713 classes. There are actually 30,452 middle-schoolers taking 11,029 classes. When contacted about these issues, MGT of America's Austin office declined to comment.

AISD board President Mark Williams said he saw the next stage of dealing with the report as a process of separating out what is currently "a laundry list of possible options," and that he would have liked a larger, qualitative component to the study, to match the quantitative component. But, he said: "If you have problems with the math, then that raises concerns about the rest of the report. That may be fair, it may not be fair, but it's certainly a legitimate concern."

Trustee and board Audit Committee Chair Robert Schneider said the board had only discussed the study in general terms, "rather than any specifics of what an individual cut might mean." There will be a staff response to the study on June 22, Schneider added, "and I would be surprised if the board as a whole would be prepared to make major decisions without questioning either the validity of the numbers or the validity of the assumptions that were made."

The district and members of the board have already voiced concerns that, by proposing cuts that would directly impact educational provision, MGT had ignored a key component of the original request for proposal. But in the early drafts, MGT worked on the principal that every saving could be implemented immediately. "Legally," Sevak said, "we can't do everything in year one, especially in positions where people have contracts, unless we declared financial exigency, which would kill our bond rating." Sevak said there had been discussions between MGT and AISD's Chief Financial Officer Steve West over MGT's headline numbers, and especially the five-year projections, "because they're just taking the numbers from years one and two and carrying them forward." In that context, he said, claiming that the district would save $400 million "makes for a good story," and that some or all of the cash would probably be reallocated to other projects rather than being cut from the budget.

Schneider said the board will continue in its commitment to finding the best ways to allocate its resources. Building the budget was already going to be tough with the arrival of new Superintendent Meria Carstarphen and the expectation that projected shortfalls would push the district into its fund reserves. "To have it clouded ... makes it a very difficult challenge," Schneider said.

Sample Numerical Errors in Executive Summary

Chapter 4, Item 4: School consolidations

Savings: $7.13 million a year for four years

MGT total: $35.6 million

Actual total: $28.5 million

Chapter 11, Item 22: Removing 18 former Johnston High teachers from payroll as permanent substitutes

Savings: $540,000 a year for five years

MGT total: $540,000

Actual total: $2.7 million

Total Five-Year Savings if All Options Used

Savings ("first" final draft): $23.3 million in year one, followed by four years at $94.6 million a year

MGT total: $392.3 million

"Actual" total: $401.5 million

Savings ("second" final draft): $25.3 million in year one, plus four years at $104.1 million a year

MGT corrected total: $441.5 million

"Actual" total: $441.5 million or $431.4 million