(Photo: J. Maus/BikePortland)
This guest essay was written by Joe Cortright, an urban economist with Impresa Consulting who also runs CityObservatory.org.
There are a lot of big questions about the Oregon Department of Transportation’s (ODOT) competence and capability. Unfortunately the new $1 million audit undertaken by McKinsey and Company answers none of them.
The audit is misleading, inaccurate and omits key facts about ODOT’s substantive management problems. In effect, the audit actually conceals some of ODOT’s most expensive blunders.
An audit that doesn’t acknowledge, much less analyze, obvious problems can’t provide meaningful solutions. For example, auditors who can’t even correctly identify the cost of the agency’s largest construction project—and who purposely omit it from their one statistical chart showing cost overruns—aren’t worth the money they’re being paid, because they haven’t done their jobs.
Why does this matter? Because the Oregon legislature is about to begin a debate over transportation funding that could result in hundreds of millions of dollars flowing through ODOT’s hands.
ODOT’s hand-picked–and apparently spoon-fed–auditors have essentially failed to look at any of the agency’s well known project planning and execution blunders of the past decade.
A little history, for those who haven’t been following the issue: This audit was undertaken at Governor Kate Brown’s request, as a way to assure legislators and the public that ODOT could be entrusted with a major new spending program–in the wake of the implosion of a 2015 transportation funding proposal because of false and misleading statements from ODOT staff about its effects on air pollution. ODOT originally chose to have the audit performed by a vendor with close ties to the department, but that contract was cancelled after press reports revealed the conflict of interest and showed the vendor was angling to be named director of ODOT. Finally, the state selected management consultants McKinsey and Company to undertake the audit for a budget of one million dollars. Earlier this week, in response to public records requests from the Oregonian and the Portland Tribune, the report (PDF) was made public.
According to her spokesperson, Governor Brown expects this report to be: “a thorough and independent performance review of the Oregon Department of Transportation.” Judged by that standard, this audit is a failure. It is neither thorough (it omits facts and gets others wrong) and it doesn’t address the agency’s performance in a meaningful way.
ODOT’s hand-picked–and apparently spoon-fed–auditors have essentially failed to look at any of the agency’s well known project planning and execution blunders of the past decade. Instead the auditors looked at data on a range of administrative trivia and conducted limited scope surveys with the department’s stakeholders. The result is a product that sheds almost no light on the substantive problems that continue to plague the agency. Rather than giving outsiders the sense that ODOT is competent, it merely shows that the agency is effective in blocking any effort to ask serious questions about its mission or performance, such as:
· Is this agency capable of managing large construction projects?
· Does it do what it says?
· Can it undertake projects without massive delays and cost overruns?
The answer to each of these questions is “No.” But you will find little if any useful information about any of these subjects in the McKinsey report.
Ignoring a quarter billion dollar cost overrun
This is an agency that’s been repeatedly plagued by cost-overruns and management failures on major projects—we’ve listed a series of them below. There are plenty of examples to choose from, but unquestionably, the poster child for ODOT incompetence is the Pioneer Mountain-Eddyville/Highway 20 project, a five-mile highway in the Coast Range between Corvallis and Newport. The project, announced in 2004, was supposed to cost $110 million and take four years. After a series of mis-steps–including illegal pollution, firing a contractor, demolishing four-partially built bridges, and repeatedly re-designing the project–it was finished in 2016 at a cost of more than $366 million. It’s ODOT’s most expensive single project.
You might think that the agency’s biggest project, especially one that went a quarter of a billion dollars over budget, and was delayed for years, might draw the attention of auditors. But you would be wrong. Look as hard as you like and you’ll find exactly one reference to it in the McKinsey report, buried in the footnote to a statistical chart on page 25 (see below).
And here’s what’s remarkable about that: McKinsey’s chart shows how much ODOT projects were over or under budget. And this chart excludes Pioneer Mountain/Eddyville.
This would be rather like the White Star Line reporting the on-time arrivals of its ships traveling between London and New York with a footnote saying “This data doesn’t include the indefinitely delayed arrival of RMS Titanic.”
Bad as this is, it’s actually worse: McKinsey’s footnote says: “This excludes US20 Pioneer Mtn-Eddyville project (Overall performed 27% higher than $140M original authorized amount).” This statement is Orwellian double-speak on at least four different levels. McKinsey has (1) mis-stated the actual original project cost estimate–which was $110 million, not $140 million; (2) omitted the actual final cost of $366 million, (3) implied (but not stated) that the actual total cost was $177.8 million (i.e. 140+27%), and (4) described a cost overrun as performing higher.
The initial cost of the project (from its Draft Environmental Impact Statement), and its final cost (from press accounts and ODOT reports) is a matter of public record. You can find both with a Google search in a couple of seconds. But McKinsey apparently could neither report the correct cost of the project, nor bring itself to use the word “overrun,” in its report. These two decisions illustrate how McKinsey, rather than revealing and fairly examining ODOT’s problems, is actually complicit in minimizing them.
Advertisement
Staying “On Schedule” by Moving the Goalposts
A second question is whether ODOT can bring its projects in on time. McKinsey reprints very summary data—again apparently obtained from ODOT, about project timeliness. There’s no indication that McKinsey independently verified these data. The report makes no mention delays of the Pioneer Mountain Eddyville project, which was originally scheduled to be completed in 2009, and wasn’t completed until 2016.
We know from previous experience that ODOT regularly “moves the goalposts” to calculate whether its project delivery is “on time.” Take the experience with the Columbia River Crossing project. Even though it was delayed multiple times (and not actually ever started), ODOT officials repeatedly assured the press and public that it was “on schedule.” They did so by simply replacing the previous schedule with a new one, and then claiming everything was fine.
To most observers, “on schedule” means completed by a specific date that is established in advance. CRC officials regularly changed the project schedule, and as a result postponed the expected date of completion, and then used this new—and later—completion date to report that the project is “on the current schedule.” In fact, ODOT Director Matt Garrett used exactly this carefully chosen phraseology to report after the project released a report extending the completion date by 26 months (emphasis added):
Despite ongoing controversy about the scope and cost of the Columbia River Crossing, project managers insist it is still on schedule and have even reduced its estimated cost by $100 million.
“The direction last spring from Gov. Gregoire and Gov. Kitzhaber on bridge type allowed us to move forward on a solid path to meet the current schedule,” said Oregon Transportation Director Matt Garrett on Friday. (Redden, J. (2011, August 26) “Officials shave $100 million from I-5 bridge project budget” Portland Tribune. .
Unless the auditors verified that the data given to them reported each project’s original scheduled completion date, rather than a revised date that “moved the goalposts,” it cannot make any meaningful statement about ODOT’s ability to deliver projects on time.
An audit that conceals facts and emphasizes trivia
As a result of these fundamental failures to validate basic facts (the actual cost of large projects, whether projects were completed according to their originally announced schedules), McKinsey’s audit is worse than useless: it is actually a source of mis-information that effectively white-washes the agency’s multiple management failures.
An audit that fails to acknowledge and document the agencies failures cannot provide any practical advice on how to prevent their recurrence.
Instead, the McKinsey audit is obsessed with superficial indicators with no demonstrated relevance to actual project delivery, cost containment or schedule completion. It focuses on arcane statistics like “paychecks processed per payroll FTE,” “office occupancy ratio,” and “median square per employee.” It makes sweeping, meaningless and undocumented assertions, such as “ODOT’s project delivery organization is highly talented,” and “ODOT’s culture of achievement and talented team of dedicated professionals ensure that projects are completed with their intended purpose.” None of this has any utility to policy-makers or the public in determining whether ODOT is actually capable of doing its job.
It’s actually possible to undertake a meaningful audit of an agency’s competence and the relevance of its program’s and strategies to the state’s policy objectives. California commissioned just such an audit of its transportation department (CalTrans), just three years ago. What they got, for a much larger department, and for a lower price tag, was a comprehensive assessment and roadmap as to how to overhaul the agency and its programs to help California build a transportation system that would meet its future needs in an environmentally sustainable way. Oregon got essentially nothing of value: the auditor’s report even lacks recommendations, which will be developed by the state’s Department of Administrative Services, with input and guidance from ODOT.
Legislators and others who were expecting an audit to give us confidence that ODOT is worthy of a massive increase in funding and will make good use of a major increase in gas taxes should be deeply troubled. We have important needs — like keeping our highways in good repair and strengthening roads and bridges survive the coming Cascadia earthquake, but we also need assurance that our investment won’t be squandered by an agency that is obsessed with building big highway projects regardless of cost.
For more background about the crumbling credibility of ODOT, read this story about the Oregon Transportation’s Commission’s attempt to wrest more control of transportation projects and funding away from the agency. The Salem Breakfast on Bikes blog also has some good commentary about the audit and OTC/ODOT’s relationship.
— Joe Cortright, @Joe_Cortright on Twitter
BikePortland is supported by the community (that means you!). Please become a subscriber or make a donation today.