Guest opinion: ODOT management audit misleads, omits key facts

A day in Salem-3

We deserve a better ODOT before we hand them new revenue.
(Photo: J. Maus/BikePortland)

This guest essay was written by Joe Cortright, an urban economist with Impresa Consulting who also runs CityObservatory.org.

There are a lot of big questions about the Oregon Department of Transportation’s (ODOT) competence and capability. Unfortunately the new $1 million audit undertaken by McKinsey and Company answers none of them.

The audit is misleading, inaccurate and omits key facts about ODOT’s substantive management problems. In effect, the audit actually conceals some of ODOT’s most expensive blunders.

An audit that doesn’t acknowledge, much less analyze, obvious problems can’t provide meaningful solutions. For example, auditors who can’t even correctly identify the cost of the agency’s largest construction project—and who purposely omit it from their one statistical chart showing cost overruns—aren’t worth the money they’re being paid, because they haven’t done their jobs.

Why does this matter? Because the Oregon legislature is about to begin a debate over transportation funding that could result in hundreds of millions of dollars flowing through ODOT’s hands.

ODOT’s hand-picked–and apparently spoon-fed–auditors have essentially failed to look at any of the agency’s well known project planning and execution blunders of the past decade.

A little history, for those who haven’t been following the issue: This audit was undertaken at Governor Kate Brown’s request, as a way to assure legislators and the public that ODOT could be entrusted with a major new spending program–in the wake of the implosion of a 2015 transportation funding proposal because of false and misleading statements from ODOT staff about its effects on air pollution. ODOT originally chose to have the audit performed by a vendor with close ties to the department, but that contract was cancelled after press reports revealed the conflict of interest and showed the vendor was angling to be named director of ODOT. Finally, the state selected management consultants McKinsey and Company to undertake the audit for a budget of one million dollars. Earlier this week, in response to public records requests from the Oregonian and the Portland Tribune, the report (PDF) was made public.

According to her spokesperson, Governor Brown expects this report to be: “a thorough and independent performance review of the Oregon Department of Transportation.” Judged by that standard, this audit is a failure. It is neither thorough (it omits facts and gets others wrong) and it doesn’t address the agency’s performance in a meaningful way.

ODOT’s hand-picked–and apparently spoon-fed–auditors have essentially failed to look at any of the agency’s well known project planning and execution blunders of the past decade. Instead the auditors looked at data on a range of administrative trivia and conducted limited scope surveys with the department’s stakeholders. The result is a product that sheds almost no light on the substantive problems that continue to plague the agency. Rather than giving outsiders the sense that ODOT is competent, it merely shows that the agency is effective in blocking any effort to ask serious questions about its mission or performance, such as:

· Is this agency capable of managing large construction projects?
· Does it do what it says?
· Can it undertake projects without massive delays and cost overruns?

The answer to each of these questions is “No.” But you will find little if any useful information about any of these subjects in the McKinsey report.

Ignoring a quarter billion dollar cost overrun

The Highway 20 project was $250 million over budget and barely merits a mention.

This is an agency that’s been repeatedly plagued by cost-overruns and management failures on major projects—we’ve listed a series of them below. There are plenty of examples to choose from, but unquestionably, the poster child for ODOT incompetence is the Pioneer Mountain-Eddyville/Highway 20 project, a five-mile highway in the Coast Range between Corvallis and Newport. The project, announced in 2004, was supposed to cost $110 million and take four years. After a series of mis-steps–including illegal pollution, firing a contractor, demolishing four-partially built bridges, and repeatedly re-designing the project–it was finished in 2016 at a cost of more than $366 million. It’s ODOT’s most expensive single project.

You might think that the agency’s biggest project, especially one that went a quarter of a billion dollars over budget, and was delayed for years, might draw the attention of auditors. But you would be wrong. Look as hard as you like and you’ll find exactly one reference to it in the McKinsey report, buried in the footnote to a statistical chart on page 25 (see below).

And here’s what’s remarkable about that: McKinsey’s chart shows how much ODOT projects were over or under budget. And this chart excludes Pioneer Mountain/Eddyville.

This would be rather like the White Star Line reporting the on-time arrivals of its ships traveling between London and New York with a footnote saying “This data doesn’t include the indefinitely delayed arrival of RMS Titanic.”

Bad as this is, it’s actually worse: McKinsey’s footnote says: “This excludes US20 Pioneer Mtn-Eddyville project (Overall performed 27% higher than $140M original authorized amount).” This statement is Orwellian double-speak on at least four different levels. McKinsey has (1) mis-stated the actual original project cost estimate–which was $110 million, not $140 million; (2) omitted the actual final cost of $366 million, (3) implied (but not stated) that the actual total cost was $177.8 million (i.e. 140+27%), and (4) described a cost overrun as performing higher.

The initial cost of the project (from its Draft Environmental Impact Statement), and its final cost (from press accounts and ODOT reports) is a matter of public record. You can find both with a Google search in a couple of seconds. But McKinsey apparently could neither report the correct cost of the project, nor bring itself to use the word “overrun,” in its report. These two decisions illustrate how McKinsey, rather than revealing and fairly examining ODOT’s problems, is actually complicit in minimizing them.

Advertisement

Staying “On Schedule” by Moving the Goalposts

A second question is whether ODOT can bring its projects in on time. McKinsey reprints very summary data—again apparently obtained from ODOT, about project timeliness. There’s no indication that McKinsey independently verified these data. The report makes no mention delays of the Pioneer Mountain Eddyville project, which was originally scheduled to be completed in 2009, and wasn’t completed until 2016.

We know from previous experience that ODOT regularly “moves the goalposts” to calculate whether its project delivery is “on time.” Take the experience with the Columbia River Crossing project. Even though it was delayed multiple times (and not actually ever started), ODOT officials repeatedly assured the press and public that it was “on schedule.” They did so by simply replacing the previous schedule with a new one, and then claiming everything was fine.

To most observers, “on schedule” means completed by a specific date that is established in advance. CRC officials regularly changed the project schedule, and as a result postponed the expected date of completion, and then used this new—and later—completion date to report that the project is “on the current schedule.” In fact, ODOT Director Matt Garrett used exactly this carefully chosen phraseology to report after the project released a report extending the completion date by 26 months (emphasis added):

Despite ongoing controversy about the scope and cost of the Columbia River Crossing, project managers insist it is still on schedule and have even reduced its estimated cost by $100 million.

“The direction last spring from Gov. Gregoire and Gov. Kitzhaber on bridge type allowed us to move forward on a solid path to meet the current schedule,” said Oregon Transportation Director Matt Garrett on Friday. (Redden, J. (2011, August 26) “Officials shave $100 million from I-5 bridge project budget” Portland Tribune. .

Unless the auditors verified that the data given to them reported each project’s original scheduled completion date, rather than a revised date that “moved the goalposts,” it cannot make any meaningful statement about ODOT’s ability to deliver projects on time.

An audit that conceals facts and emphasizes trivia

As a result of these fundamental failures to validate basic facts (the actual cost of large projects, whether projects were completed according to their originally announced schedules), McKinsey’s audit is worse than useless: it is actually a source of mis-information that effectively white-washes the agency’s multiple management failures.

An audit that fails to acknowledge and document the agencies failures cannot provide any practical advice on how to prevent their recurrence.

Instead, the McKinsey audit is obsessed with superficial indicators with no demonstrated relevance to actual project delivery, cost containment or schedule completion. It focuses on arcane statistics like “paychecks processed per payroll FTE,” “office occupancy ratio,” and “median square per employee.” It makes sweeping, meaningless and undocumented assertions, such as “ODOT’s project delivery organization is highly talented,” and “ODOT’s culture of achievement and talented team of dedicated professionals ensure that projects are completed with their intended purpose.” None of this has any utility to policy-makers or the public in determining whether ODOT is actually capable of doing its job.

It’s actually possible to undertake a meaningful audit of an agency’s competence and the relevance of its program’s and strategies to the state’s policy objectives. California commissioned just such an audit of its transportation department (CalTrans), just three years ago. What they got, for a much larger department, and for a lower price tag, was a comprehensive assessment and roadmap as to how to overhaul the agency and its programs to help California build a transportation system that would meet its future needs in an environmentally sustainable way. Oregon got essentially nothing of value: the auditor’s report even lacks recommendations, which will be developed by the state’s Department of Administrative Services, with input and guidance from ODOT.

Legislators and others who were expecting an audit to give us confidence that ODOT is worthy of a massive increase in funding and will make good use of a major increase in gas taxes should be deeply troubled. We have important needs — like keeping our highways in good repair and strengthening roads and bridges survive the coming Cascadia earthquake, but we also need assurance that our investment won’t be squandered by an agency that is obsessed with building big highway projects regardless of cost.

For more background about the crumbling credibility of ODOT, read this story about the Oregon Transportation’s Commission’s attempt to wrest more control of transportation projects and funding away from the agency. The Salem Breakfast on Bikes blog also has some good commentary about the audit and OTC/ODOT’s relationship.

— Joe Cortright, @Joe_Cortright on Twitter

BikePortland is supported by the community (that means you!). Please become a subscriber or make a donation today.

Subscribe
Notify of
guest

34 Comments
oldest
newest most voted
Inline Feedbacks
View all comments
SaferStreetsPlease
SaferStreetsPlease
7 years ago

Please contact Governor Brown to share this article and your piece of mind!

Phone: (503) 378-4582
Online: https://www.oregon.gov/gov/Pages/share-your-opinion.aspx

Stephen Keller
Stephen Keller
7 years ago

Done! Thanks for the link.

Mark smith
Mark smith
7 years ago

The Devil you know vs. the Devil you don’t. Kate isn’t stupid.

rick
rick
7 years ago

What else to expect in Oregon?

B. Carfree
B. Carfree
7 years ago
Reply to  rick

Is it really this bad throughout the state? The city I live in is, but I was hoping that Eugene’s governmental failures were the exception and not the rule. For example, our city manager is required to file an annual climate change report and he somehow manages, with no cited references or data, to convince the city council that we are doing great in spite of a continued loss of people participating in active transportation. (Transportation is the number one source of climate change emissions in the PNW thanks to hydroelectric power.)

Ronald A. Buel
Ronald A. Buel
7 years ago

Very nice piece Joe. ODOT Executive Director Matt Garrett is incompetent and so is much of his staff. You could have included the recent work on McLoughlin, which more than tripled the original estimate. The prevarication by ODOT on the CRC was incredible, going well beyond the matter of the schedule and the air pollution estimates. I am afraid that the legislature is going to try to solve the “congestion” problems in Portland by building more lanes, which will just make the congestion worse. There is a lot of pressure from the oil-highway construction-auto lobby to build more lanes, and in our Legislature, the building trades unions and AFL-CIO join them in that effort. We should expect to see more of the same going forward, unless Kate continues to step up, which she very well could do. The legislative Dems care more about the building trades union and filling their pork barrels than they do about the environment or climate change.

David Hampsten
7 years ago
Reply to  Ronald A. Buel

Mind you, the same could be said about PBOT. Here we are, celebrating the OHSU tramway, forgetting the huge cost over-runs and delays. Then there is Cully, 11 years delayed after it was fully funded. Of course, the Gibbs Street pedestrian bridge is the all-time winner, 48 years delayed, unless you count the sidewalks promised for SW and East Portland…

Hello, Kitty
7 years ago

Wait — are you expecting me to believe that one of the biggest management consulting firms around would produce a report full of meaningless statistics and useless data, possibly designed to flatter a potential customer? What next? Humans evolved from monkeys?

Stephen Keller
Stephen Keller
7 years ago
Reply to  Hello, Kitty

“Humans evolved from monkeys?”

I’m not sure, but given observations of recent humanity, you may have the order of events reversed.

Spiffy
7 years ago
Reply to  Hello, Kitty

I thought we evolved from fish…

Anne Hawley
7 years ago

Fantastic report, Joe. Discouraging, but excellent. Is this in front of the Governor yet? The (larger) media? It seems like a critical piece of information in turning the tide against this foolish and wasteful “more lanes” infrastructure plan that I keep hearing about.

J. Heard
J. Heard
7 years ago

Very interesting article.

I was intrigued by the comparison to the audit of California’s system – why did we get such an inferior product for such a higher cost?

When I looked into this, the real question, how did we end up hiring McKinsey and Company instead of having SSTI do our audit like California did? And my answer is: this audit was not designed to address ODOT’s performance.

*According to her spokesperson, Governor Brown expects this report to be: “a thorough and independent performance review of the Oregon Department of Transportation.”*

If you look at the “List Of Questions From Contract” at the back of McKinsey’s report, you will note that the only question that even addresses performance is this one:
“Is ODOT structured in a way that allows for efficient and effective operations?”

That question does not address performance directly – it just asks if the STRUCTURE ALLOWS for ODOT to perform.

Based on all the evidence that I can find, this report was not intended to address ODOT’s performance. This is a “management audit”, not a “performance audit”.

If you look at the performance metric graph shown in the article in the context of the report, its purpose is to address the ““Organization enablers” in capital program management … processes, structures, and tools that companies use to consistently deliver their capital programs”. The graph is not designed to answer “how well does ODOT perform?”, but instead to answer “how effective are ODOT’s organization enablers”?

*Rather than giving outsiders the sense that ODOT is competent, it merely shows that the agency is effective in blocking any effort to ask serious questions about its mission or performance.*

So effective, in fact, that it managed to head off the auditors from even asking the questions, as it was not included in their scope of work.

Al M
Al M
7 years ago

Another phony audit? Who woulda believed it?

curly
curly
7 years ago

ODOT Week! Can’t wait to hear more about the department entrusted with doling out transportation funds.
Thanks for your commentary. I read all the positives in the Tribune article and wondered if it was the same agency we’ve been dealing with for years! Grateful that someone is keeping an eye on the hen house……

rachel b
rachel b
7 years ago

Wow. And, ugh. Disheartening. Many thanks for the excellent reportage.

Leo
Leo
7 years ago

Let’s not forget the collusion between the Legislature, MPO’s and Regional Solutions. Their whole system is designed to create the “need” for another roads project, which the Legislature will happily oblige so that they can appear responsive to constituents needs.

wsbob
wsbob
7 years ago

Have another, different departmental audit of ODOT, from the one that has been conducted. Find the reasons for the cost overruns. It’s no surprise that big governmental agencies have cost overruns.

If it had any cost overruns…and I imagine CalTrans may well have, was the audit California conducted of its transportation dept, CalTrans, able to show why that dept had cost overruns? That is, in terms of thorough, forensic budgetary investigation, rather than simple accusations of incompetence, mismanagent, etc? Why did the costs rise, and where did the additional money go?

Allegations of incompetence can make for great stories, especially when they’re not substantiated. Our sitting president of the U.S. benefited from a situation that may have had some similarities….the NYC ice rink the city somehow never was able to complete because of project complications and cost overruns…until the great white knight rode in, to somehow seemingly magically do what the city could not do, on schedule and under budget. Truth? Or, mythology?

And with regards to ODOT’s project budgets, were the overruns legitimate? Or were personnel with that dept truly letting money get away for unjustified reasons?

Dan Kaufman
7 years ago

How about a safety audit, an equity audit, and a pollution audit?

ODOT and the Legislature is getting to spend millions to shore up for an earthquake that our grandchildren may never feel rather than focusing real funding on saving lives, reducing GHGs, and providing needed infrastructure (like sidewalks) to marginalized communities. These are the metrics they should be judged upon as much as any other.

Hello, Kitty
7 years ago
Reply to  Dan Kaufman

Rather than choose between those two important goals, there is a third way — we could pay for both.

Jonathan Maus (Publisher/Editor)
Admin

good analysis J. Heard,

The way I see it, the audit was done to give lawmakers and the public more confidence in ODOT. We must trust ODOT not just so they spend new revenue wisely — but if ODOT isn’t trusted by the public and legislators it makes it much more difficult to pass a good bill. Wait until this debate starts and lawmakers start pointing out ODOT’s over-budget projects and other problems.

those big expensive projects aren’t just a management issue, the project’s themselves indicate that ODOT leaders can’t be trusted to create a package that reflects our state’s needs. We need a completely different focus than big freeway projects that will do nothing for congestion, climate, health, efficiency, budget concerns, and so on.

wsbob
wsbob
7 years ago

Of course, there are people in Oregon that would prefer the state put less money towards road projects supporting increased use of motor vehicles for travel and transport. It’s likely that many more people in the state, do favor this prioritization of transportation budget spending.

Instead, what I think people in the state get very anxious about, are big projects planned and approved to proceed with construction, whose pre-construction for various reasons, rises dramatically, even it seems sometimes, astronomically. People want to know where the money is going. What are the reasons ODOT faces project situations in which the project budget sometimes explodes?

If ODOT has had project costs rise because for example, responsible staff in the dept didn’t get the word straight from the engineers and contractors…that might be cause to wonder about competence of one or more of the staff.

Incompetence isn’t due simply to transportation depts not applying specific amounts of budget monies to mode infrastructure a comparatively small percentage of road users would prefer over the infrastructure upon which there is the greatest demand from the public.

rick
rick
7 years ago

Metal-studded tires need to be banned.

Steve Smith
Steve Smith
7 years ago

This is a huge issue for Portland and the region. ODOT remains the largest impediment to advancing city and regional goals. Think about Barbur, Powell, Lombard, 26th, 82nd. Look at the opportunity cost associated with freeway widening that ODOT is championing on the fast track. How many hundreds of millions do they want to spend on the Rose Quarter and elsewhere? To what benefit?

Thanks Joe
Thanks Joe
7 years ago

Thanks, great job! While everyone wrings their hands about climate change, this is the reality. The state is about to push through a huge freeway pork bill that will waste billions of dollars to dramatically increase urban sprawl, road death and the state’s carbon footprint.

After the CRC and Highway 20 fiascoes, there was absolutely zero accountability. Why should anyone trust ODOT with a dime more?

curly
curly
7 years ago
Reply to  Thanks Joe

I think the key word is “Trust”.

GlowBoy
GlowBoy
7 years ago

Wow, I didn’t realize the MLK/Grand viaduct had ended up coming in 3x over budget. That project shouldn’t have even happened. They should have spent $5-10M to knock down the viaduct and revert to the surface streets that MLK and Grand are for the rest of their lengths.

I appreciate Joe Cortright’s work, and always learn something from him.

Hello, Kitty
7 years ago
Reply to  GlowBoy

I have long forgotten the details, but that option was discussed and pretty readily dismissed for some good reasons, one of which was getting across the UPRR tracks. There might have also been some issues with the fill in that area, which is largely sawdust, and they might have had to build a bridge-like structure through there even if it was solely at ground level.

JDSmith
JDSmith
7 years ago
Reply to  GlowBoy

That would have created another grade crossing with UPRR and the Max line. Not a good idea for such a high volume highway.

Ronald A. Buel
Ronald A. Buel
7 years ago
Reply to  JDSmith

In fact, the plan that we submitted, and was rejected by PDOT and ODOT, would not have required another grade crossing but still would have gotten down to grade level after the railroad tracks going south. It would have been much cheaper, but PDOT and ODOT wanted to serve the businesses who wanted to park their trucks under the ramps that were subsequently built. That’s one way you get to triple the cost — you take the more expensive choice and then mislead people about how much it will cost.

GlowBoy
GlowBoy
7 years ago
Reply to  Ronald A. Buel

Wow, THAT is some expensive parking! Way more than a million dollars per parking space, any way I can figure it.

$5 billion a year
$5 billion a year
7 years ago

The legislature is discussing a $5 billion dollar-a-year increase in taxes for more freeways. That’s about $5,000 for every household in Oregon. Here’s a video of highway pork advocates discussing the project list:
https://www.youtube.com/watch?v=bH3lB8e2hDk

Matthew in Portsmouth
Matthew in Portsmouth
7 years ago

I actually think it’s misleading to call this report an “audit”. As a Certified Public Accountant licensed by the states of New York and Oregon, I know what a financial statement audit and other types of professional accounting/auditing engagements are. Any audit/review/attestation engagement that is signed by a licensed CPA has to be conducted in accordance with generally accepted auditing standards (GAAS) issued by the American Institute of CPAs. GAAS set out rigorous standards to be applied in looking for evidence, and drawing conclusions about the matter under review. GAAS, while set by the AICPA are subject to a public due process in which anyone can participate if they choose.

Management consulting reports, of the type prepared by McKinsey are not typcially prepared according to the AICPA’s standards, and I am not personally aware of any standards that management consultants have to use in preparing their reports.

Hello, Kitty
7 years ago

There is an important standard that all management consultants have to follow in situations like this: under no circumstances are they permitted to do anything to jeopardize a future business opportunity. Also, where possible, all reports should maximize the amount of jibber-jobber management speak, designed to elevate the speaker and obscure any sort of actual information that might inadvertently be conveyed while furthering whatever point-of-view the hiring authority wants to see promoted.

Robert
Robert
7 years ago

Looking to a funding package this year, major highway projects should perhaps be held back pending warranted scrutiny of ODOT’s governance, priorities, and capacity to deliver.

But should local communities and important safety projects have to suffer from a lack of confidence in ODOT? Could a prospective funding package do one or both of the following?:

-provide design and capital construction funds only or primarily to counties and cities
-specifically give counties and cities greater authority over the process and timeline of designing and managing safety projects on urban state highways that run through their jurisdictions–without necessarily requiring transfer of ownership