Q&A: Roger Sessions on the total cost of failure

Tackling complexity key to cutting cost, waste and risk, says software expert

You've heard of total cost of ownership for IT, well now we may be approaching a new IT metric — the total cost of failure (TCF). According to US-based software expert Roger Sessions, the cost of failed IT investment is much, much higher than many would realise when total failed investments and their opportunity costs are included.


See also: $5.4 billion — the cost of IT failure in NZ

Computerworld New Zealand questioned Sessions about the methodology used in his white paper "The IT Complexity Crisis: Danger and Opportunity" and the implications of his findings:

You put the cost of IT failure in New Zealand at US$3.9 billion a year (NZ$5.4 biulion). How solid do you think this figure is?

In order to be more confident about this number, we would need a lot more research. There is interest on the part of Universities in conducting this research, but it will take time. When I discuss this with other experts in the area of IT cost, as many think I am too low as those that think I am too high. Most agree that the range of error is within 30 percent. We shouldn’t focus so much on the exact number as much as the magnitude of the number. Whether the number is as low as US$2.6 billion or as high as $5.2 billion, is not the point. The point is that a very large amount of money is being lost to the New Zealand economy through complexity related IT failure.

Keep in mind that this $3.9 billion includes not just the cost of the IT failures themselves, but also the indirect costs to the economy as a whole. Large IT failures ripple out in many directions, and the larger the failure, the further the ripples travel.

Have you ever seen anyone else try to quantify the cost of IT failure? If not why not? If so, why do you think your approach is superior?

The most often quoted numbers of IT failure are the Standish CHAOS numbers. They look at IT failure from a different perspective, namely percentage of successful projects rather percentage of successful IT budget. So, while they consider rate of failure, they don’t consider the cost of that failure.

My approach makes more sense. When you just look at percentage of failure, as does Standish, you don’t get an accurate financial picture. The reasons is that your statistics are skewed by low-end, inexpensive projects that have relatively little risk but also relatively little impact regardless of whether they succeed or fail.

Still, their numbers back up my analysis. They find that large complex projects of over $10 million in cost have less that a two percent chance of being completed successfully. Would you buy a top of the line automobile if you knew that you had less than a teo percent chance of being happy with your purchase?

What are the elements of the opportunity cost of IT failure? Are organisations aware of this or do they just try not to acknowledge it?

When a large, complex, expensive IT system fails, the last thing anybody wants to understand is the cost of that failure. At the executive level, most energy is spent showing that first of all, the failure wasn’t really that bad, and second of all, it was really somebody else’s fault. Blame avoidance, not failure avoidance, is the standard operating procedure in most large organisations.

You seem to think New Zealand is in a great position to break the mould of ICT failure, but we have our share of IT disasters. If we have all these benefits, how come we still get it wrong so often?

It’s not enough to have a winning hand. You need to know how to play it. And you must have the will to accept risk.

Politically, risk is anathema. Most politicians and corporate executives would rather listen to the same large consulting organisations and follow the same procedures as they have in the past. This in itself is not surprising. What is surprising is that they would continue do so in the face of overwhelming odds (98 percent, according to Standish) that the advice is flawed and that the processes don’t work. They say that madness is doing the same thing over and over and expecting different results. If so, the way we build large complex IT systems is nothing short of madness.

But the good news is that it doesn’t need to be this way. In the war against complexity, New Zealand holds a winning hand. Now is not the time to fold. Now is not the time to ask somebody else to play for you. Now is the time to place your bets. There is a huge amount of money in the pot.

Presumably more New Zealand projects here fall into the smaller, less complex variety.

NZ projects may be smaller than their US counterparts, but they are still well into the complexity ranges where failure is likely. I don’t have figures to do a more precise calculation. I could be off by 20 percent in my estimates. We should focus not on the exact cost of the failures, but on the magnitude of the cost. We are talking about the potential to add many millions of dollars per day to theNew Zealand economy. How many exactly? We can’t be sure. But it is a lot.

Are you serious about NZ's advantages? I sense projects are being done better here than in the past and being broken down into more discrete manageable chunks (the huge Police project failure, INCIS, taught many that lesson in the 1990s). But I thought that was pretty universal?

I am actually cautiously optimistic that New Zealand will take a leadership role in the war against complexity. A unique constellation of factors appears to be coming together in New Zealand that fuels my optimism.

First, New Zealand is big enough to have a complexity problem. I calculate that the New Zealand economy is losing over US$14 million per day to uncontrolled IT complexity. The typical New Zealand medium to large sized company could improve its cash flow by 10 percent with very little cost by addressing this problem.

Second, New Zealand is not too big. The United States has similar problems, but the sheer magnitude of the politics in the US makes the effort seem overwhelming.

Third, New Zealand has a healthy ego. You value self sufficiency and the No. 8 wire mindset. You don’t like the idea of outsourcing large IT projects overseas and especially don’t appreciate it when those projects fail.

Fourth, you have a healthy ecosystem of small to medium sized consulting organisations that will greatly benefit from an organized assault on IT complexity. What could be better than both the client and service providers benefiting while curbing rampant losses?

You can see all of these issues play our in your recent $28 million failure of the Government Shared Network project (see ComputerWorld New Zealand 14 Apr 2009).

Once the project was initially defined as a $28 million dollar project, it was already in deep trouble...

Had complexity been considered from the beginning, here is how it could have played out. The government could have done a preplanning complexity analysis of the project. The goal of this would have been to break the large complex project up into the optimal (from a complexity and cost perspective) number of smaller projects. Let’s say the optimal number of sub-projects would have turned out to be five. With five sub-projects, the overall complexity reduction would have been around 50 percent. Since complexity is directly related to cost, the overall cost would also have been reduced by 50 percent We have already slashed $14 million from the project, and we have barely begun!...

However I need to inject a word of caution. Blindly splitting up a large project into smaller sub-projects often adds more, not less complexity. This is because complexity is highly sensitive to where the splits occur. It is critical, therefore, that the splitting up not be done based on “gut-feel” but be done based on a methodical complexity analysis so that the result is the least complex possible configuration of sub-projects.

Clearly the way you logically split projects is different using SIP (Simple Iterative Partitions). Given that you've been here a bit, is it being received more readily in New Zealand than elsewhere? Any examples you can cite?

I have recently given a number of talks in New Zealand to standing room only audiences. The response has been phenomenal. And this response includes the public sector, the private sector, and educational institutions. This is particularly gratifying, because our best hope of addressing the problem will come if we have a strong coalition of all of these groups. The bottom line is that Kiwis have a low tolerance for waste, especially when that waste is only benefiting overseas companies.

We are getting good traction in NZ, but, to be honest, progress is frustratingly slow. I understand that it takes time to teach people new processes, even more so when these new processes are coupled with new ways of thinking. I should have more patience. But every day we put off solving this problem in New Zealand we lose another $10 million dollars. We could be putting that money into education, into public services, into health care. Almost anything would be better than what we are putting it into now, IT failure.

You refer to some very large programmes going on here in New Zealand – health and Auckland – are you trying to get involved? What benefits would your approach have?

I am trying to get involved in anyway possible. Complexity is a bad thing all around, and I want to be part of the solution.

There are many benefits to solving the complexity problem. The most obvious is cost. When we reduce the complexity of a large system by 50 percent, we reduce its cost by an equal amount. And the second most obvious is failure. Reducing the complexity by 50 percent reduces the failure rate by 50 percent.

There are numerous technical costs of complexity. Even when we succeed in delivering complex systems, they are inherently unsecure, unscalable, unreliable, and difficult to maintain.

Then there is the cost to society. People die in hospitals when health care IT systems fail. People lose faith in government when they see one IT failure after another.

There is the cost to local businesses. These large, complex systems are almost always farmed out to foreign companies. To add insult to injury, New Zealand taxpayers are not only paying a huge cost for massive IT failures, they sending that money overseas.

I am a small company (even by Kiwi standards!) so I am not angling to be a primary contractor on any of your projects. I am first and foremost, a teacher. I want to teach companies in New Zealand how they can do IT more effectively. I want to teach your public sector how they can involve local resources and at the same time reduce cost and failure rates. I want to teach your consulting organisations how they can effectively play in this new approach to outsourcing.

Once this has been achieved, I hope to make New Zealand a model for the rest of the world. The same problems that are costing New Zealand $14 million per day are costing the world economy US$6 trillion per year. So there are a lot of people that will be paying attention to your successes!

With that said, I look forward to being involved in this revolution. But I play the role of a catalyst. The beautiful thing about SIP is that once an organisation has been trained, they need little on-going support.

Join the newsletter!

Error: Please check your email address.

Tags management

Show Comments
[]