Request for proposal to shoot the developer

Levels of software development project failure have remained steady for 20 years. We're good at development, but we're even better at market manipulation. This is the biggest secret in the software development industry, and I'm going to get shot for daring to publish it.

You have little control over the cost of your software development projects. Development often costs twice as much and takes twice as long as you originally imagined. Why is that?

This week we’re going to examine the RFP (request for proposals) process, and see if we can learn anything from it. I’m going to try not to be too cynical with this, and you can smack the back of my hand if I mention golf courses again.

A typical RFP will contain a set of functional requirements, a set of non-functional requirements and a set of rules. We can ignore the rules and instead concentrate on the other aspects, the requirements.

The idea is simple: tell a company what you want and they’ll tell you how much it costs to provide it. Typically, though, your requirements contain multiple choices, options that you can exercise given certain events. For example, you may say that user data is stored in a database, but if the LDAP (directory) project were completed on time you would need integration with this instead. Often there are lots of these variable paths through requirements.

Now, think about the poor developer trying to respond to your request. They should estimate each path through the project, and if they’re good then they’ll provide figures for worst, likely and best-case scenarios. With only six variable parts to a specification you have 30 possible paths through the project, and with three estimates each this is a staggering 90 figures.

The usual technique is to pick the path through the requirements that leads to the lowest cost. State that this is what you’ve done and quote for the simple path only. This produces a good-looking response, with a low figure. Usually the figure is all that is remembered in following discussions, not the fact that this is for the “sweet path”.

If a proposal is accepted the scope is often signed off first, and the contract becomes “fixed-price”. By this we mean that the agreed features will be provided for the agreed cost.

What happens next is pure vaudeville. Development goes ahead until it gets to the first decision point, the first variation on requirements specified in the RFP. Let’s just assume that, for whatever reason, the client decides not to take the “sweet path”.

This constitutes a “change in requirements”, and if the developer is following a classical process, such as RUP (Rational unified process framework), this will then go through “change management”.

Change management is another name for extortion. I’m not exaggerating. The client pays for the developer’s time to sit down and consider and, if necessary, investigate the impact of the change. The cost of the change is then reported back to the client, and if acceptable the two parties amend the initial contract, adding the additional charges and changing the completion date.

So as a developer you’re happy, you’ve just won more work and made more money. As a client, though, things aren’t so rosy. You’ve just agreed to spend more than you wanted to, and agreed to get the software later than you’d originally planned.

Controlled? Only insomuch as it is possible to cancel the project and throw away all your previous investment, rather than choose to spend more money and time on a feature you really need.

This, of course, only discusses the first “change”. What happens on the sixth, when you’re three-quarters of the way though the project? The cost gets higher, sometimes too high, and sometimes you don’t get the features that you really need. Can you say “risky”?

The further through a project you get, the more expensive simple changes become. This is mostly because of software entropy; the tendency for software designs to deteriorate over time, where time is measured in changes, which is the main reason for the exponential cost of change curve.

The irony, of course, is that you had told everyone that these were possibilities way back at the RFP stage. Why couldn’t your respondents provide you with a more realistic budget expectation?

Well, to be honest, the fault lies with us, the software developers. For the last 30 years we’ve acted unprofessionally. Basically because we’ve made so much money doing things this way we don’t want to change, so we’ve never really made any effort to improve things. The level of project failure has remained more or less steady for 20 years now. We’re good at software development, but we’re even better at market manipulation. This is the biggest secret in the software development industry, and I’m going to get shot for daring to publish it.

How about this: instead of gathering all your requirements and rushing out to RFP, you first of all contact an XP team, who’ll send a couple of people in to help you gather your stories. These will be taken back to some developers, who’ll spend, maybe, a day with your representative, estimating them.

From here you can, if you want to, go to RFP. It’ll say something like this:

“We request tenders from interested parties to build a time sheet system for our mobile sales team who will interact with the system via their mobile phones. We would prefer a J2EE solution, and rough estimates put this at about six weeks’ work for a team of four programmers. Only agile teams should respond.”

This will get you responses, containing exact and comparable figures. You will get the usual company profile information to help you make your decision, but the main part, the figures, will be easy to compare. You can select reputable companies and then compare them on price.

Of course, you can still select your supplier at the golf course, but at least you will have followed a reasonable process before you did so. (Oops, sorry, too cynical, must remember to delete this last bit before it goes to press.)

Dollery is a Wellington-based IT consultant. Send letters for publication in Computerworld to Computerworld Letters.

Join the newsletter!

Error: Please check your email address.

Tags extreme programming

Show Comments
[]