This is essentially the central argument of Jim Highsmith’s new book, Agile Software Development Ecosystems. Highsmith, who was a speaker at last year’s Software Development Conference, is one of the industry’s most respected methodologists and development consultants.
Highsmith offers case studies from companies around the world, including Trimble Navigation in Christchurch. Trimble is an example of what he calls “a homespun ASDE [agile software development ecosystem]”. He reports that it has a very agile approach to things, and that its biggest hassle is fighting off other people’s ideas of good process. Trimble has one that works and doesn’t want to change it.
Highsmith uses the word “eco-system” so as to extend the language we use to describe what we do. He is right in saying that the words “process” and “methodology” are too restrictive and carry with them too many connotations that just don’t fit the agile world.
He notes that management guru Arie De Geus wrote, in 1977, “There is accumulating evidence that corporations fail because the prevailing thinking and language of management are too narrowly based on the prevailing thinking and language of economics”, and that, “They forget that their organisations’ true nature is that of a community of humans”.
So an ecosystem is defined as a group of interdependent organisms, their environment and the constant flow of information between them. Apply this to an organisation and the organisms become people. Every individual will attempt to reach their daily goals in their own unique way.
Another of Highsmith’s favourite subjects is complex adaptive systems (CAS). These are systems which are trying to achieve a goal, and which are populated by agents that must collaborate on tasks to help the system achieve the goal. Each agent is free to approach their task in the optimal way.
Now make the system’s goal to produce software and make people the system’s agents. What have you got? A software development project. The interesting thing about CASs is that if left unconstrained, interesting patterns of collaboration and behaviour emerge. These patterns die off if they’re not adding any value to the system; if they are, they’re reinforced. This is an application of chaos theory.
Every process rule you add to that system constrains the emergent behaviour, reducing opportunities. If we maintain the view of the system as living in a vacuum, we have to add constraints. For example, a company working on a new system to calculate income tax for the IRD will have to meet certain professional and legal requirements such as auditability and accountability. These are unlikely to emerge from a system that’s purely software requirements-driven.
Fortunately, a project doesn’t occur in a vacuum; it's part of a bigger concern. This is what makes it an ecosystem. It is not only the people and their goals; it’s the environment in which it occurs. If the environment is taken into account then patterns for auditability and accountability may well emerge.
This has a profound effect on the way we manage and control projects. It means that we can no longer predict the fine details of a project.
That doesn’t mean that we can’t do project estimates for cost and schedule. We don’t change the way that’s done. A rather jaded senior developer and a rather jaded business development manager get together with the rather jaded project sponsor and try to guess the figure the sponsor thought of before entering the room. Of course, it’s different for big projects. For them it’s the same process, but the business development manager's manager joins in, and the meeting is held on the golf course (doh! I wasn’t meant to mention golf again – sorry).
It also means that we can’t predict even the details of the final software. We will get exactly the software we need and that we will spend as little as possible on getting it. The software we get will be as close a fit for its purpose as this group of people could possibly get it given the level of investment, and because of the ecosystem view that software will have maintenance costs as low as could be achieved given the level of investment.
Highsmith’s new book is about agile processes, and the views of the people behind the most popular ones. He describes extreme programming (Beck, Cunningham, Jeffries), scrum (Ken Schwaber, Sutherland, Beedle), lean development (Charlette), crystal methods (Cockburn), feature-driven development (De Luca, Coad), dynamic systems development method (Bennekum et al) and his own adaptive software development.
On the way he discusses the nature of business and social interaction. It’s a very well written and well-presented book, and is easy to read. I might not agree with everything Highsmith says but I’m glad he said it, and I’m glad of the perspective he offers. The catalogue of agile methods is very useful for anyone searching for the best way for them.
Anyone involved in software development, whether programmer, manager or business analyst, and anyone who can identify with Dilbert, should read this book (Agile Software Development Ecosystems, Jim Highsmith, Addison Wesley, ISBN: 0-201-76043-6). It’s a way out.