Not Waving But …

In a post about context switching Dmitri Zimine inadvertently shows how a strong commitment to specific Agile Practices can undermine the agile principles.

The story is how a development iteration was screwed up by the organisation’s attempts to expedite an unexpected request from Sales. The story is hypothetical and is there to make a point so the characters actions seem a little extreme.

I will summarize the post:

1. A two week iteration has just left planning and development is starting
2. Sales Guy comes up with urgent request
3. Sarah the Developer makes a vague, non-committal estimate for two hours
4. Development Manager suggests not doing it because iteration is already planned
5. Project Manager uses the “its just two hours” to push the work
6. Sarah ends up spending three days on the work - iteration is “dead”, developers are demoralized, commitment to agile practices is damaged.

Dmitri’s view of what should have happened: Sarah is left alone. Development Manager offers the Project Manager two options:

1. “No” : Put the request in for the next iteration (effectively waiting 4 weeks for feature).
2. “Drop Everything” : Cancel the whole iteration and re-plan with this on top.

What principles do I feel have been sacrificed here?

• Individuals and interactions over processes and tools
• Customer collaboration over contract negotiation

The Development Manager’s basic problem was that he did not interact and collaborate, he had negotiated a process up front and stuck to that process. He also failed to use any opportunity to gather more data to help him make a better decision.

There are a couple of injections that could have been made.

Developer gives a non-committal two hours estimate:

injection: “Lets time box this, have a look for an hour then come back with something firmer. Ask Cassandra (another experienced, but more pessimistic, developer) to look at it with you)”

outcome: We get a better estimate, hopefully nearer the correct order of magnitude, Sarah has some backup when she makes the re-estimate.

Project Manager uses “It’s just a couple of hours”

injection: We’ll lets see, have you discussed this with the sponsors? They need to know that they may loose something for this. We should have a better estimate in an hour, can you schedule the call for then?

outcome: The business stakeholders who have a better idea of the financial impact will get to make an informed decision.

With those two injections the positions move. If Sarah and Cassandra come back with “two to four days” then the stake holder’s discussion will be more focused. If all the business sponsors still agree this is really more important then a re-plan is not out of the question but everyone will know that it is a significant event.

Context is important:

• Does this Sales Guy do this to you every iteration? Is this the first time in a year?
• Is this totally out of the blue? As Development Manager its good to know what’s hot with the sales team.
• The Project Manager does not seem to have bought in to the process? Why not?

No one likes expediting, it always causes disruption and should be seen as an exception. If it is the norm you really are in trouble. It will happen from time to time, for sound reasons and your process should be able to absorb it.

Any development system is a Complex Adaptive System and one of the requirements for a successful Complex Adaptive System is that it does not violate the Law of Requisite Variety which says: “The larger the variety of actions available to a control system, the larger the variety of perturbations it is able to compensate.”

The development Manager had two reactions to the request: “No” or “Everybody drop everything! we have to re-plan the entire iteration because Sarah has to do a couple of hours on something else”.

These seem unreasonable to me.

However, after the injections we get: “The business sponsors don’t think its worth a two to four day disruption, we’ll do it in the next release” or “Everybody drop everything! we have to re-plan the entire iteration because something really important has come up that is going to take several days.”

If we were not doing Scrum we might even be able to shuffle the plan “in flight” but that’s a different discussion. If the team has a rotating firefighter role can they do it? There are nearly always options.

A lot of people ask for repeatable and efficient development processes. One of the biggest arguments against many of the agile practices is that they introduce inefficiencies:

“Refactor the codebase? But it is more efficient to just do it right the first time.

“Write tests while I code? But its more efficient to write them afterwards when I know what to test”

As for Repeatable, that is the Holy Grail of process to many managers. If they can have a repeatable process then they can just write the plan, kick off the project, and watch the software being delivered.

The problem is that in the quest for efficiency and repeatability we can loose the more valuable qualities of effectiveness and reliability.

No amount of efficiency will help if you are developing the wrong thing and, brutally, the only repeatable process most software shops have mastered is when to give up on the specified process and get on with shipping something.

When you are thinking about your process forget “repeatable” and ask yourself: “Will this reliably deliver what we want?”

When think about some practice ease up on how “efficient” you will be and ask yourself: “By doing this will we be more or less effective?”

This is a (partial) list of operational tools and habits that I have found extremely useful in the past:

1. One Click deployment: Each component of the system must be deployable in a fully automated fashion. There must be NO manual steps, a checklist is not good enough. If you can transparently audit it all the better.
2. One Click rollback: Your rollback in case of a failure must be fully automated. Again there must be NO manual steps.
1. The one-click should not require explicit access to the deployment servers.
2. If multiple components are to be deployed then this should be done by a script.
3. NEVER give developers write access to the production servers, an accident WILL happen if you do. Read access is fine and as you have one-click, audited, remote deployment they don’t really need it do they?
4. Everything builds from a tagged source in a source management system or is a third party library of a known version (This may seem obvious but is often not the case).
5. The separate components in a distributed system must be loosely coupled enough that they can be deployed independently. This is not always possible but it is definitely worth the effort. You want to avoid having to rollback multiple systems supported by different groups because of a single failure.
6. If you have to deploy a non-backwardly compatible change to the communication protocol (it happens) then do a protocol release on its own.

The one that generally causes the most fuss is 4; but if your system requires regular access by developers you have a problem.

I have been following JP’s musings on Project Management and Communication focusing on estimation and reporting.

For a successful technology, reality must take precedence over public relations, for nature cannot be fooled. : Richard Feynman

In my last entry I discussed the social reasons why I felt that a lot of projects miss their deadlines. Here is the setup for the scenario outlined there.

Richard Glass in Facts and Fallacies of Softwaring Engineering notes that one of the two most common causes of runaway projects is poor estimation (The other is unstable requirements)

• Estimation is usually done at the wrong time (too early)
• Estimation is usually done by the wrong people (management, architects, senior developers)

To which I would add:

• The delivery rate of any one team/project combination is a unique value that is very different to any other combination.
• No one is any good at estimating large software tasks, especially if someone else is actually going to do the work.
• No one should be estimating a software task if they won’t be involved in carrying it out.
• No one is a born estimator. It takes practice.

Many developers/development managers do not realise that “The Map is not the Territory” and , finally, few development managers really understand the above

So, when the project starts to slip, and the estimates are not being met many development managers feel that someone is doing something wrong now. They don’t realise that an error has already occurred and the results of that error have been captured into a number of documents, Gant charts and public commitments.

Tom DeMarco points out that managers often think they have the power to make people think faster by putting on the pressure. So, given that they have committed to a timetable they feel the best way to get back on track by applying pressure - to make doubly sure they make everyone think longer too by increasing the hours.

Finally, to save embarrassment, they finesse the data until they recover the lost ground…

There are better ways.