Since the end of Netdosis I’ve been thinking about what to do now for a living. I’ve known for a while that I couldn’t just go back to Agile coaching – but I didn’t know why my feelings against it were so strong. So I sat down, did a lot of reading and a little bit of thinking, and presented my after-the-fact rationalization in the form of a talk given twice, once in English and once in German (video of the talk). Here’s the digest, combined with the insights I got from all the feedback.
I make my case by bringing forward three claims:
- We cannot prove that Agile practices work better than any other process
- Agile comes with adverse reactions
- Being focused on Agile as a solution distorts our view
Let’s consider the points in more detail:
1. We cannot prove the efficacy of Agile practices
During the last decade there have been many attempts within Software Engineering to copy medicine’s approach of proving that a drug or a therapy works to the benefit of patients. Clinical trials are supposed to be the backbones of Evidence-Based Medicine; in order to be reliable those trials must follow strict rules, some of which are: randomness, triple-blindness, adequacy of study groups and the usage of real endpoints.
The equivalent of clinical trials in empirical software research are called controlled experiments. There were experiments on pair programming, test-driven development and a few other techniques. You might want to read Making Software) for typical examples or The Leprechauns of Software Engineering for bad, invented and badly interpreted ones.
The problem with programming experiments is that they sometimes randomize participants, but they usually – if not always – fail to gather adequate study groups and fail to use real endpoints:
- They are typically run with a limited number of computer science students. I like to compare that to a clinical trial about the benefits of brain surgery performed by 3rd-year medical students.
- They never measure anything like the real “endpoint” of software development like “the effort required to develop a sufficiently complex feature and maintain it for a couple of years”.
Whereas those two problems could theoretically be overcome by pouring enough money (think: Millions of Euros) into a study to sponsor a few dozen teams of professional software developers who evolve the same kind of system for – say – five years, there is one flaw in programming experiments which IMHO cannot be got rid of: the lack of blindness. Can you imagine yourself developing software TDD-style without knowing you’re using TDD? When you don’t have (triple-)blindness, any result of your experiment might come from the technique under research or it might come from some unknown contextual circumstances, aka placebo effect. You will never know!
This fundamental difference to (some type of) medical therapies leads me to the conclusion that controlled experiments will NEVER be able to prove that one development practice works better than another one mostly independent from its context. But please… Do not confuse this lack of proof with proof for the opposite.
Conclusion No.1: Since we have no basis for claiming that Agile will work any better than what our coaching clients are doing now it is unhelpful and misleading to say (to a client or in public) that “Agile is better than X” – for any X. We can only judge the efficacy of our interventions on a case by case basis – this is a whole different approach to scientific action.
2. Agile comes with adverse reactions
If our current methods of Agile coaching worked as advertised, by now there’d be thousands of organizations with a severalfold increase in productivity and a self-sustaining, self-optimizing Agile process. Instead, organizations spend millions on coaching and training just to obtain a few percent improvement regardless what you measure. And many times, the negative effects on a company will not even be considered.
One of the main sources of adverse effects is the agile utopia; as Agilists we promise a lot: higher productivity, happier employees, satisfied customers etc. In next to no cases we can keep all those promises, and we’ll thereby create a bunch of disillusioned and disappointed people. Those will either stay (and henceforth be demotivated) or leave; neither is usually in the best interest of the organization at hand.
What’s more, our utopia is other people’s dystopia. Going back to open-plan offices, working with competitive colleagues on a minute by minute basis and having to report their “progress” every single morning, might look like a nightmare to many. Those will either stay (and try to sabotage the change) or leave; neither is usually in the best interest of the organization at hand.
There are many more ways to do harm by forcing the Agile way:
- Important customers might leave because they don’t get what they were once promised.
- Lower (temporary) output can lead to less turn-over and less profit.
- If you bring on change that doesn’t stick or doesn’t produce the intended improvements there’s a good chance that the company won’t be able to go back to the old status quo and will be in a more dysfunctional state than before.
- Last but most important to me: The company’s energy is spent on some non-essential process change, although it could be used for the important stuff.
One last thing about the unwanted side-effects of treating our clients with the Agile medicine: There are many valid reasons for people to not change (right now or in the way we want them to). “People over Process” should teach us to respect that.
Conclusion No.2: Before and while we’re applying Agile drugs and surgeries, weighing risks vs benefits is one of our most important duties.
3. Wearing Agile glasses distorts our view
We Agile pundits are so convinced of our theories why Agile practices are superior that we tend to develop a huge blind spot in our vision and just don’t see how people succeed with different approaches – or we dismiss their success as pure luck, or as un-sustainable or as a rare edge case. To get you started here’s a few “un-agile” practices I’ve seen work reasonably well:
- There are (lots of) successful distributed software projects.
- Sometimes code reviews work and pair programming doesn’t.
- High quality software with little or no test automation exists.
- Component teams can also deliver and feature teams can fail.
- Some software has been successfully built using a top-down design approach.
Moreover, parts of our Agile “theory” are just made-up. E.g. scrutinize the “evidence” for the “perfect” team size of 7+/-2. Afterwards, try to validate the alleged flattened cost-of-change curve with empirical data.
Conclusion No.3: There is no lack of pundits in the field of software development. Most of them are one-sided and bring their improvised theory and ready-made solutions to the table. That way they tend to promote their preferred solution without having understood the problem in the first place. Agile consultants are no different.
What I will change for myself
Bashing the state of the art is easy. Doing things better is a lot trickier. This is what I’m going to try:
- I’ll get rid of “Agile” (or “Scrum” or “Lean” or ) as a label. Together with the title I get rid of Agile’s potential nocebo effect.
Finding a new job title is tricky, though. My current one *Software therapist* might be more inventive than “Agile consultant” or “Scrum coach” but it’s not better in any objective sense. Let me know as soon as you come up with something.
- I will try to become a problem helper – instead of a solution provider. It’s essential to identify the types of problems that trigger my passion. As for me, I found out that the more management-related issues do not fit any more, so I’ll stick with clients who can profit from technical and team-related support. Working on problems that matter – to me and to the people I work with – is crucial for keeping up energy and passion.
- Being a problem helper requires that I have the tools to tackle the issues at hand. Agile methods will remain an essential part of my toolbox, but they cannot be the only ones. Getting rid of tools that don’t fit my style and storing the sharp ones in a safe place are as important activities as collecting ever more stuff.
- Putting my skin in the game…
Putting skin in the game
Professionals – like consultants, coaches, trainers or investment counselors whose main activity is to give advice – all operate under a common pattern: They benefit from the upside of their job, i.e. they get money for advising. Sometimes they even get more money when their client seems to benefit more. Their client, however, has to bear all the risk. When the client looses money, the consultant will – in the worst case – not get their pay. Have you ever seen a CEO of a stock corporation who will cover essential parts of “his/her” company’s loss?
This asymmetry in risk is especially tragic because it is mirrored by an asymmetry of information: The consultants are experts in their field – and therefore have more knowledge about the risks than their clients do. We can spot this asymmetry in all kinds of dysfunctional and immoral behavior – the financial crisis being the most prominent example.
An effective solution to this problem is to put people’s skin in the game. Here’s an example from the Code of Hammurabi: “If a builder builds a house and the house collapses and causes the death of the owner of the house, that builder shall be put to death.” This approach works in two ways:
- When their skin (or money or live) is at stake people tend to be more careful and thorough while weighing risks and benefits.
- Bad builders (and bad consultants) will become rare in the long run since they either die or go bankrupt.
The idea is ancient but it recently got fresh support from Nassim N. Taleb, e.g. in Antifragile and The Skin In The Game Heuristic. Taleb claims that it’s the moral thing to put your skin in the game even if your customer does not require it. The question remains…
HOW can I put my skin into the software game?
Be the first customer of your product. Founding a software-based start-up is one means to do that, investing serious money in your client’s is another.
Never teach or advise something that you either haven’t done yourself before or wouldn’t do if you were in your students’ shoes. For full-time trainers or teachers this can be hard.
Become an employee. Most employees have more skin in the game than the outside consultant. Being employed without invested can only be a first step, though.
Attach your compensation to your client’s long-term satisfaction. E.g. add a massive refunding clause to your contracts – and explain it to your customer.
Personally, I am at the very beginning of putting my skin into the game: I’ve removed a few technical topics from my portfolio in which I’m only well-read – but not invested. I’ve made the experience that the mere suggestion of adding a refunding clause into the contract changes the communication style. People, especially managers, do not expect others to willingly take a share of their risk.
Agile is still (one) part of my professional life but calling myself Agile is not. Putting more skin into my professional life is dear to my heart: Don’t follow my advice, look at what I do.
Stuff I read while preparing the topic and the talks:
N. Taleb: Antifragile
B. Goldacre: Bad Pharma
G. M. Weinberg: The Secrets of Consulting
L. Bossavit: The Leprechauns of Software Engineering
A. Oram, G. Wilson: Making Software
N. Taleb, C. Sandis: The Skin In The Game Heuristic for Protection Against Tail Events
Christian Weymayr: Die Homöopathie-Lüge
O. Maasen, C. Matts, C. Geary: Commitment
Richard Feynman: Cargo Cult Science