Veterans of TDD: Lasse Koskela

May 26, 2014

I “recorded” episode 6 using email ping pong with Lasse. Since I knew that his company is involved in funding start-ups, the TDD vs learning-what-product-to-built angle was especially interesting to me.

Lasse Koskela has been practicing test-driven development for over a decade. He has written two books on the topic and introduced many teams to the technique. Lasse works as “Principal Generalist” for Reaktor. You can reach him through

Q: When was your first contact with TDD and what did you think about it at the time?

Lasse: It’s hard to say exactly when the first contact was. I have a goldfish-like memory but I do remember reading up on Extreme Programming around 2001 when I was switching jobs and soon started promoting agile methods within the company so I guess it was around 2001 when I first learned about TDD. It would take a couple of years for me to become really infected, though.

Q: What did eventually convince you that TDD is a worthwhile approach?

Lasse: There’s no one specific thing or moment when that happened but I’ll tell you a story that kind of describes how I got to the point of being convinced: In the early 2000’s I was working on projects at big companies – Fortune 500 type of places – meaning there were usually a lot of people involved, responsibilities were spread around, half of the project staff seemed to be managers, and there was hardly any test automation to speak of, not to mention manual environment setup, deployment, and things like that. Very often, deploying a new version of a web application to a testing environment would go through a third party infrastructure team and a separate testing team might take a couple of days before they would actually get to test what you had built.

In that place I couldn’t afford to deliver code that didn’t work – the slow feedback loop would have cost my sanity – so I started writing unit tests more thoroughly, refactoring code more, and looking at integration and system-level test automation. On one project we even got a non-technical business representative write executable acceptance tests for us in an Excel spreadsheet, which we would then feed into our homegrown test automation framework.

At some point, being in this mental mode of wanting to write a lot of unit tests, seeing a clear value to creating them and repeatedly executing them, it wasn’t much of a jump to give TDD a try. I had read about the method before and “understood” how it was supposed to work and how it would purportedly help me. So I started doing it. And kept trying. I vaguely remember it taking maybe 6 months before TDD had become second nature but all that time I “knew” that it worked, despite me forgetting and slipping every now and then. I felt like programming test-first made perfect sense.

I credit much of that transition being so unnoticeable to the fact that I was already intimately familiar with unit testing and the kind of designs that make unit testing easy or difficult. I was already bought into the idea that code should be modular and testable. TDD simply made it easier to end up with such designs. At times it felt like magic even though it’s really a very simple dynamic.

Q: What has changed in the way you practice and teach TDD since the early days?

Lasse: Well, for one thing I don’t teach TDD as much as I used to. Nowadays I seem to be doing a small handful of trainings a year whereas in 2005-2006 I remember doing a class every other week.

What’s changed in my practice? Clearly the biggest thing is that I find myself doing less test-first programming than before. That’s not because I wouldn’t find TDD useful but rather because certain system conditions required for test-first programming aren’t in place.

Q: This is closely related to my next question: Are there situations in which you consider TDD not to be the right approach for developing software? If so, what other techniques and approaches would you recommend in those situations?

Lasse: One specific situation, which is why I’ve recently done less test-first programming, is one where you simply don’t know what you want the code to do. For instance, if it’s the first time our team is doing video recording on an Android device I have no idea what API’s the code should call, in which order, what constitutes as valid input, etc. I can read all the documentation I want but really the best approach I know of in such a situation is to go ahead and hack something together that seems to work. You might call it a spike, a prototype or exploration but it wasn’t done test-first. And, me working with all these new technologies and API’s, I find myself doing that kind of exploration much more often than back in the day when I stayed firmly within my technical expertise, Java-based web apps and backend systems.

Q: What do you think is TDD’s relevance in today’s world of lean startups, functional and concurrent programming, continuous delivery and mobile devices everywhere?

Lasse: I’ve been pondering that every now and then, especially regarding the startup context. Our company funds early stage startups through our Reaktor Polte arm and I get to advise and consult these startups every now and then. What it boils down to is that whether the code is written test-first or test-last generally isn’t even near my major concern. I guess it’s a bit like agile development in that the first thing to do is to make sure we’re doing the right thing. Once you start feeling comfortable that this is the case, that’s when it starts to make much more sense to invest your energy in doing things right.

Some people say that you don’t need to do TDD if the software you’re working on doesn’t have to work. In a way, for a startup that’s kind of the case. The code doesn’t have to be bulletproof from day one. It doesn’t have to be sustainable. On the other hand, when you’ve found an angle that does generate revenue it becomes increasingly important that the technology is solid and maintainable.

Many thanks, Lasse, for answering my questions!


Other episodes of the series:

Veterans of TDD: Willem van den Ende

May 22, 2014

The interview of episode 5 is with Willem van den Ende. He’s going into quite some detail about what function languages bring to the TDD table and where you might handle development in those languages differently.


Willem is a long-time practitioner of all things Agile. He is one of the founding fathers of XP Days Benelux and still very much into training, coaching and practising software development, which he has done in a dozen languages (or more). You can learn more about Willem on and by following him on Twitter.

Q: When was your first contact with TDD and what did you think about it at the time?

Willem: I started reading the C2 wiki, at that time known as the portland patterns repository, shortly after it got started in 1996 or 1997. Various people were writing about what would later become Extreme Programming. I vaguely remember reading the pages about test-first-programming and thinking ‘that looks interesting, but I don’t quite get how it works’.

Refactoring and iterative development resonated more with me at the time. I had just removed some defects from a fresh legacy C++ GUI by removing 90% of the code while keeping the functionality. Other people on that project were writing end-to-end tests, and we would run them every night on all workstations in the company. That had great value for finding defects, but written after the fact, with little communication in the development team, had negligible impact on the design of the rest of the system. ‘My’ GUI fell outside of that, so I ended up doing piecemeal refactorings in the Solaris C++ debugger. It allowed modifications in the implementation with reloading, as long as you did not change any interfaces.

Q: What did eventually convince you that TDD is a worthwhile approach?

Willem: This was several years later, in 1999 or 2000, after Kent Becks’ eXtreme Programming book finally came out. Me and a colleague, Erik Groeneveld, had a bit of time on our hands, so we sat together and did a small exercise. It only took an hour and a half. We were going: ‘that is very interesting, I have no idea how it works, but the design we ended up with was completely different from what we had planned, and much better’. I’ve used that sitting together for an hour and a half a lot in coaching and teaching. It does not take long to get a feel for how TDD works, and get hooked to try it in more situations. After that it is difficult to go back and do what you did before (most of the time).

Q: What has changed in the way you practice and teach TDD since the early days?

Willem: I delete tests more often after I’m done building up a design, keeping only the more interesting ones. I worry much less about integration and acceptance tests. I prefer to put software in the hands of the users. I’ve been teaching ATDD and BDD from the beginning. In projects I find it much more important to go for that elusive Metaphor, and find a common language that all stakeholders can understand. Whether you automate some of that understanding and how, is less important.

When you do ATDD or automate BDD tests, you often end up with a couple of level of indirection. Maintaining those indirections is not always worth it. I was hugely impressed when Ward Cunningham showed the swimlanes visualization he made when he was working for the Eclipse foundation. He was making a workflow system, and the acceptance tests were a visualization of workflow examples. My takeaway from that is, if you can find the metaphor, and the coverage that automated customer-facing end-to-end tests can have has value in your situation, make the effort and tailor a solution. So for my most recent project that involved writing customer facing end-to-end tests for a simulator, we tried to specify end-to-end tests in BDD’s given-when-then format, but none of the specs that came out were interesting enough from the customers perspective, or thorough enough from the developers perspective. The developers had tried FitNesse before that, and it just did not provide enough details for the clients. The clients could all program, or read programs up to a point (PhD’s in physics, econometrics, that kind of thing). The customers wanted to know the calculations that went in to the acceptance tests. We ended up writing acceptance tests in Clojure, with a small support library to keep the tests clean and readable.

It was great to sit with the customers and go through the tests we wrote, to check that we were all understanding the same thing, and the simulator was doing what they expected it to be doing. Don’t assume your users or clients are dumb, make an effort to find the most expressive way to communicate. And be very selective in the number of automated end-to-end tests you have. For any moderately interesting system, maintaining them while the system changes can be a huge burden. Don’t forget that you want to be able to keep evolving. And if you can’t find an effective and efficient way to do end-to-end tests, leave it for a while, focus on shipping your software and get feedback from real users. Try end-to-end tests later and see if you can come up with something better. It often takes a few iterations.

In teaching, I spend much more time letting participants figure things out for themselves. We’ve started explaining TDD with Mock objects through finding responsibilities in your software, and hexagonal architecture as a metaphor for overall system design – as opposed to using Mocks to break nasty dependencies.

I’m considering dropping ATDD and BDD from my trainings. I’ve found value in unit-level TDD in every single project I’ve done since learning it. ATDD and BDD much less so, it can easily lead to bureaucracy and test maintenance with little value. So I prefer to make sure we ship first, get feedback from actual use. Show, don’t tell. If you think of valuing “working software over comprehensive documentation”, is ATDD or BDD in our project the former, or the latter? With frequent and early deliveries I don’t have to ask that question. That is not to say that there is no value in ATDD or BDD, I just don’t think there is a single way of doing it that is valuable in a large enough number of projects to put in to open enrollment trainings anymore.

Q: Are there situations in which you consider TDD not to be the right approach for developing software?

Willem: When I write a two line bash script. Or when I can use the ‘tests’ as the program itself. I’ve been exploring logic programming in the forms of Answer Set Programming. It does not work well for all problems, but when it works, your ‘tests’ are the program.

I have written somewhat larger programs without tests on purpose. It sort of works for a while, until my programs grow bigger. Adding tests then, and refactoring, is not so much fun. I

I also worked on one project where the people and the framework we were using was not very amenable to TDD. In that case I prioritized more frequent delivery, and automating the many manual steps that were needed to make a delivery. I tried TDD first, though, because it is one of the best value for effort practices I know, and you don’t need much to get started.

Q: What do you think is TDD’s relevance in today’s world of lean startups, functional and concurrent programming, continuous delivery and mobile everywhere?

Willem: I find that I go faster with TDD than without, and so do most of the project I’ve coached – in languages like Java, PHP, C#, Ruby – within a couple of weeks. So I think if you believe, as some people seem to do, that lean startup and TDD are opposites, you may have missed a few tricks.

Most of the time I find TDD cheaper to set up than continuous delivery. So I usually start with TDD and go from there. Having said that, I’m trying to set myself up with a cheap and simple continuous delivery solution, so that I can quickly ship and, equally important, roll back. It’s a yes-and situation. Without shipping to users, nothing much matters. But without TDD or a different mechanism to get early feedback on our design and code quality, you cannot continue for long.

Functional programming is a big space, a complex system in and of itself. There are some aspects that can help reduce the number of not-so-interesting tests. You might come across statements like these: “I don’t need TDD because I do everything in the REPL”. I guess people who make this statement have never used a Smalltalk environment in anger, or worked on a LISP machine. Both have significantly more interactive power than, say, Clojure’s REPL. I just completed a project in Clojure, and yes, the REPL is useful. I have used it a lot. And still I wrote tests first (and sometimes after since I did try REPL driven development). Boy was I  glad I had some integration tests when I needed to port the software to Windows near the end of the project. Those were tests I initially had written to drive what API I wanted on the application side, and to find the APIs I needed on the OS side. I was also glad to write tests to figure out how to design some of the more difficult parts of the application.

In Clojures’ case, the JVM just is not that good on reloading code, and since your editor lives outside the REPL, it never knows when you’ve renamed a method; that’s why code that you expect to break, because it calls the function by its old name, stil runs and vice-versa. You get used to it, so you just reload very often, but it is a very different experience from a Smalltalk environment where you keep working in the same live image for days or weeks at a time, and you can modify all the tools (like the debugger) to your heart’s and workflow’s content. In Haskell I’m not that fluent, I use the REPL mostly for finding out what types my code has, and compiling my code as I go, sending it to the REPL.

Even when you have something as highly interactive as a Smalltalk environment, there are limits to growth. I recently worked on a product in Smalltalk. It had only a few integration like tests, some of which were read all the time, so you could not say it was in a known space. The product had problems growing beyond the initial number of customers for several reasons, some of them non-technical. One of the main technical reasons I could see, was that most of its development was done with Smalltalk’s super REPL, the debugger. It’s great, very dynamic, you can change everything on the fly while debugging, adding methods as you go. In this case that also meant that the logic was very hard to follow. Adding a new programmer on the existing code meant long pair-debugging sessions, and the code was much more complicated than it had to be. We added a new component, dependent on the rest through a clearly defined messaging interface, that was much easier to reason about.

Remember where TDD came from? Oh, right. Smalltalk. That is what Kent, Ward, Ron and Chet programmed in. Dynamic environments are great for TDD, because you can get fast feedback. At the same time you need it, because a year from now you will have forgotten your REPL or debugger session, but your tests have remembered. So I call bullshit on the REPL argument. Having said that, there are projects like Gorilla-REPL moving the Clojure REPL in a more interactive workbook-like direction. But having something that was not build for interactive use like the JVM underneath means that it will take a while. There are people working on making reloading faster and more predictable, but it is going slowly.

There are a lot of potential benefits, and activity, in the functional programming space. Some mean you have to write fewer tests (static or gradual typing, clearly delineated spaces where mutable state and IO go), others mean you can write more powerful tests (Haskells’ QuickCheck).

There is another argument I often hear: “I Don’t need TDD because I have static typing”. Don’t be fooled by Java’s, and to a lesser extent, C#’s half baked type systems. There are type systems out there that will, similiar to TDD, help you drive your design. Having said that, even in Haskell with its advanced type system I still write some tests to drive out interesting behaviour and interfaces. Test-after works slightly better than in say, Java, because of the advanced type system, and that most of the program are simple functions. Nevertheless, I still find working test-first useful to figure out more complicated parts of the program.

When embarking on something new, try to understand how you are working, where your feedback is coming from. Then understand where the ‘new’ thing is coming from. While trying the new thing out, keep reflecting and check your understanding against your experiences. I did a project in Clojure, because it was a good fit for the people, the project and the problem, and I was curious. Now I’m trying Haskell, because I wanted to try a dynamic and more static functional language to get a feel for the differences.

Q: Willem, you’ve tried out TDD in lots of different environments and on different platforms. Do you see other tools, e.g. in functional languages, that could have similar benefits like TDD with less trouble?

Willem: I’ve just started a new project using Yesod, a somewhat Rails like framework in Haskell. Not that I like big frameworks per se, but I don’t have enough understanding of Haskell and its ecosystem to collect my own stack out of libraries. Michael Snoyman, Yesod’s maintainer seems to have good taste in selecting libraries and delineating dependencies. One of the reasons for choosing Yesod, is that it makes good use of Haskell’s type system to prevent mistakes like creating a route named “/hello” in your application, and then making a typo when you create a link in html, e.g “<a href=”helo”>”. In most frameworks that would require end-to-end tests or manual tests to turn up, here it is a compile error. Also very interesting is using the type system to make sure that two sides of a remote procedure call are using the same data. Yesod together with Fay (a subset of haskell that compiles to relatively compact javascript) looks promising, as well as research in haste. Again, fewer slow and brittle end-to-end tests to write, leaving more time to focus on interesting aspects of your problem domain.

Both Clojure and Haskell force you to be explicit when you want mutable state. Constants are the norm, variables the exception. This makes parts of the program easy to test, a test provides the input, and you just check the output. In addition, Haskell forces you to clearly mark the places where you use IO. These two combined were mind-bending for me. Not that I normally have IO everywhere, but this forces you to really think hard about the design of an application. After doing the Clojure project I’m getting more comfortable with it; although I cheated with the use of IO here and there, I only used one bit of mutable state close to the end of the project. The problem lent itself well for that approach, so that helped.

An interesting development I found is that some aspects of functional programming are trickling down to the curly bracket languages, e.g. lambdas in Java, but more interestingly, gradual typing for PHP with Facebook’s Hack language. I have not tried it, and I expect it will help more on the defect prevention side than the drive-your-design side, but at the very least it will improve largely untested PHP code, and allow PHP programmers to focus their TDD efforts on more interesting parts of the application.

I would not say all this comes with less trouble. Like TDD, learning languages like Clojure and Haskell and trying to understand their idioms and culture takes time. Your performance will probably go down more or less, before it goes up. I think I’ve found a good way to explain the use of Clojure to other people, I hope to find the same for Haskell, so I can use one of the two with more people around me.

On the research front, I’m keeping an eye on dependently typed languages like Idris, where you can specify even more constraints than in Haskell. Where in TDD we work with examples, and hope we have enough examples to cover ourselves, you go more in the direction of providing complete proofs.

Underlying the application of TDD are a few things for me. XP’s question: ‘What has changed in technology over the last ten years that allows us to do things differently?’ and TDD’s ‘Test everything that could possibly break’. New technology allows us to reduce the number of things that could possibly break. But that number is not zero, and it’s unlikely that it will ever be. Instead we will probably use that reduced number to build more complicated, more interesting systems, so we will still benefit from TDD. The other question is: “Where do you get feedback from?”. Never forgot to combine TDD and other technical practices with a healthy interest for those that use your software, and how you can make their lives better.

Many thanks, Willem, for answering my questions!


Other episodes of the series:

Veterans of TDD: J B Rainsberger

May 16, 2014

Episode 4 is the longest of all episode so far and it is based on an hour-long interview I did with J B Rainsberger a few weeks ago. The interview was very intense and we got into a lot of details. Nevertheless, I had to cut the hour down by at least half. So if something is not clear, it’s my fault as an editor.


J. B. Rainsberger helps software companies better satisfy their customers and the businesses they support. He has learned to write valuable software, overcome social deficiencies, and built a life he enjoys. He has traveled the world helping people get what they want out of work and out of their lives. Recently he has launched to help even more people start getting the advice they need with minimal investment. He lives in Atlantic Canada with his wife, Sarah, and three cats.

You can contact Joe through

Q: When was your first contact with TDD and what did you think of it at the time?

JBR: It was either in 1999 or 2000. I was working as a programmer at IBM, about three years into my professional career, and I reached that point where I really felt uncomfortable with the quality of my work. I would come into the office, I would find out that I created a bunch of bugs; I would spend the day trying to fix them. If I was lucky I’d even get to spend some part of the day trying to add some new behavior. I would go home believing that I did good work and when I came back the next day there’d be more bug reports. And this cycle just kept continuing. Eventually I reached the point where I felt like I just wasn’t making any progress at all, and it started to really make me feel quite anxious. I would go in to work starting to think ‘Not only am I not making this better, but I don’t know when I’m going to finish’.

So finally one day I just said ‘No, I have to do something else’, I said the magic sentence for any professional programmer: ‘There’s got to be a better way’. So I opened up a search engine and typed in ‘How to test Java code’. I found and I also found an early draft of the book ‘Extreme Programming Installed’. So I downloaded JUnit and started playing around with it. I read this book that had some examples of how to do test-first programming in this very strange language, Smalltalk, which I didn’t understand. And with these two things together I started practicing test-first programming – as we knew it at the time. I just started with this idea of: If I write the test then I’ll know exactly when I’ve made a mistake and I can fix it as soon as I have made it.

What I always tell people is that Extreme Programming will divide the world into two groups: the group of people that immediately thinks ‘This is crazy and will never work’, and the group of people that immediately says ‘This is the only way it could possibly work, and how stupid are we for not doing it this way all along’. And I was in the second group.

Q: So nobody had to convince you that this was a worthwhile approach?

JBR: I believed it straightaway. But I wasn’t really convinced by it until I had my first industrial-strength experience, and this happened just soon after. I had gradually moved into a team of one; I was working on my own component, which was fairly independent from the rest of the system. That gave me a lot of autonomy and it gave me the opportunity to try any technique I wanted to make my work better. And then the release deadline was coming up. I remember I was in a project-status meeting, Friday afternoon, about three weeks from the code-freeze date and I had been going through this cycle for a few weeks: fixing a couple of bugs while creating two new ones. I said to my manager in this meeting: ‘Bad news, I’m not going to make this deadline; the ship is leaking faster than I can plug the leaks. The good news is I have just been playing around a couple of months with this idea of test-first programming where I write the test to make it clear what I’m trying to do, and then I write the code to make that test pass. So here is my proposal: Let me work on this at home where I have no distractions. I’ll start again, I’ll build the whole thing again, and if I can’t finish it in three weeks, you can fire me.’ I was 24-25 years old, so I could afford to be this reckless with my career. So he said: ‘Fine, go home, go as fast as you can, come back when it’s done.’

And that’s what I did. I went home; I would spend every day following the steps of test-first programming. I rebuilt my little component from scratch. I worked until I couldn’t stay awake any longer, went to sleep, woke up the next day and kept working. I ended up finishing all the work in nine 14-hour-days and towards the end I was: write one line of code, run the tests, which would take 10-12 minutes to run. Because I was exhausted, I would take a nap while the tests were running. I’d wake up when the tests had finished and then I would take the next step. Towards the end I was doing only three or four code changes every hour, but I managed to redo three months of work in nine days. Now I know that some of that was the second-system-syndrome, but that was what gave me a feeling of the so-called ratchet effect: the idea that you take a step forward and then the ratchet locks in place and now you can’t fall very far.

What is funny to me now: Back then I didn’t know anything of how to use this as a design technique. I was using this purely as a technique for avoiding bugs, for discovering mistakes as soon as I made them. But that experience of rewriting three months of work in nine days really crystallized for me the potential of this technique. I wasn’t worried about going faster, I wasn’t worried about good design, I was only worried about making steady progress and eliminating this feeling of never going to be finished, of always making it worse instead of better. This episode made it crystal clear for me that test-first programming and I were very compatible, that it matched the way I worked, the way I thought, and it helped solve real problems I had. I was hooked.

Q: What has changed in the way you practice and teach TDD since those early days?

JBR: Early on my emphasis was on removing or avoiding defects, on discovering mistakes quickly. The big change happened when I began to see how I changed the way I thought about design. The first obvious change was that I felt more comfortable making fewer design decisions upfront, that I could let design emerge and that that was okay. So for a long time I was thinking ‘Oh, this is a design technique’. And then a few years later I really started to understand that the act of practicing TDD became a way, not just to design software, but to think about design. TDD became a technique for learning about the principles of design. When I teach TDD, I talk about this progression: It goes from a testing technique to a design technique to a learning technique.

It’s like chess. You can learn the rules in five minutes, but the strategies of chess take a lifetime to master. The bad news is: You are never going to be a perfect designer. The good news is: There is always something more to learn about design and here’s the way to learn it.

Q: Are there situations in which you consider TDD not to be the right approach for developing software?

JBR: Absolutely. If I’m going to throw it away in five hours, then TDD is not all that valuable. If I’m working in a situation where I know I will throw it away soon, if I really understand the technology, if I really master the libraries, then I like not to bother. Now, as a long-time practitioner of TDD, it feels like I’m doing TDD really quickly in my head, even when I don’t do it in tests. So I certainly don’t do TDD with the same discipline in those situations as I do it in other situations. If I don’t need to maintain software over a long period of time, I can feel more comfortable about writing fewer tests. If the cost of it failing is very low, then I don’t worry too much about whether I write tests for it. That’s particularly true in cases where there’s something small, and I know that if I get it wrong I can just rewrite it from scratch. I feel pretty comfortable doing something a second time. I don’t like doing it a third, fourth and fifth time, but I don’t mind doing it once, hacking it together, and then – if I see it’s going to take on a longer life-span – throwing it away and rebuilding it with more discipline.

Ron Jeffries has his old saying: ‘Whenever I don’t do things in a disciplined way, sooner or later that causes problems for me, and I wish I had done it in a disciplined way.’ For a long time I used to take that to mean: You should always do everything in a disciplined way. And I think everyone goes through that phase and I think that’s okay.

Financial people use the term ‘overpaying for a guarantee’. When you apply too much discipline too often you’re overpaying for a guarantee. People have to go through this period in order to really understand when they are being wasteful. It’s one thing to say ‘You don’t always have to be perfectly disciplined’. That doesn’t mean that you understand when to be perfectly disciplined and when not to be. I only know some very obvious places where there’s too much discipline, like when I’m going to throw it away in five hours or two days. But as soon as somebody else wants to use my software, I want to write tests. As long as I feel comfortable redoing some part of it again or adding tests later, it increases the number of times where I feel comfortable lowering my discipline a little bit.

That’s what technical debt is supposed to be all about. Technical debt is not supposed to be all about sloppy code, it’s about making that conscious informed decision. I’m willing to let my discipline suffer now knowing that I might have to add some extra discipline back later. That’s good debt. That’s the kind of debt where you borrow a little bit of money, as long as you know that this will help you obtain more profit later. I think that a lot of people overestimate how good they are at making that trade-off. So when people ask me about “When should I not do TDD?” I say: “As long as you’re prepared for the consequences of not working with discipline, you can afford to work with less discipline.”

Q: I’ve seen this work for individuals, maybe for small teams, but for a company I’m always skeptical if they will ever recover from their technical debt.

JBR: Enterprises tend to not make the kind of conscious good debt decisions. They tend instead to panic because they have low cash flow; then they do all the typical afraid-project-manager things; they make all the typical project manager mistakes that come out of fear. I’ve never seen a larger organization, dozen or hundreds of people, which has institutionalized a sensible approach to technical debt. I’ve seen individual project managers who understand “The Mythical Man-Month” and “Peopleware” and books like that, who succeed in insulating one or two teams from the insanity of the rest of the enterprise. But I’ve never seen that successfully institutionalized. It usually only takes one or two people to panic, and then the rest of the enterprise panics too.

Q: What do you think is TDD’s relevance in today’s world of Lean Startups, functional and concurrent programming and mobile everywhere?

JBR: There are a few things. I worry about TDD’s relevance. I have to make the distinction here between the mechanics of TDD and what I believe is the intent of TDD. The mechanics are following the rules, writing tests first, and so on, while the intent is providing a way for people to understand more deeply how to design well over time.

Let’s go into some of those things: The little experience I have with Mobile, so far has shown me that mobile development seems to rely a lot more on frameworks that don’t have competitors. In Android and iOS and so on, there is a small number of very widely adopted frameworks that weren’t designed with TDD in mind and the mechanical part of TDD is difficult to do in those environments. You seem to spend a lot of time side-stepping the frameworks. It’s just the same as in the early days of Java web development when there was just Struts and people spent a lot of their time side-stepping Struts. And then Spring Web MVC came into the picture, which was for me the first Java web framework that made TDD easier because of its more decoupled, more interface-heavy design style. I haven’t seen the Android version of Spring Web MVC yet. It’s possible that there will never be one. So it worries me a little that we’re going to have a generation of programmers who are used to say: ‘It’s just a 99 cent game’ or ‘The failure modes in this app are so unimportant that we don’t have to get it very right, we just have to get it somewhat right’.

Lean Startup encourages people to move into this direction a little bit, and that makes me uncomfortable. There’s going to be a bunch of Lean Startups who – once they get going and realize how many corners they’ve cut in order to get early earned revenue – will realize that their discipline deficit is costing them money and that they need more disciplined techniques. That’s one reason, I think, Lean Startup folks need to understand the discipline of TDD – or something like it. Early on going fast enabled them to bootstrap and to do a better job of getting going with less cash. But eventually cash flow is no longer the bottleneck, the bottleneck will be profit and they are going to wish they had a more disciplined approach. I assume that the Lean Startups that are still doing well in 2016, 2017 and 2018 are the ones that have some kind of disciplined approach that they deploy at the appropriate moment. It doesn’t have to be TDD; it might be something like it.

Functional is the one area where I really don’t know because I haven’t done a lot of functional stuff. I like to say I’m an object-oriented guy who has a functional style, but I don’t understand functional design deeply enough to say that with confidence. The little bit of functional design I’ve seen and understood, I got from a friend who taught me Haskell for three solid days. I had this vague feeling that the libraries are so good that it doesn’t feel like there’s anything to get wrong, so it doesn’t feel like there’s anything to test. You just figure out how the pieces go together and it just works. The kind of isolated object testing or isolated function testing that I talk about in “Integrated Tests are a Scam” doesn’t apply very much in the functional world. What I think will matter more is what TDD has taught me: the design principles; the discipline in judgment; context-independence and modularity; the mindful approach to programming; the idea that TDD encourages me to stop every few minutes and really think about what I’m doing, rather than trying to figure out everything upfront. Maybe I’ll do TDD in my head instead of writing the tests out.
When working in functional languages you need to be even more disciplined, and the question is: How did you learn that discipline? TDD happens to be the way I developed it, some other people have some other techniques, and in the 2020s there might be a whole generation of programmers who gained that discipline without ever using TDD. When that starts to come out, I’ll be really interested to learn what that is. And maybe I’ll have to unlearn a little bit of this TDD stuff.

Q: You already mentioned your talk “Integrated Tests are a Scam”. As I understand the talk, you recommend to replace most or all integration tests by collaboration and contract tests. Is that a fair description?

JBR: Yes, the only thing I would change is the word ‘integration’ to ‘integrated’. Steve Freeman and Nat Pryce [in "Growing Object-Oriented Software Guided by Tests"] talk about integration tests. They  specifically mean the tests that show that my stuff integrates with their stuff correctly, which is what I think of as collaboration and contract tests. ‘Integrated’ tests are those tests where we put clusters of objects together, and when a test fails we don’t know which object is to blame. So let me refine your statement a little bit: I don’t like to use integrated tests to find mistakes that, if I had just paid a little bit more attention, I wouldn’t have made. I use the term ‘basic correctness’, which means: If we had perfect technology with infinite resources and infinite patience, would we compute the correct answer. I don’t like to find mistakes in basic correctness with big tests. I prefer to find them with collaboration and contract tests; and then I get all kinds of really good design feedback from those small tests. However, there are a bunch of things that integrated tests are better at finding: They are better at finding emergent behavior that we didn’t intend. They are better at illustrating when the system does something that it shouldn’t do.

The scam part is a different question all together.  The scam part is where I think big tests encourage us to design poorly; and designing poorly makes it harder to write small tests, so we write more big tests, which encourage us to design more poorly; and that cycle continues and that’s the scam. I think that integrated tests actually make the problem worse that we think they solve. It’s like buying Aspirin that gives you a bigger headache.

Q: I see the theoretical beauty of your basic-correctness-approach. What I find missing is that – in order to make this approach work – people have to track their contracts in a more formal way to see that.

JBR: Yes, I’ve heard this a bunch of times. When somebody says this to me I reply: ‘What you’re saying means, that you have to really understand how to integrate with other people’s stuff. Well, of course you do!’ Imagine for a moment that you could perfectly understand every aspect of my API that you intend to use. Imagine that by some magic you could do that without writing everything down. Then you don’t need to write everything down. But for most people, most of the time, if they don’t write that stuff down, they don’t think about it precisely enough to get it right. I’m just offering one way to do it: contract and collaboration tests.

Many thanks, Joe, for answering my questions!

Other episodes of the series:


Veterans of TDD: James Shore

May 9, 2014

Episode 3 is the  shortened transcript of an interview I did with James Shore via Skype; that’s why it’s a bit longer than the previous ones.

James Shore is a long-time practitioner of Extreme Programming and a thought leader in the Agile software development community. He is author of the seminal book “The Art of Agile Development”. James is also founder of the Agile Fluency™ project with Diana Larsen. Learn more at or read about the fluency model at

In his lastest screencasting project Let’s Code JavaScript  he is immerging into the intricacies of test-driven JavaScript. You can find more information about James and how to contact him on

Q: When was your first contact with TDD and what did you think about it at the time?

JS: I first heard about XP from a colleague of mine in 1999. We were working on a small, four-programmer project. I was the technical lead and one of the people on the project said: “Have you heard about the C2-wiki, Ward Cunningham’s wiki?” I said: “No”. He said: “It’s really cool, you should check it out. They are talking about this thing called Extreme Programming (XP) – we should do it”. I was like: “Yeah, right. We’re doing this other thing called ‘Feature-Driven Development’. Let’s do that instead.” So we didn’t do XP, we didn’t do test-driven development, but it turned me onto Ward’s wiki and I thought: ‘This is ridiculous! These are the dumbest ideas ever.’ What is this pair-programming stuff, test-driven development? But I was really into software process; and the XP stuff looked really weird but also fascinating. So after that contract ended I had an opportunity for a little tiny project to do web scraping with Perl and I thought: ‘Hey, let’s try TDD’ and it worked out really well. You know, TDD for regular expressions is a match made in heaven. So when that same company brought me back to continue working on the same project, the one they wanted to do with XP, I said: Let’s try this XP thing. And we tried it and we did everything, TDD, we did pair-programming, we did all the XP practices – huge success. The team really jelled, we really wrote excellent code and I was hooked.

Q: And what did eventually convince you that TDD is more than just working in this one instance but a worthwhile approach overall?

JS: I have always been pretty willing to experiment, especially with things that look ridiculous. And so when I tried it on that Perl project I gave it a real shot. I was lucky in that TDD is a very good fit for testing regular expressions and it worked really well. That was all it took.

Q: What has changed in the way you practice and teach TDD since those early days?

JS: That is a difficult question to answer. In the early days, you know, the very first project I did, I was doing a lot of reading about TDD and I would say that one major difference is that on that first project, we were doing a web application that was basically the kind of thing that would be a perfect fit for Rails today. It was basically a CRUD app that gave a web interface to a database. Doing TDD with that wasn’t particularly easy because there was no application logic. There was just a lot of data moving around, a lot of database access and yet we used TDD heavily. We didn’t use evolutionary design for that project. We used some upfront design and then evolutionary design. The upfront design we came up with was overly complex. It was a five-layer architecture; between each layer we used mock objects to isolate each layer for testing. And that was not pleasant, it really didn’t work well. The rest of the team which wasn’t very proficient with Object-Oriented Programming, found it extremely confusing. We found it to be brittle. Of course, at that point there weren’t a lot of good mocking tools or anything like that either. So one thing that I took away from that was that mock objects are a seductive idea that are probably indicative of design flaws. So since those early days I have moved more and more towards seeing mock objects as an easy way to get started with TDD and a sign of a flaw that needs to eventually be worked out with evolutionary design. That’s a crusade I’ve been on for, I guess, 14 years now and I’m continually refining my understanding of how to do that well.

Q: So would you say that holds true for the kind of mock approach that Steve Freeman and Nat Pryce are proposing in their book?

JS: I don’t know. I have not read their book ‘Growing Object-Oriented Software, Guided by Tests’. I really want to, it’s on my list. I want to actually sit down and study it, and write blogs about it and stuff like that, like really go through it. And I haven’t had the opportunity to sit down with them and talk with them in person about it, although I wish I could. I do know that when we introduced mocks in 2000, it was based on the mock object paper that they’ve had part of, you know, the first mock paper. It was done to the best of our understanding of how mock objects work. We weren’t mocking out third-party dependencies, we were mocking out objects we controlled, which, I understand, is part of what they care about, and we were designing our code through interfaces. But I don’t know for a fact that we were doing it the way they would have.

Q: Are there situations in which you consider TDD not to be the right approach for developing software?

JS: Absolutely. TDD has a learning curve. It’s not appropriate if you’re dealing with software that is a prototype, that is going to be thrown away or where there’s immense time pressure and there’s no or only minimal consequences for increased costs down the road. And those costs are higher if you haven’t done TDD before or if you’re trying to do TDD in an entirely new problem domain. TDD is a lot harder to do in environments that haven’t been created with TDD in mind. So, for projects that are going to live less than six weeks I would definitely not use TDD if I didn’t already know how to do it in that environment. For projects that are going to live more than three months I almost certainly would, depending on the costs. And for projects that are going to live more than a year I unquestionably would.

Q: So it’s just a matter of the time-frame of how long your project is going to live?

JS: Oh yeah. There’s no technical reason not to use TDD. I haven’t found a single type of thing I couldn’t figure out how to TDD eventually, and the mental rigor that TDD brings is so powerful, as well as the sense of confidence of being able to refactor so important, that I will use it whenever it makes financial sense or whenever the investment makes sense.

Q: What do you think is TDD’s relevance in todays world of lean startups, functional and concurrent programming, continuous delivery, and this mobile everywhere?

JS: Well, that is a huge list of things. I think that TDD is relevant anywhere that you are trying to create code you want to change. And we can go into specific cases from there if you like.

Q: Let’s dive into the Lean Startup context where many people argue that – because requirements will change so often – it’s not worth to fix them using automated tests.

JS: If someone is saying that TDD makes your software hard to change, then they are doing TDD wrong. This goes back to the mock-objects approach. I’ve seen many people who misunderstand mock objects and use TDD to basically test-drive the lines of code they are writing and thereby lock their design in concrete; and, yes, that makes code that is incredibly difficult to change. But in my mind, TDD is about enabling refactoring and enabling evolutionary design. So if your code is changing rapidly then it’s even more valuable to have TDD on it, so you can change it easily without worrying about breaking things. Now that said, in a Lean Startup environment you want to create validated learning. You want to understand what your market is, you want to create experiments and learn from those experiments rapidly and in some cases those experiments are entirely throw-away. But if you are doing something you’re going to keep and change, I would use TDD.

Q: James, you’ve recently been doing a lot of TDD in the JavaScript space – do you think that test-driving software that runs, at least to a considerable degree, in a browser is a new game or is it just the same old techniques applied to a different thing?

JS: It’s absolutely the same techniques. I’ve been doing this screencast for two years now – and doing TDD with JavaScript is just like doing TDD in any other language. There are, at least in terms of design principles and TDD techniques, some things that are unique to JavaScript; specifically that not all browsers behave the same way. So it’s really important in JavaScript to do cross-browser testing which we do in the screencast with Karma. And JavaScript is a dynamic language rather than a static language, so there are some issues there that are different than doing TDD with, say Java. I would say that the biggest difference with JavaScript, once you get away from the whole browser issue, is that, because JavaScript is such a loose language, in terms of “it doesn’t lock things down”, you can go in and basically change any function anywhere. This makes it a lot easier to do things, like spies, and to really poke-in and get stuff done when you need to, even if it’s not great form; that can actually be really, really useful for when you are getting started trying to figure out how to test or how to program something. Just don’t leave it in; be sure you take that mess out.

Q: I seem to remember that you’re not a big fan of acceptance-test-driven development. Why is that, and how do you tackle the problem of verifying the customer’s business requirements?

JS: So ATDD really came out of Ward Cunningham’s work on Fit which was inspired by some work he did in WyCash, a financial company. I talked a lot with Ward about it and my understanding is that – while he was at WyCash – Ward wanted to make sure that the software they were creating, behaved in a way that stock analysts needed it to; and the way the stock analysts worked was with spreadsheets. So he would ask them to give examples of behavior in a spreadsheet. He took those spreadsheets and actually hooked them up to the WyCash program, so that WyCash would display in the spreadsheet the answer it was getting. That turned out to be a fantastic way for the analysts to communicate with the development team about how the software was working. That type of communication between the business experts and the programming team is absolutely crucial. It’s one of the foundational concepts of Agile that we have close-knit communication between our development team and the rest of the business.

And so Fit was born in trying to recreate that; and Selenium came along and Cucumber came along. These later tools have kind of forgotten the original purpose, which is that WyCash spreadsheets were created to create communication with business stake holders. When I look at what people are doing with those tools I don’t see that kind of communication. What I see is a rigid desire that people write a script for the program to follow. Now, when I first got involved with Fit, I was excited about this idea of having business people and programming teams collaborating more strongly, I got along with the project doing consulting on it. What I saw in the field was really demoralizing for me. What I saw was that you go to these business folks and say “Give us these examples and you have to write it like a script so the program can follow it” and invariably I would get two answers: the first answer was ‘I don’t want to do this. Give it to the test department’, and the second answer was: ‘How do I know this is actually really working? How do I know this is working and you’re not just programming to give me the answer I want?’. So what was happening with Fit was that it was being seen as an added burden for these people and, second, it was seen as a way for the programming team to get out of doing the work they were supposed to do. There was a lot of distrust. What it wasn’t being used for, at all, was any sort of really solid communication between the programmers and the business folks about how the program worked and how it was supposed to work.

Q: So, do you recommend to replace the usage of the tool with just more meetings, more talking?

JS: No, I never recommend just more meetings; meetings are pointless. But, if you are having really good communication and that happens to be in a meeting, that’s great. It’s about the communication and the collaboration. Examples on a white board are ultimately the value we got out of Fit and that’s where all the value was. Actually codifying those examples into a funky, not quite human programming language did not have a whole lot of value. The reason this worked for Ward back in WyCash was that he didn’t go to the business team and said ‘I’ve got this tool and you’re going to use it and I’m going to tell you the format to use and give me what I want’. No, he went to the business folks and said ‘I see that you already created these examples in a spreadsheet – let me do something cool’. And that’s very collaborative, it’s very helpful and that’s powerful. He was doing the business folks a favor, he was asking them, ‘How can I take what you’re already doing and make it better’. If you can to that, then ATTD is great. But that’s not what I see people doing with Cucumber and similar tools. What I see people do is pushing their needs on the rest of the organization. And as a result, what they are doing is being ignored. And so it’s turning into yet another way of writing tests, just clunkier than using a programming tool.

Q: That’s about my list of questions. Is there anything you would like to add as an oldtimer?

JS: Yeah, I do have two things I would add. First is: Don’t give up. Because TDD is moments to learn and a life-time to master. I’ve been doing it for 14 years and I’m still learning new things. And there are some things that are really, really easy to do with TDD and that’s what you tend to learn in the class room. But once you start getting into the real world and having to deal with databases and third-party systems and multiple interacting classes, it becomes a lot more challenging and that’s what takes a life-time to learn. If doing TDD in that environment doesn’t cause you to discover new things about how to do design, then you’re probably not trying hard enough.

And the second thing is, for beginners especially, the key to learning TDD is minimizing the time to your next green bar. So as you learn TDD, as I see from beginners over and over again, they take steps that are too big. My recommendation for people who are starting out is to set a stop-watch on your desk – not every day, but as an exercise – and time how long it takes you to get from green bar back to green bar. See if you can get that time down really small. See if you can get the amount of code you write down to just 2 or 3 lines of code between each step. That’s a really valuable exercise and you’ll learn a lot from doing that.

Q: How short should the steps be?

JS: There is no right answer. What I find is that I write just a few lines of code when I know what I’m doing and then I can write just a few lines of code between each step. But there might be a lot of thinking involved in getting to those few lines of code. So it’s not so much about the time it’s about not doing a huge amount between each step. If you really understand the problem-space, if you really are good at TDD you will be able to take really small steps where the number of lines you write are maybe one line of test, one line of production code or two or three. I definitely aim for less than five lines of test before I get my red bar and less than five lines of production code before I get my green bar.

Many thanks, James, for answering my questions!


Other episodes of the series:

Veterans of TDD: Steve Freeman

May 2, 2014

Episode 2 of the series welcomes Steve Freeman as its guest…


Steve Freeman is the most prominent representative of the “London School of TDD”. Together with Nat Price he’s author of “Growing Object-Oriented Software, Guided by Tests” which is arguably the most influential TDD book in recent years. You can follow Steve’s tweets (@sf105) and contact him through

Q: When was your first contact with TDD and what did you think about it at the time?

SF: Sometime around 1997/1998. I was working at OTI in London and we were following the emerging discussion about the C3 project on the C2 wiki. At first our general response was “this couldn’t possibly work”, but we kept trying things and it turned out that they did work. The other event around that time, was that Kent Beck spoke at the SPA conference, so we were able to find out more about how the various practices actually worked.

Q: What did eventually convince you that TDD is a worthwhile approach?

SF: Trying it and finding it helpful for me. I also had a couple of nice counter examples where one member of the team stayed up all night to “fix” something which took the rest of the team the following morning to recover.

Q: What has changed in the way you practice and teach TDD since the early days?

SF: Some of it is raw technique, such as learning how to slice a feature into increments, and how to work with levels of testing. Nowadays, I put a great deal of emphasis on making the tests readable and expressive, rather than asserting every last little detail; actually, I always did but now it’s my top priority. I’ve also come to a better understanding of the use of TDD as a “thinking tool”, to help me clarify my ideas before coding. I’ve also been thinking about using system-level testing to influence large-scale designs.

Finally, I’ve learned that TDD is a deep skill, like anything in programming. A couple of days of training and a flip through a book (even one as good as ours :) is no more than a taster.

Q: Are there situations in which you consider TDD not to be the right approach for developing software? If so, what other techniques and approaches would you recommend in those situations?

SF: Nat Pryce likes to cite Manny Lehman’s categories of software. There are some kinds of “algorithmic” system (P-Systems) that should be addressed by thinking hard — although I’d still think that TDD can contribute to the implementation. I mostly work on “messy” E-Systems that evolve with their environments and don’t have clean requirements. This is where I think TDD, in its larger sense, is particularly appropriate.

Other possible exceptions are where something has to be thrown together to prove a point or where the programmer is exploring a space (I’m thinking of Sam Aaron’s live music coding). There’s also a new generation of “Test in Production”, where the production infrastructure is so robust that programmers can safely experiment with the real system. The point here is to be aware of what I’m prioritising and of what risks I might be accumulating. Kent Beck has a metaphor about prioritising for low-latency vs. sustained throughput of features.

Whatever techniques I’m using, I find there’s usually still value in stopping to think about what I want to have happen before I start coding–even if those assertions aren’t automated.

Q: What do you think is TDD’s relevance in today’s world of lean startups, functional and concurrent programming, continuous delivery and mobile everywhere?

SF: That’s quite a list, and I’m not sure it describes a majority of software environments yet. The short version is that it seems like a good idea to me to have some kind of regression testing, especially if I want to move fast safely. Given that, I find it easier to write the tests first because, in practice, I won’t write them afterwards and that’s too late to help the code. There are some advanced environments where this doesn’t apply, but I wouldn’t risk that until I really understood what I was giving up and what I would do to compensate.

Q: You are one of the prominent figures of the “London school of TDD” which stresses the use of mock objects to specify the collaboration contracts between objects – as opposed to merely  checking an individual object’s behaviour. Would you say that collaboration testing is more valuable than state-based testing? What do you reply to critics who dismiss mock objects for missing out on the “real” integration?

SF: I’ve become very tired of having this argument with people who haven’t read the book and don’t understand what we’re talking about. I don’t prioritise one over the other absolutely, I prioritise the one that’s appropriate for the type of object being tested. If I have an object that has behaviour and I’m doing “Tell, Don’t Ask” then I can only test interactions. That doesn’t apply to other kinds of objects.

The thing about integration only applies if you use mocks to test against external interfaces, which I don’t (we’ve been saying that for over a decade). I use integration tests for that.

The one thing I literally don’t get, although I see it with some good people, is the inability to cope with pre-defined constraints which led to, amongst other things, mockito. That can go away once Java finally has closures. I’ve now stopped fighting this one and use it as a competitive advantage. If people want to listen, we can have a civilised discussion.

Many thanks, Steve, for answering my questions!


Other episodes of the series:


Get every new post delivered to your Inbox.