I've been a preacher of test-driven development (TDD) since my initiation to test-first development some time around 2001. And as far as programming goes, I feel crippled if I can't plug a feature in by means of TDD. Sorry. Wrong. Correction follows. I am crippled if I can't plug a feature in by means of TDD. For me TDD is primarily a design aid - a way to be able to think about the solution at hand. If I don't have that tool available, due to e.g. too tightly coupled legacy code, my ability to be productive is reduced. If I work with you, I want you to do TDD, because then I know you'll make the same design considerations I am making. And that is the main reason for me to advocate TDD. It's a hygiene factor, where the absence of it feels like if I'm typing on a keyboard that lies beneath a mattress.
But what should be your motivation for doing TDD? I dont believe that me saying cheese pretty please on my knees is very convincing in that regard. I remember some debacle from 2007 and onward when Robert C. Martin stated that in his opinion a software developer cannot consider him self a "professional" if he did not do TDD. I'm not really sure that it convinced anyone to do TDD, maybe even the exact opposite. So why do I think you should do TDD? Maybe because it provide better quality products? Or maybe it yield better designs? Because it improves communication? Should you do it to avoid being slapped in the face with a cod from your other team-mates that are doing TDD? Does it come out on top after doing a cost-benefit analysis? What should be the important motivational factor here? I might give an answer before I'm done with this blogpost, but first some good old-fashioned arguing.
I'm writing this, because recently I read this blogpost by Jacob Proffitt (it trickled up on the frontpage of Hacker News) and it ripped open some old battle-scars on the topic. The blogpost is somewhat old, but still a good read. The topic of the post is a criticism of another blogpost written by Paul Haack and the article "On the Effectiveness of Test-first Approach to Programming" by Erdogmus et al. Proffitt points out several problems with both the conclusions of Haack's blog post and the article he criticizes. And he allows me to highlight yet another problem that can show up in discussions about TDD. Let me quote the following section from Proffitt's blogpost:
More interesting still would be the correlation between "Testing First" and unit test quality. Ask yourself this, which unit tests are going to be better, the ones created before you've implemented the functionality or those created after? Or, considered another way, at which point do you have a better understanding of the problem domain, before implementing it or after? If you're like me, you understand the problem better after implementing it and can thus create more relevant tests after the fact than before. I know better where the important edge-cases are and how to expose them and that could mean that my tests are more robust.
He concludes that your tests will be better, if they are written after implementing an artefact. Sounds like a reasonable argument, right? Well. I'm so sorry, and I hate to say this so bluntly, but the argument is baloney. The first question in the paragraph should rather be the following: "Ask yourself this, which unit tests are going to be better, the ones created after writing a lot of tests while implementing the functionality, or those bolted on after just implementing the functionality?" I guess it's obvious what my answer to that question is. The point being - I'm not entirely convinced that the author really get TDD, and that is a big obstacle in these kinds of discussions. Anyhow. Bad form by me - as mentioned, it is somewhat old, and the paragraph is taken out of context.
I have been looking for academically published empirical results regarding the merits of TDD for quite some time, and I've managed to dig up some publications around this topic (including the previously mentioned article by Erdogmus et al.) Maximillien and Williams report that, measuring a team in IBM using TDD, the defect rate was reduced by 50% with minimal extra effort, compared to using ad-hoc testing. Later George and Williams reported higher quality with slightly higher cost (The TDD'ers being 18% more expensive than the control-group, and the authors speculate that the cost-difference was because the developers in the control-group did not write automated tests that they were required to do.) Nagapan, Maximilien, Bhat and Williams studied several teams working at Microsoft, and the core finding was that development cost increased 15-35% (measured in a subjective fashion, details omitted) whilst the pre-release bug density decreased with 40-90% (yes, one of the teams had a pre-release bug-density at 90% lower). There are some more studies, involving e.g. using student groups (with lacking experience) implementing the same features using either TDD or… that other approach - I don't know what to call it really - plain-old-hacking? But I think studies involving software professionals are more relevant. This is not a survey or meta-study, but the trend seems to be that if developers use TDD, quality is improved, with a small increase in pre-release cost.
As far as I'm concerned, the academic results places the choice between TDD and plain-old-hacking, squarely in the management camp. The development cost might be slightly higher,1 but an increase in quality that reduces both uncertainty and time spent in the "stabilization" period between feature-ready and actual release-time, should tilt the decision in favor of TDD. Plus, I'm quite confident that we can extrapolate the evidence to mean lower cost during maintenance and future development as well. Therefore, from a managerial and economical perspective, the only time you don't use TDD, is if you know you are building a short lived (e.g. development and maintenance totaling to less than a man-year) prototype that you know you are going to throw away (but damnit, the prototype was so successful, that you couldn't afford to chuck it… So there you are then, doing plain-old-hacking again writing legacy code from the start.) But this is opinion, based on others and my own anecdotical experience. And of course I want TDD to be "better" in some way, so I am definitely susceptible to confirmation bias.
Any such studies are, by their very nature, anecdotes. And the weaknesses with anekdotes are easy to point out by nay-sayers. E.g. we don't know the differences in skill-sets, perhaps the team who did TDD was rockstar programmers and still used more time, whilst the other team was mediocre hackers who could barely run the compiler and still delivered their product quicker than the rockstars. Who knows. Stupid rockstars who used TDD then. TDD-nay-sayers will latch on to any such weakness that can be found and never let go. Any (positive) academic result on TDD is bound to fall dead on the ground (I guess confirmation bias works both ways.)
On the other side, empirical evidence about TDD (or any other fab engineering tactics, like for example pair-programming) are also bound to be short sighted. Of practical reasons, they don't shed any light on the long term effects on a codebase evolving over the full lifetime of a library or a product. This puts us back to plain old rhetoric and creation of hot-air, breathing out what-if scenario argumentation, where the better and most convincing lawyer win the truth. We can not know how TDD influences productivity, quality and costs in a truly positivistic sense. That is, until we can simulate a skilled team of programmers, and then start two simulations at exactly the same time in their lives - in one which they use TDD and the other where they use plain-old-hacking - run the experiments a gazillion times (I don't have the time to brush up on the maths required to figure out how many experiments we need to run before we get a statistically significant result) and compare which one come out top cost-wise.
So what am I getting at really? After reading Proffitts blog-post, I realized that the academics is pretty much a straw-man when it comes to any one persons motivation for doing TDD. The real motivation should be the sentiment that as a software developer is a crafts-man. TDD, BDD, Design patterns, SOLID, Pair Programming or what ever technique or practice cannot be dismissed, without having put enough effort in to learning it, and practicing it long enough to do it well, and then choose to do it or not. After putting in the effort, I can make up my own mind. It's an important part of evolving and growing. That's why you should learn how to do TDD. Not because the quality of code becomes better (subjective and hard to measure, but it does, trust me on this), not because it make you work faster (you probably don't, especially not at the beginning), not because it makes the release-phase more predictive (it does that to), not because it makes you sexy (try "I do test-driven development" as a pick-up line) but - because in the end it expands your skill-set and the way you think about code.
If you don't do TDD, I don't think it says anything about your professionalism or skills as a developer. I do on the other hand believe that it might say something about you as a person. If you are not doing TDD, you are not doing TDD because you don't know how to do it. Because it's a skill, and learning this skill takes effort. Learning TDD takes a lot more effort than just doing the bowling kata a couple of times. Applying it in all kinds of different situations takes practice practice practice. And some have a bias towards not making that effort, unless the payoff in the other end is known. That's okay. I can understand that. Others yet have a bias towards not wanting to feel incompetent while learning the new skill, as plain-old-hacking have always worked in the past. I understand that as well. And then I think you are either one or both of those kinds of persons, or you just havent heard about TDD yet (unlikely, if you read this… )
So it boils down to these two kids swimming in a lake, where one say "Come over here, the water is warmer." The other kid can't know if that's the case. The only way to find out is to swim over and find out if the other kid was telling the truth. You can't understand the feeling you get, when you can do a totally safe large refactoring of your codebase, unless you have done it. Or the joy of plugging together the components you wrote using TDD and then have everything just working, without having experienced it. You can't understand how the unittests guides your design choices, without having the experience to understand how the unittests guides your design choices. When you start to get that, you are sucked in. You are test infected. And now, real quick, why isn't this a circular argument! Come over here, the water is warmer, and you'll understand it all so much better when you view it from this angle. I promise.
Footnotes:
1 I'm not convinced there is any added cost if you use TDD with, say, dynamically typed languages like Ruby, Python or Lisp.
No comments:
Post a Comment