Why should you learn and use TDD in the first place?

I've been a preacher of test-driven development (TDD) since my initiation to test-first development some time around 2001. And as far as programming goes, I feel crippled if I can't plug a feature in by means of TDD. Sorry. Wrong. Correction follows. I am crippled if I can't plug a feature in by means of TDD. For me TDD is primarily a design aid - a way to be able to think about the solution at hand. If I don't have that tool available, due to e.g. too tightly coupled legacy code, my ability to be productive is reduced. If I work with you, I want you to do TDD, because then I know you'll make the same design considerations I am making. And that is the main reason for me to advocate TDD. It's a hygiene factor, where the absence of it feels like if I'm typing on a keyboard that lies beneath a mattress.

But what should be your motivation for doing TDD? I dont believe that me saying cheese pretty please on my knees is very convincing in that regard. I remember some debacle from 2007 and onward when Robert C. Martin stated that in his opinion a software developer cannot consider him self a "professional" if he did not do TDD. I'm not really sure that it convinced anyone to do TDD, maybe even the exact opposite. So why do I think you should do TDD? Maybe because it provide better quality products? Or maybe it yield better designs? Because it improves communication? Should you do it to avoid being slapped in the face with a cod from your other team-mates that are doing TDD? Does it come out on top after doing a cost-benefit analysis? What should be the important motivational factor here? I might give an answer before I'm done with this blogpost, but first some good old-fashioned arguing.

I'm writing this, because recently I read this blogpost by Jacob Proffitt (it trickled up on the frontpage of Hacker News) and it ripped open some old battle-scars on the topic. The blogpost is somewhat old, but still a good read. The topic of the post is a criticism of another blogpost written by Paul Haack and the article "On the Effectiveness of Test-first Approach to Programming" by Erdogmus et al. Proffitt points out several problems with both the conclusions of Haack's blog post and the article he criticizes. And he allows me to highlight yet another problem that can show up in discussions about TDD. Let me quote the following section from Proffitt's blogpost:

More interesting still would be the correlation between "Testing First" and unit test quality. Ask yourself this, which unit tests are going to be better, the ones created before you've implemented the functionality or those created after? Or, considered another way, at which point do you have a better understanding of the problem domain, before implementing it or after? If you're like me, you understand the problem better after implementing it and can thus create more relevant tests after the fact than before. I know better where the important edge-cases are and how to expose them and that could mean that my tests are more robust.

He concludes that your tests will be better, if they are written after implementing an artefact. Sounds like a reasonable argument, right? Well. I'm so sorry, and I hate to say this so bluntly, but the argument is baloney. The first question in the paragraph should rather be the following: "Ask yourself this, which unit tests are going to be better, the ones created after writing a lot of tests while implementing the functionality, or those bolted on after just implementing the functionality?" I guess it's obvious what my answer to that question is. The point being - I'm not entirely convinced that the author really get TDD, and that is a big obstacle in these kinds of discussions. Anyhow. Bad form by me - as mentioned, it is somewhat old, and the paragraph is taken out of context.

I have been looking for academically published empirical results regarding the merits of TDD for quite some time, and I've managed to dig up some publications around this topic (including the previously mentioned article by Erdogmus et al.) Maximillien and Williams report that, measuring a team in IBM using TDD, the defect rate was reduced by 50% with minimal extra effort, compared to using ad-hoc testing. Later George and Williams reported higher quality with slightly higher cost (The TDD'ers being 18% more expensive than the control-group, and the authors speculate that the cost-difference was because the developers in the control-group did not write automated tests that they were required to do.) Nagapan, Maximilien, Bhat and Williams studied several teams working at Microsoft, and the core finding was that development cost increased 15-35% (measured in a subjective fashion, details omitted) whilst the pre-release bug density decreased with 40-90% (yes, one of the teams had a pre-release bug-density at 90% lower). There are some more studies, involving e.g. using student groups (with lacking experience) implementing the same features using either TDD or… that other approach - I don't know what to call it really - plain-old-hacking? But I think studies involving software professionals are more relevant. This is not a survey or meta-study, but the trend seems to be that if developers use TDD, quality is improved, with a small increase in pre-release cost.

As far as I'm concerned, the academic results places the choice between TDD and plain-old-hacking, squarely in the management camp. The development cost might be slightly higher,¹ but an increase in quality that reduces both uncertainty and time spent in the "stabilization" period between feature-ready and actual release-time, should tilt the decision in favor of TDD. Plus, I'm quite confident that we can extrapolate the evidence to mean lower cost during maintenance and future development as well. Therefore, from a managerial and economical perspective, the only time you don't use TDD, is if you know you are building a short lived (e.g. development and maintenance totaling to less than a man-year) prototype that you know you are going to throw away (but damnit, the prototype was so successful, that you couldn't afford to chuck it… So there you are then, doing plain-old-hacking again writing legacy code from the start.) But this is opinion, based on others and my own anecdotical experience. And of course I want TDD to be "better" in some way, so I am definitely susceptible to confirmation bias.

Any such studies are, by their very nature, anecdotes. And the weaknesses with anekdotes are easy to point out by nay-sayers. E.g. we don't know the differences in skill-sets, perhaps the team who did TDD was rockstar programmers and still used more time, whilst the other team was mediocre hackers who could barely run the compiler and still delivered their product quicker than the rockstars. Who knows. Stupid rockstars who used TDD then. TDD-nay-sayers will latch on to any such weakness that can be found and never let go. Any (positive) academic result on TDD is bound to fall dead on the ground (I guess confirmation bias works both ways.)

On the other side, empirical evidence about TDD (or any other fab engineering tactics, like for example pair-programming) are also bound to be short sighted. Of practical reasons, they don't shed any light on the long term effects on a codebase evolving over the full lifetime of a library or a product. This puts us back to plain old rhetoric and creation of hot-air, breathing out what-if scenario argumentation, where the better and most convincing lawyer win the truth. We can not know how TDD influences productivity, quality and costs in a truly positivistic sense. That is, until we can simulate a skilled team of programmers, and then start two simulations at exactly the same time in their lives - in one which they use TDD and the other where they use plain-old-hacking - run the experiments a gazillion times (I don't have the time to brush up on the maths required to figure out how many experiments we need to run before we get a statistically significant result) and compare which one come out top cost-wise.

So what am I getting at really? After reading Proffitts blog-post, I realized that the academics is pretty much a straw-man when it comes to any one persons motivation for doing TDD. The real motivation should be the sentiment that as a software developer is a crafts-man. TDD, BDD, Design patterns, SOLID, Pair Programming or what ever technique or practice cannot be dismissed, without having put enough effort in to learning it, and practicing it long enough to do it well, and then choose to do it or not. After putting in the effort, I can make up my own mind. It's an important part of evolving and growing. That's why you should learn how to do TDD. Not because the quality of code becomes better (subjective and hard to measure, but it does, trust me on this), not because it make you work faster (you probably don't, especially not at the beginning), not because it makes the release-phase more predictive (it does that to), not because it makes you sexy (try "I do test-driven development" as a pick-up line) but - because in the end it expands your skill-set and the way you think about code.

If you don't do TDD, I don't think it says anything about your professionalism or skills as a developer. I do on the other hand believe that it might say something about you as a person. If you are not doing TDD, you are not doing TDD because you don't know how to do it. Because it's a skill, and learning this skill takes effort. Learning TDD takes a lot more effort than just doing the bowling kata a couple of times. Applying it in all kinds of different situations takes practice practice practice. And some have a bias towards not making that effort, unless the payoff in the other end is known. That's okay. I can understand that. Others yet have a bias towards not wanting to feel incompetent while learning the new skill, as plain-old-hacking have always worked in the past. I understand that as well. And then I think you are either one or both of those kinds of persons, or you just havent heard about TDD yet (unlikely, if you read this… )

So it boils down to these two kids swimming in a lake, where one say "Come over here, the water is warmer." The other kid can't know if that's the case. The only way to find out is to swim over and find out if the other kid was telling the truth. You can't understand the feeling you get, when you can do a totally safe large refactoring of your codebase, unless you have done it. Or the joy of plugging together the components you wrote using TDD and then have everything just working, without having experienced it. You can't understand how the unittests guides your design choices, without having the experience to understand how the unittests guides your design choices. When you start to get that, you are sucked in. You are test infected. And now, real quick, why isn't this a circular argument! Come over here, the water is warmer, and you'll understand it all so much better when you view it from this angle. I promise.

Footnotes:

¹ I'm not convinced there is any added cost if you use TDD with, say, dynamically typed languages like Ruby, Python or Lisp.

Presenter First with Qt

Somewhat recently a friend of mine sent me an article on the Presenter First pattern - a 'recipe' to build applications using the Model-View-Presenter (MVP) pattern with Test Driven Development (TDD). The article seemed to be tuned against C# and Java, so I wondered how I would apply Presenter First with Qt and C++. This is a short experiment with implementing a UI with Qt, using the Presenter First pattern.

The idea with the 'Presenter First' pattern, is to implement the Presenter first in a test-driven fashion. In the MVP-pattern, the presenter only see an abstract representation of the view and the model, listens to events from them and glue their behaviors together. In the PresenterFirst formulation from the aforementioned article, model and view are interfaces. Since we are using Qt, we want to utilize the signal and slot mechanism so both classes would be abstract classes that somehow inherits from QObject. For starters the Model can inherit QObject, and the View can inherit QWidget. Both will be abstract with only pure virtual functions and slots. While going through this blog-post I will put emphasis on the test-code and the Presenter code, and for the most part regard the View and Model code as 'given'. As you will see, the tests we implement turns out to correspond very closely to the usecases we are implementing and we are able to test all aspects of the behaviour of the application under user interaction.

The working example will be a simple "editor" with two controls: a list-view to list available entities by name on the left, and a text edit-view for editing the content associated with the name on the right. Lets imagine that the content are equations, so we call it an equation editor. When writing this, I wanted to try using a component from the Qt itemviews, because they are intentionally written in a way that merges the View and Controller components of a MVC-design. ¹ This makes them a less-than-perfect fit with the MVP and PresenterFirst, but we'll see if we can make due never the less.

The ui should look something like this. Equation names in the left listview, the actual equation in the textedit on the right.

When interacting with the editor, we would like to implement the following user-stories:

• When I select equation named E in the dropbox, the equation should show in the equation editor
• When I click 'new equation', a dialogue should show for me to select a name for the new equation, and the new equation should afterwards show in the item-view on the left
• When the user edit the equation-text for an equation X, then select a different equation Y, and then again select the recently edited equation X, the originally written text for equation X should be displayed in the editor
• When I click on a selected equationname, I should be able to edit it, and only the new name should be used to reference the equation

It won't be an equation editor yet after implementing them, but we'll be off with a good start! It doesn't matter which of these stories we implement first, even though the last story could benefit somewhat from doing the first one in the list first.

Let's dive right in. We start with the add-new-equation story. The first tests for the presenter could thus look something like this:

void
PresenterTest::test_addNewEquation(){
  MockEquationView * v = new MockEquationView;
  MockEquationModel * m = new MockEquationModel;

  Presenter p( v, m );
  QString equationName("new equation");
  v->user_createNewEquation( equationName );

  m->verify_newEquationCreated( equationName );
  v->verify_modelChangedRecieved( 1 );
}

This reveals the basic MVP design. The Presenter have references to an abstract Model and an abstract View, and listens for updates via some sort of event-model to glue their behaviours together.

The overall design of Model View Presenter, and presenter first. The presenter only knows about the abstract representations of view and model, and listens to events.

During the development we will implement the abstract classes EquationView and EquationModel, with their mock-objects MockEquationView and MockEquationModel. The 'View' here represents the target UI form, while the 'Model' stores equation-names and equations and ties them together. After we are done, we add ConcreteEquationView and ConcreteEquationModel - the latter developed with the help of TDD.

We drive the test by telling our fake view that the user want to "create a new equation", and afterwards we verify that there was a new equation created in the model and that the view got an update about changes in the model.

To see what's going on better, I'll prefix all the functions on the fake version of the view used to "simulate" user interactions in order to drive the presenter and the model with "user_" - so we don't confuse the method-names from the test-double with the methods of the "real" view. Also - I'll roll my own mocks, I will not use a mocking library in these examples (even though I could have benefited from using something like Google Mock.)

How should we implement this behavior? If the presenter listen to a signal from the view that asks for creation of a new equation we would be in good shape!

Presenter::Presenter(EquationView* v, EquationModel* m)
 : view_(v),
   model_(m)
{
  connect(view_, SIGNAL(createNewEquation()), this, SLOT(addNewEquation()));
}

This makes it obvious where the events come from. It also clarifies a presenter-first idiom: We 'drive' the Presenter through simulated user behavior, created by programmatically interacting with a fake view-class.

Note that the createNewEquation and addNewEquation signal-slot pair have no arguments, but from the user story we are expecting a specific name for the equation to be created - the story even specify that a dialog should pop up. We can't have dialogues popping up in our automated unittests - there's no user on the other end - so I want to stub out that behavior. There are many ways to achieve this. In the example article that inspired this blog post, they fired an event to show the dialogbox. Instead in this example I just ask the view to create a new equation name for me through createNewEquationName, and let ConcreteEquationView pop up the dialogue, get the name, and return this from an interface function. For the test-view, I implement createNewEquationName so it returns the name passed with user_createNewEquation().

void
Presenter::addNewEquation()
{
  QString eqname = view_->createNewEquationName(); 
  model_->addNewEquation(eqname);
  view_->modelChanged();
}

The code above should be sufficient to make our test pass. The modelChanged() function can be a pure virtual function or slot on the View object, and in our MockView we just register that it has been called - we don't need to introspect the view in order to know that this test passes.

Below is the state of EquationView, MockEquationView and EquationModel at this point. The verification code in MockEquationModel is similar to that of MockEquationView.

class EquationView : public QObject
{
public:
  Q_OBJECT;
  EquationView(QObject * parent = NULL);
  virtual ~EquationView();

  virtual void modelChanged() = 0;
  virtual QString createNewEquationName() const = 0;
signals:
  void createNewEquation();
};

class MockEquationView : public EquationView
{
public:
  Q_OBJECT;
  MockEquationView(QWidget * parent = NULL)
    : modelchanged(0)
  { 
  }

  virtual ~TestEquationView()
  {}

  void user_createNewEquation(QString equationname) 
  {
    createNewEquationName_ = equationname;
    emit createNewEquation();
  }

  virtual QString createNewEquationName() const 
  {
    return createNewEquationName_;
  }

  virtual void modelChanged() 
  {
    modelchanged++;
  }

  void verify_modelChangedReceived( int correctupdatecount ){
    QVERIFY(modelchanged == correctupdatedcount);
  }

private:
  QString createNewEquationName_;
  int modelchanged;
};

class EquationModel : public QObject
{
public:
  Q_OBJECT;
  EquationModel(QObject * parent = NULL);
  virtual ~EquationModel();
  virtual void addNewEquation(QString equationname) = 0;
};

This completes the first story. Next we enable selection of different equations in the listview. To be certain that this story is complete, we need a model with several items, and we need to verify that the correct equation text is set in the text-edit after changing the index.

void
PresenterTest::test_indexChanged()
{
  MockEquationView * v = createView();
  EquationModel * m = new MockEquationModel;
  populateModel(m); // adds three equations. 

  Presenter p ( v, m );

  int indexnr = 1; // indexnr < number of equations defined in Model.
  v->user_changeIndexTo(indexnr);
  v->verify_currentEquationIs(m->getEquation(indexnr));
}

This drives a little bit of design. We expect that, when we change equation in the listview, the view must be told what equation to put in the equation text-box. So it seems that the Presenter must listen to some change-of-index event, be able to fetch the equation for this index from the model, and then pass the correct text from the model, to the view. The presenter needs a slot like this:

void
Presenter::changeEquationIndex(int equationidx)
{
  v->setEquationText(m->getEquation(equationidx));
}

The view class need a corresponding signal that we can emit from MockEquationView::user_changeIndexTo(int indexnr) (or from the concrete view when we implement that.) This signal-slot pair needs to be connected in the Presenter constructor.

Presenter::Presenter(EquationView* v, EquationModel* m)
{
  connect(v, SIGNAL(createNewEquation()), this,  SLOT(addNewEquation()));
  connect(v, SIGNAL(equationIndexChanged(int)), this, SLOT(changeEquationIndex(int x)));
}

Our fake view can then verify that it got the correct equation text. Our test passes. Now we know that the View have been told to display the correct data both in the listview and the equation editor. The story is complete.

The third story, and the third test, is for the edit equation story. When we edit the equation text in the text-editor on the right in the view, the text should be sent to the model.

void
PresenterTest::test_editedTextIsSetInModel()
{
  MockEquationView * v = new MockEquationView;
  MockEquationModel * m = new MockEquationModel;
  populateModel(m); // adds a couple of name-text pairs

  QString newText = "My new Equation.";
  int activeIndex = 1;
  v->user_changeIndexTo(activeIndex);
  v->user_editEquationText(newText);

  m->verify_equationText(activeIndex, newText);
}

In the test we simulate that the user chooses an equation through user_changeIndexTo, and then simulate that the user_editEquationText to something. Then, in order to verify that our Presenter works as we want it to we check that the equation-text sent to the model is correct.

In order to make this test pass, we need two things: A signal from the view that tells the presenter that text have changed, and a corresponding slot on the presenter that updates the equation-text associated with the current active equation. That means we would need to keep track of what the current equation is. The tracking of current state belongs in the presenter.

Presenter::Presenter(EquationModel * m, EquationView * v, QObject * parent) :
  QObject(parent),
  model_(m),
  view_(v),
  currentEquationIndex(0)
{
  connect(v, SIGNAL(createNewEquation()), this,  SLOT(addNewEquation()));
  connect(v, SIGNAL(equationIndexChanged(int)), this, SLOT(changeEquationIndex(int x)));
  connect(v, SIGNAL(equationChanged(QString)), this, SLOT(currentEquationTextChanged(QString)));
}

void
Presenter::changeEquationIndex(int equationidx)
{
  currentEquationIndex = equationidx;
  v->setEquationText(m->getEquation(currentequation));
}

void
Presenter::currentEquationTextChanged(QString equation)
{
  m->setEquation(currentEquationIndex, equation);
}

We added the connection in the constructor, added currentEquationIndex as state in the presenter, and implemented setActiveEquation. A handful of lines of code, and our test is passing.

No for the final test: Editing the name of an already added equation. So far we've only used signals and slots from Qt, and we have talked about how new equations come to be (and that this requires the View to receive a modelChanged signal) and how the text in the equationview is updated. But we have not talked about how the listview with equation-names to edit is populated - this have been a detail we could just gloss over so far. Now that we should be allowed to change the name of the equation in the listview, we need to supply an editable model from Qt's modelview components to the view and then test that the data stored in it is updated. Granted, we could make this work without using an QAbstractItemModel - we could somehow have kept the names in our model in synch with what's in the view by setting a QStringList of equation-names every time the model changes or adding and removing names by some other means. This would increase the amount of logic we would have to add to the concrete View-implementation - the code we are trying to keep as minimal as possible. By using an QAbstractItemModel on the itemview and simulate the interaction from the MockView, everything is more or less automated from the ConcreteEquationView's viewpoint.

For our next test then, the view must change the name for an item, and we need to check that the name is updated in the model.

void
PresenterTest::test_equationNameIsChanged()
{
    MockEquationView * v = new MockEquationView;
    MockEquationModel * m = new MockEquationModel;
    QString originalname = "Name0";
    int idx = m->addNewEquation( originalname, "Some text");
    Presenter p(m, v);
    QString newname = "Name1";
    v->user_changeEquationName(idx, newname);

    QCOMPARE( m->getEquationName(idx), newname );
}

The key driving function of the test - user_changeEquationName - simulates the behavior of the listview by first creating a correct QModelIndex for the name and then call abstractlistmodel->setData(index,data) to update the name stored in the model.

There are several valid alternatives for how to get an QAbstractListModel from our EquationModel:

• The Presenter can act as a proxy and inherit from QAbstractListModel and get the necessary data from it's EquationModel reference.
• We could create a concrete EquationListModel that inherit from QAbstractListModel and which take an EquationModel pointer as a constructor argument and sends getData and setData calls to it (again representing our model by proxy.)
• The EquationModel it self could inherit from QAbstractItemListModel, and be both.

What matters most at this point, is that these are the options. There are three. Not four or five. A getter or factory-method on EquationModel that returns an QAbstractItemModel is a jolly bad idea. ²

In order to quickly make the test pass, I chose the last option - the EquationModel inherit from QAbstractItemListModel - then I only need to add a setData function to it, and use it to update the equation-name.

The constructor of Presenter now only need to be updated so that it passes the EquationModel (as a QAbstractListModel) to the EquationView, and after a correct implementation of setData my test passes.

Presenter::Presenter(EquationView* v, EquationModel* m)
{
  connect(v, SIGNAL(createNewEquation()), this,  SLOT(addNewEquation()));
  connect(v, SIGNAL(equationIndexChanged(int)), this, SLOT(changeEquationIndex(int x)));
  connect(v, SIGNAL(equationChanged(QString)), this, SLOT(currentEquationTextChanged(QString)));
  v->setModel( (QAbstractListModel*) m ); // cast for emphasis
}

Voila, all user stories implemented test first, without ever having to open an application with a Widget. So far so good.

There are several observations we can make while wrapping up here. Notice first how all the tests "simulate" the user interaction. The driving functions get names that are very simmilar to the user stories: user_editsEquationText is a good example. Also, notice how none of the tests actually calls functions on the Presenter, save from the constructor. The tests only calls functions on the mocks, and the behavior of the Presenter is then implicitly verified. All Presenter slots and functions can in other word be private. Both of these observations are in line with the experiences reported in the previously mentioned article.

I also want to emphasis how the tests are agnostic to many of our design decisions. Notice for example how test_changeEquationName is totally ignorant to our actual design choice - we can choose any one of the three alternatives proposed for getting a QAbstractItemModel, and the test still stays the same. Presenter First drives you to implement your tests on a comfortable distance from your design, where they support your design decisions without geting into the way when refactoring.

When using Presenter First with Qt, all UI-classes in most cases only need to relay signals from the relevant ui-components to the Presenter via it's own signals, and implement the slots to set the data to display in the correct ui components. A test of the GUI simply need to verify that it is hooked up - e.g. that pressing a button makes the view emit the correct signal - as the test-driver for the presenter tests the actual functionality behind.

There is a full example with a view implementation in addition to the tests at bitbucket. It's not an equation editor yet, but as far as being an example of Presenter First with Qt, it suffice. Thank you for reading!

Footnotes:

¹ This is, btw, the design choice in Qt I have the biggest beef with as it lures the unwary developer to add tons of behavior in view classes. The result is often classes that mix responsibilities and are difficult to test.

² I also might mention here that I usually just avoid the QTreeWidget by default. In order to avoid the duplication of data and the efforts needed to keep the QTreeWidgetItems in synch with the underlying datamodel I like to implement a QAbstractItemModel around my data instead.