2012-02-19

Discovering Design

Discovering Design

1 A simple editor

In this post, I'll try to demonstrate how good and reusable designs and components sort of pop up - or emerge - almost all by them selves, as long as you pay attention to some very simple rules - the system for simple design described by various eXtreme Programming practitioners, and communicated by Kent Beck in his book Extreme Programming Explained as follows:

In prioritized order, your code always:

  1. Runs all the Tests
  2. Reveals all the intention
  3. Have no duplication
  4. Contain the fewest number of classes or methods

In this post, most (if not all) programming is "rear mirror driving", meaning that I intentionally avoid thinking to much of the future. Instead I try deliberately to keep only three things in my head (leaving four of my seven memory banks for the programming part.) The first thing is the simple rules, second the current test/requirement and finally the current state of the code (you can be a fusspot and point out that this is a lot more than three things. I'll let that pass.)

I'm not claiming that rear-view-only is a brilliant strategy for writing software. But it's a good way to program in order to make the point I'll be making in this post - namely that these simple rules fosters good designs, and that reusable components (some times surprisingly) tend to just show up in the source, without much forethought, when adhering to them. Keeping my attention to the simple design-rules make the source code itself talk, and it describes how it want to be in order to elegantly assimilate the features I'm adding. Putting (accurate) forethought into the mix might of course improve efficiency - but no forethought today.

Also, as a side-note I might mention that while working with this I discovered that this blogpost is just an attempt at a practical demonstration of what Martin Fowler wrote about in his excellent keynote from 2004.

The programming task we are embarking on is to write a simple text editor with undo and redo capabilities. A short list of requirements are known up front and written in a working document I'll refer to as my todo-list (given below.)

- Add characters to a "document"
- Delete characters from the "document"
- After performing any edit, the user should be able to undo the
  last edit.
- The user also should have the ability to regret undoing, by
  performing a "redo".
- The capability to perform copy and paste will be very nice to
  have.

The code in this post will be written in D (Because I just spent a little time to learn the basics - be warned, I might get some D-idioms totally wrong. Feel free to let me know if I do.)

2 Starting with baby-steps

The content of my file initially looks like the following - the minimum I need, to have something that compiles in to an executable and to which I can add unittests.

unittest{
}
int main(){
  return 0;
}

This is one of the many nice features of D, btw. It has built in unittests. The code in the unittest-sections is compiled in and executed before the main function, if the program is compiled with the -unittest switch. A language that takes unittests seriously at its core. Nice. I can relate to that.

Being a practicing TDDer, I'll start with my first simple test - testing that an empty document is empty. The following does of course not compile:

unittest {
  Editor ed = new Editor;
  assert(ed.size() == 0);
}


int main(){
  return 0;
}

By "faking it 'til I make it" I add the base scaffolding and a null-op implementation that will make my test pass. I'll toss in a run returning 1 just to se the test fail - it does, with a nice display of the current stack. After adding the necessary code, my file looks like this:

class Editor { 
  size_t size() {
    return 0;
  }
}

unittest {
  Editor ed = new Editor;
  assert(ed.size() == 0);
}


int main(){
  return 0;
}

This first section just establishes the basic familliar TDD working pattern:

  1. Write a failing test (failing compilation counts as a failing test)
  2. Make it run (possibly with a minimum effort or by faking it)
  3. Refactor in the green.
  4. Lather, rinse, repeat

I'm writing and programming as I go, trying to mimic the style of Kent Beck's book "Test Driven Development". Also the case-studies in Robert C. Martin's "Clean Code" has been an inspiration.

3 Writing to the document

I'll write the first real test: add some text to the document, and check that the size is identical to the size of the added text. The test fail. I then concatenate any edits to the internal document in the write-function, and the test passes. Fail, pass, clean, fail, pass, clean… The testrunner yields no output - and no news is good news with our current testing framework. The code looks like the following at this point, with a test that the size of the current document matches what we added to it.

class Editor { 
  size_t size() {
    return document.length;
  }
  void write(string edit){
    document ~= edit;
  }

  private string document;
}

// Editor is empty after initialization
unittest {
  Editor ed = new Editor;
  assert(ed.size() == 0);
}

// Editor contains as many characters added, after a write.
unittest {
  Editor ed = new Editor;
  string text = "some text we're adding";
  ed.write(text);
  assert(ed.size() == text.length);
}

At this point we have a simple, but not very useful, document editor that allows us to add a string to an internal document and check how much we've added to it. The unittests are separated in two commented blocks, in order to run with separate states - avoiding leakage of state in between tests.

4 Revealing test intentions

Looking in the rear-view mirror, I already notice two things. First - the comments I've added to describe what's happening in the unittest blocks. Rule 2, the code reveal all intention. Doesn't the comments reveal the intent? No - the comments are not code. They are prose. The code it self should reveal the intent. The comments could instead be names for functions on an object - lets call it a "fixture" (Ok, I didn't invent that just now. Sorry, contaminated state) - then the code would reveal the exact intent.

My first step would thus just be to move the tests as is in to a class, and call this function from the unittest-blocks like follows:

class EditorTest{
  void test_ed_is_empty_after_initialization(){
    Editor ed = new Editor;
    assert(ed.size() == 0);
  }
}

unittest {
  EditorTest fix = new EditorTest;
  fix.test_ed_is_empty_after_initialization();
}

It is equivalent with the old code in functionality, and the name of the function clearly expresses it's intent. I run tests, all green (still no output) and follow up with an equivalent edit for the other unittest-block.

5 Adding undo to the editor

At this point I'm quite happy with fixing the expressed intent, so I would like to put some more production code in there, postponing the duplication of initialization that now cropped up in the file.

Adding undo seems like a natural choice from my todo. I can think of three situations I can undo and test right away - undo on an empty document, undo after one edit, and undo after multiple edits. I add those to my todo-list, and starts on implementing undo after one edit. Wich is equivalent with resetting the state of the document - I should probably put reset on the todo-list as well.

I add a failing test for undo after one edit, fake it so it passes by implementing reset, done, commit.

6 Derailed by rule two and three again

The fixture looks like this now, and as you might observe, I have three lines initializing the Editor and three unittest-blocks executing a named function on our fixture.

class EditorTest{

  void test_ed_is_empty_after_initialization(){
    Editor ed = new Editor;
    assert(ed.size() == 0);
  }

  void test_ed_contains_as_many_characters_added_after_a_write(){
    Editor ed = new Editor;
    string text = "some text we're adding";
    ed.write(text);
    assert(ed.size() == text.length);
  }

  void test_undoing_one_edit_is_equivalent_with_resetting_the_state(){
    Editor ed = new Editor;
    string text = "some text we're adding";
    ed.write(text);
    ed.undo();
    assert(ed.size() == 0);
  }
}

unittest {
  EditorTest fix = new EditorTest;
  fix.test_ed_is_empty_after_initialization();
}

unittest {
  EditorTest fix = new EditorTest;
  fix.test_ed_is_empty_after_initialization();
}

unittest { 
  EditorTest fix = new EditorTest;
  fix.test_undoing_one_edit_is_equivalent_with_resetting_the_state();
}

My last clenaup fixed the intention. Now I have some duplication I would get rid of. From the looks of it, I'll probably do something simmilar for every test I'll be adding. It seems like a regular pattern of sorts, doesn't it? Create an object, manipulate it, and then verify it's state or behaviour (now, was that familiar… )

I also would like to keep the clean state between every function in the test as it is now - this calls for some kind of re-initiation mechanism to set up the test before calling the test method, and maybe something to tear it down afterwards.

The three unittest blocks look identical on an abstract level, so I should be able to find some kind of abstraction or mechanism to merge them in to one. In other words, I want the single unittest block to look somewhat like this:

unittest {
 EditorTest fix = new EditorTest;
 fix.runTests();
}

My plan will be to have an array of closures, and I can build that in the constructor of EditorTest for now. Also, following the pattern from xUnit, I add a setUp function in order to re-initialize the fixture.

This is a point where I have to leap - I know all the changes I need to do, but I don't understand at the time of writing how to "ease in to it" in small, stable intermediary steps.

I start by adding the new unittest codeblock as above, with a failing runTests function. It fails with a long stack-trace that looks like the following.

banach:Discovering rolvseehuus$ dmd -unittest editor.d && ./editor 
core.exception.AssertError@editor(26): Assertion failure
----------------
5   editor                              0x0000958d onAssertError + 65
6   editor                              0x000129f2 _d_assertm + 30
7   editor                              0x000025ef void editor.__assert(int) + 27
8   editor                              0x000026d9 void editor.EditorTest.runTests() + 21
9   editor                              0x00002873 void editor.__unittest4() + 39
10  editor                              0x000025d2 void editor.__modtest() + 26
11  editor                              0x00009a01 extern (C) bool core.runtime.runModuleUnitTests().int __foreachbody260(ref object.ModuleInfo*) + 45
12  editor                              0x00004db3 int object.ModuleInfo.opApply(scope int delegate(ref object.ModuleInfo*)) + 79
13  editor                              0x000098f2 runModuleUnitTests + 134
14  editor                              0x00013122 extern (C) int rt.dmain2.main(int, char**).void runAll() + 38
15  editor                              0x00012c99 extern (C) int rt.dmain2.main(int, char**).void tryExec(scope void delegate()) + 29
16  editor                              0x00012c33 main + 179
17  editor                              0x000025a5 start + 53
banach:Discovering rolvseehuus$ 

Now quick, which test failed? I notice that the time I'm spending to figure out what test is failing, is increasing with the number of unittest blocks I'm adding. This time it's obvious what is failing as it failed intentionally, but pretty soon, I'll start refactoring some model code, a couple of unforeseen tests will fail, and I'll be wasting time by trying to figure out which one. The two seconds spent looking for a familiar function-name in the above stack-trace, warrants a reporting mechanism in my test-system. I add the issue to my todo-list, and go back to setting up my list of test-functions in the constructor.

I move the tests in to a list of closures one by one, running the tests in between every time. I also add the setUp function. When done, my fixture and the unittest-block looks like this:

class EditorTest{

  this(){
    functions = [ delegate(){ test_ed_is_empty_after_initialization();},
                  delegate(){ test_ed_contains_as_many_characters_added_after_a_write();},
                  delegate(){ test_undoing_one_edit_is_equivalent_with_resetting_the_state();} ];
  }

  void setUp(){
    ed = new Editor;
  }

  void runTests(){
    foreach(func; functions){
      setUp();
      func();
    }
  }

  void test_ed_is_empty_after_initialization(){
    assert(ed.size() == 0);
    assert(false);
  }

  void test_ed_contains_as_many_characters_added_after_a_write(){
    string text = "some text we're adding";
    ed.write(text);
    assert(ed.size() == text.length);
  }

  void test_undoing_one_edit_is_equivalent_with_resetting_the_state(){
    string text = "some text we're adding";
    ed.write(text);
    ed.undo();
    assert(ed.size() == 0);
  }

  private Editor ed;
  private void delegate()[] functions;
}

unittest {
  EditorTest fix = new EditorTest;
  fix.runTests();
}

I "registered" all known test-functions in the constructor, and created a runTest function that loops through all tests, executing setUp before every test. I've succeeded in eliminating the duplicate initialization line for the editor - rule 3 compliant. Nice.

But darn! This refactoring subtly changes the behavior of the test-system: All tests are not executed, if one of them fail (How could that happen? Mhm… I know - now test for that functionality…) I decide to quickly fix this with a try-catch-finally block with some state variables and do some experiments with failing tests in order to make sure all is hunky-dory. The completed runTests function now looks like the following:

class EditorTest{
  // ...
  void runTests(){
    int passedCount = 0;
    int failedCount = 0;

    foreach(func; functions){
      bool passed = true;
      try{
        setUp();
        func();
      }catch(Throwable e){
        passed=false;
      }
      finally{
        passed ? passedCount++ : failedCount++; 
      }
    }
    writeln(" Executed ", passedCount + failedCount, " tests");
    writeln(" Passed ", passedCount);
    writeln(" Failed ", failedCount);
  }
  // ...
}

There - all tests are executed no matter what happens, and I get a report telling me if all or just some succeeded. As you might notice, I don't get that full report I was asking for some sections ago about exactly what test failed, but now it is quite clear to me how I can add such a report. A mapping between function and function name instead of an array, and a collecting mechanism in runTests to gather the results. I don't need to add this right away though - I can add some production code instead to get some traction.

Was the refactoring a lot of work? It is hard to tell while I'm writing this - I'm constantly being interrupted by my 8 months old son, while squeezing in some programming and writing every now and then when he is sleeping or playing for himself - but I don't think I ever have spent more than two minutes in "coding mode" thus far in this post (as opposed to "writing mode")

7 Back on track - undo some more

The next test I want to add, is for undoing the last edit after multiple edits. I write a test that iterates through a list of strings and add them to the editor while concatenating them. Finally I write a string to the editor and undo that edit. The editor's internal document and the one generated while executing the test should now be identical:

void test_undo_after_multiple_edits(){
  string[] edits = ["one", "two", "three"];
  string document;
  foreach(edit; edits){
    document ~= edit;
    ed.write(edit);
  }
  ed.write("lasttoberemoved");
  ed.undo();

  assert(ed.getDocument() == document);
}

I compile and execute and - doh - everything passes, and it still looks like only three tests were executed… Didn't I just add the fourth?

banach:Discovering rolvseehuus$ dmd -unittest editor.d && ./editor 
 Executed 3 tests
 Passed 3
 Failed 0

Of course! I needed to add the function in the constructor. That short term memory.. The test fails as expected when fixed:

banach:Discovering rolvseehuus$ dmd -unittest editor.d && ./editor 
 Executed 4 tests
 Passed 3
 Failed 1

This makes it clear to me, that the duplication I have in the tests - that I must add them two times, both in the fixture and the constructor, is a form of duplication that can result in some confusion and Zen-breakage. I add it to my todo-list and continue with the current task - making the failing test pass.

When having the failing test for undo after performing several edits, we are finally getting to the core of the matter. How should we go about storing and changing states inbetween the history of edits? A really good solution would be doing what versioning systems like e.g. git and mercurial does - just store the latest diffs or edits in a stack, and use these to recreate the old document given the current document. It sounds like a lot of work though, so in order to just quickly make my test pass a straight forward solution would be to push the entire current document on to the stack, and then make the edit afterwards. Here is the resulting editor:

class Editor {

  size_t size() {
    return document.length;
  }
  void write(string edit){
    history ~= document;
    document ~= edit;
  }
  void undo(){
    document = history.back();
    history.popBack();
  }
  string getDocument(){
    return document;
  }

  private string document;
  private string[] history;
}

While implementing this functionality, an extra test failed - and I didn't know which one. I wasn't so deep in to things that I wasn't able to immediately figure out what was wrong, but the lack of a good report about what tests are failing is starting to get on my nerves. What will happen when I have a hundred tests? I move reporting from the test-runner up a notch on my todo-list (The reason for the surprising fail was popping the end of an empty array - so this prompts me to write a test for calling undo on an empty editor as well.)

But behold. Now I clearly have a useful editor-ish component! I can add things to it, and I can undo edits. Awsome.

8 Naming the tests

What to do next? This might be a good time to show my todo-list so far. It contains all requirements mentioned at the beginning, things I've discovered underway, and the technical issues that have surfaced.

x Add characters to a "document"
x undo empty
x undo with only one edit
x undo with multiple edits
x After performing any edit, the user should be able to undo the last edit.
- clearing or resetting the editor, and that this can be undone.
- Report from test
- Delete characters from the "document"
- The user also should have the ability to regret undoing, by performing a "redo".
- Duplication w.r.t creating tests.
- The capability to perform a copy and paste will be very nice to have.
- Optimize storage for undo-stack.
- Duplication when adding tests to the sut

The number of tests are increasing, so I choose to go for the reporting mechanism. I could add a name together with the function delegate in the constructor. This adds to the amount of things needed to be done in order to set up a test, but I decide that knowing what test is failing is worth that extra cost, at least temporary - I'll get back to this later when resolving the todo about duplication when creating tests. Here is the changes for the edit required in order to add a name for each test:

banach:Discovering rolvseehuus$ hg diff editor.d 
diff -r 56b27d29e4be editor.d
--- a/editor.d  Mon Jul 25 12:10:03 2011 +0200
+++ b/editor.d  Mon Jul 25 14:36:02 2011 +0200
@@ -27,11 +27,11 @@
 class EditorTest{

   this(){
-    functions = [ delegate(){ test_ed_is_empty_after_initialization();},
-                 delegate(){ test_ed_contains_as_many_characters_added_after_a_write();},
-                 delegate(){ test_undoing_one_edit_is_equivalent_with_resetting_the_state();},
-                 delegate(){ test_undo_after_multiple_edits();},
-                 delegate(){ test_undo_on_empty_does_not_raise_an_exception();}];
+    functions = [ "test_ed_is_empty_after_initialization" : delegate(){ test_ed_is_empty_after_initialization();},
+                 "test_ed_contains_as_many_characters_added_after_a_write": delegate(){ test_ed_contains_as_many_characters_added_after_a_write();},
+                 "test_undoing_one_edit_is_equivalent_with_resetting_the_state": delegate(){ test_undoing_one_edit_is_equivalent_with_resetting_the_state();},
+                 "test_undo_after_multiple_edits": delegate(){ test_undo_after_multiple_edits();},
+                 "test_undo_on_empty_does_not_raise_an_exception":delegate(){ test_undo_on_empty_does_not_raise_an_exception();}];
   }

   void setUp(){
@@ -42,7 +42,7 @@
     int passedCount = 0;
     int failedCount = 0;

-    foreach(func; functions){
+    foreach(name, func; functions){
       bool passed = true;
       try{
        setUp();
@@ -98,7 +98,7 @@
   }

   private Editor ed;
-  private void delegate()[] functions;
+  private void delegate()[string] functions;
 }

Even though changing to an associative array in this case was incredibly simple, I cringed a little bit. This is a point where I really would have liked to have a test for the runTests function. I add this to the todo-list. Observing a test failing is still test good enough for me though.

I implement the reporting mechanism in runTests just by collecting a list of names for tests that fail, and write a report on it afterwards. The new runTests function looks like this:

void runTests(){
  string[] failedTests;
  foreach(name, func; functions){
    bool passed = true;
    try{
      setUp();
      func();
    }catch(Throwable e){
      passed=false;
      failedTests ~= name;
    }
    finally{
      passed ? writef(".") : writef("F");
    }
  }
  writeln("\n\nExecuted ", functions.length, " tests\n");
  if( !failedTests.length ){
    writeln("All tests executed successfully!");
  }else{
    writeln("There were errors!\n");
    foreach(fail; failedTests){
      writeln("Failed test: " ~= fail);
    }
    writeln("");
  }
}

I added some formatting, and the resulting code - with formatting and test naming - is not any bigger than the previous version in terms of lines of code. A successful run looks like the following:

banach:Discovering rolvseehuus$ dmd -unittest editor.d && ./editor 
.....

Executed 5 tests

All tests executed successfully!
banach:Discovering rolvseehuus$ 

Injecting an error looks like this:

banach:Discovering rolvseehuus$ dmd -unittest editor.d && ./editor 
F....

Executed 5 tests

There were errors!

Failed test: test_ed_is_empty_after_initialization

banach:Discovering rolvseehuus$ 

If you think the report look a bit like if it was inspired by Python's test framework, you are indeed right - I liked it a lot when I did Python programming for a living.

9 Redo!

After having working tests, and a testing system that tells me what test fails if any, I feel confident enough to add the next feature. I choose Redo. How should Redo look? Redo should only work after performing an undo (A clarification was needed with the customer. E.g. me.) So redo after writing, is a null-op. Also, writing something to the document clears the redo-stack. I add two cases to my todo-list: Redo after a plain edit does nothing to the document and checking that redo works after doing several undos (being psychic, I know that I can easily fake that redo after one undo, while multiple undos/redos are more difficult to fake.) I quickly implement the test for no change with redo after edit - it is almost to easy to write, but I'm guessing it will be a nice hedge later on when I implement the real redo functionality.

Implementing the test for multiple undos/redos is trivial: Add several edits, then undo all and redo all. The internal document after redoing should then match the one we had after adding the edits.

void test_redo_with_undos(){
  string[] edits = ["one", "two", "three"];
  string document;
  foreach(edit; edits){
    document ~= edit;
    ed.write(edit);
  }

  foreach(count; 0..edits.length){
    ed.undo();
  }
  foreach(count; 0..edits.length){
    ed.redo();
  }
  assert(ed.getDocument() == document );
}

The test fails predictably. (Phew - I remembered the tedious scaffolding this time.) Implementing by pushing the items popped while undoing over to a redo stack, and then push these back when redoing will work nicely. My first obvious implementation make the test pass, but it makes the previously test for redos without any undos fail. (I somewhat expected that, nice.)

It turned out that it was because an exception was rised when I tried to pop the back from an empty array again. There is no way to separate a regular failure from an error like this in the current testing framework… I make a note of that in my todo-list.

The undo/redo functions now looks like this (I renamed the history member variable to undoStack, as it paired up nicely with the redoStack name. This communicates intent nicely.)

void undo(){
  if(undoStack.length > 0){
    redoStack ~= document;
    document = undoStack.back();
    undoStack.popBack();
  }
}
void redo(){
  if(redoStack.length > 0){
    undoStack ~= document;
    document = redoStack.back();
    redoStack.popBack();
  }
}

I noteice that these two functions looks almost identical, and add undo/redo duplication to my todo-list.

I also notice some more duplication in my unittests. Both the redo-tests was cut-and-paste jobs, and I copied from testundoaftermultipleedits. The duplications I'm thinking of can be seen in the following snippet:

 void test_undo_after_multiple_edits(){
   string[] edits = ["one", "two", "three"];
   string document;
   foreach(edit; edits){
     document ~= edit;
     ed.write(edit);
   }
   ...
 void test_redo_without_undos(){
   string[] edits = ["one", "two", "three"];
   string document;
   foreach(edit; edits){
     document ~= edit;
     ed.write(edit);
   }
   ...
void test_redo_with_undos(){
   string[] edits = ["one", "two", "three"];
   string document;
   foreach(edit; edits){
     document ~= edit;
     ed.write(edit);
   }
   ...

I forge ahead and eliminate the duplication by wrapping multiple consecutive edits in one function with a name that communicates it's intent, reducing the total line-count from 180 to 172:

class EditorTest {
  ...
  string addEditsToEditor(string[] edits){
    string document;
    foreach(edit; edits){
      document ~= edit;
      ed.write(edit);
    }
    return document;
  }
  ...
}

Out of convenience I allow it to return the expected document after concatenating (that squeamish feeling again… My function does two things, and the "and return what you should expect from it" intention isn't communicated in the name. I hope to get back to that later.)

Reading over my todo-list, I notice the point I wrote about clearing the redo-stack after an edit. At this point I was not 100% sure how it would behave but suspected that it wouldn't work (a new display of my very short-lived short-term memory) - and the following test confirmed my suspicion.

void test_edit_after_undoing_reset_redo_state(){
  addEditsToEditor(["one", "two", "three"]);
  ed.undo();
  ed.write("Unrelated");
  auto document = ed.getDocument();
  ed.redo();
  assert(ed.getDocument() == document);
}

Clearing the redo-stack in the write-function fixed it. And that completes the redo-functionality. I feel a surge of accomplishment due to traction!

10 Fixing the painful scaffolding

I really miss the convenience of just adding a function with a name that start with test, in order to add a test to the test case - I keep forgetting to set up the name-function-map. Lets just have a look at the constructor of my testcase:

class EditorTest:
  ...
  this(){
    functions = [ "test_ed_is_empty_after_initialization" : delegate(){ test_ed_is_empty_after_initialization();},
                  "test_ed_contains_as_many_characters_added_after_a_write": delegate(){ test_ed_contains_as_many_characters_added_after_a_write();},
                  "test_undoing_one_edit_is_equivalent_with_resetting_the_state": delegate(){ test_undoing_one_edit_is_equivalent_with_resetting_the_state();},
                  "test_undo_after_multiple_edits": delegate(){ test_undo_after_multiple_edits();},
                  "test_undo_on_empty_does_not_raise_an_exception":delegate(){ test_undo_on_empty_does_not_raise_an_exception();}, 
                  "test_redo_without_undos":delegate(){test_redo_without_undos();}, 
                  "test_redo_with_undos":delegate(){test_redo_with_undos();}, 
                  "test_edit_after_undoing_reset_redo_state":delegate(){test_edit_after_undoing_reset_redo_state();}];
  }

That bunch of duplicated names and functions are clearly a hog in the machinery. It also looks like something that could be generated automatically.

In other modern languages, one might at this point reach for a reflection mechanism in order to go through the functions on the testcase object, and then execute every function that starts with the name "test" between a setUp and tearDown pair. Due to the lack of reflection on runtime objects in D, we cannot follow this procedure, but we CAN do something I've blogged about previously: Utilize the very capable code-generating and compile-time reflection mechanisms in D in order to build this map for us. Also, the responsibilities for EditorTest have grown to function as both test runner and test case implementation - the name doesn't really reveal what the class is doing. I don't feel like naming it EditorTestAndTestRunner to fix that. It seems that I could separate these responsibilities by extracting the test runner from EditorTest.

I start with separating the testcase from the testrunner. Poking at the code a bit, I realized that I could change my constructor to a function, and use it from the TestRunner to query about what test-functions it contains. I also realized I needed a TestCase abstraction in order to make the setUp function available to the runner. The resulting code for the TestRunner together with the relevant parts of my test and the unittest-block itself looks like follows.

class TestCase {
  public abstract void setUp();
}

class TestRunner {
  this(TestCase tCase){
    testCase = tCase;
  }

  void addTestFunction(string name, void function(Object o) func){
    testFunctions[name] = func;
  }

  void runTests(){
    string[] failedTests;
    foreach(name, func; testFunctions){
      bool passed = true;
      try{
        testCase.setUp();
        func(testCase);
      }catch(Throwable e){
        passed=false;
        failedTests ~= name;
      }
      finally{
        passed ? writef(".") : writef("F");
      }
    }
    writeln("\n\nExecuted ", testFunctions.length, " tests\n");
    if( !failedTests.length ){
      writeln("All tests executed successfully!");
    }else{
      writeln("There were errors!\n");
      foreach(fail; failedTests){
        writeln("Failed test: " ~= fail);
      }
      writeln("");
    }
  }

  private void function(Object)[string] testFunctions;
  private TestCase testCase;
}


class EditorTest : TestCase {

  // This was the constructor, now a function that takes a TestRunner
  // as an argument
  void collectTests(TestRunner tr){
    tr.addTestFunction("test_ed_is_empty_after_initialization", 
                    function(Object o){ (cast(EditorTest)o).test_ed_is_empty_after_initialization();});

    tr.addTestFunction("test_ed_contains_as_many_characters_added_after_a_write", 
                    function(Object o){ (cast(EditorTest)o).test_ed_contains_as_many_characters_added_after_a_write();});

    tr.addTestFunction("test_undoing_one_edit_is_equivalent_with_resetting_the_state", 
                    function(Object o){ (cast(EditorTest)o).test_undoing_one_edit_is_equivalent_with_resetting_the_state();});
    tr.addTestFunction("test_undo_after_multiple_edits",
                    function(Object o){ (cast(EditorTest)o).test_undo_after_multiple_edits();});

    tr.addTestFunction("test_undo_on_empty_does_not_raise_an_exception",
                    function(Object o){ (cast(EditorTest)o).test_undo_on_empty_does_not_raise_an_exception();});
    tr.addTestFunction("test_redo_without_undos",
                    function(Object o){ (cast(EditorTest)o).test_redo_without_undos();});
    tr.addTestFunction("test_redo_with_undos",
                    function(Object o){ (cast(EditorTest)o).test_redo_with_undos();});
    tr.addTestFunction("test_edit_after_undoing_reset_redo_state",
                    function(Object o){ (cast(EditorTest)o).test_edit_after_undoing_reset_redo_state();});
  }
  ...


unittest {
  EditorTest fix = new EditorTest;
  TestRunner runner = new TestRunner(fix);
  fix.collectTests(runner);
  runner.runTests();
}

The testrunner can only run the tests for one test case. This is clearly limiting. I've already mentioned the need for a test of our test-runner. I't seems like I can easily add that test now, save for the urge I have to have multiple test-cases in one runner. But the refactoring I need to do in order to support several testcases in the runner need to test that the runner works. A catch 22 of sorts. In order to "bootstrap" testing of the TestRunner, I add the test to EditorTest and plan to separate the tests when I have support for multiple testcases later on (I add multiple testcases to the todolist.)

Also, notice how the testcase could lend a hand and allow it self to be added to the TestRunner, if it defined the collectTests function on EditorTest. I choose to utilize that when adding the test of the TestRunner.

After forcing the test to fail, and implementing it, the test looks like this, with inlined test case and all.

void test_test_runner_calls_test(){
  string callSignature = "callMe called";
  class LoggingTestCase : TestCase {
    void test_callMe(){
      log ~= callSignature;
    }

    void collectTests(TestRunner runner){
      runner.addTestFunction("callMe",
                             function(Object o){(cast(LoggingTestCase)o).test_callMe();});
    }
    public string log;
  }

  LoggingTestCase tcase = new LoggingTestCase;
  TestRunner runner = new TestRunner(tcase);
  tcase.collectTests(runner);
  runner.runTests();
  assert(tcase.log == callSignature);
}

The test passes, I have my self-referential test, phew. But look - the output from my testrunner suddenly is a bit gibberishy:

banach:Discovering rolvseehuus$ dmd -unittest editor.d && ./editor 
.........

Executed 1 tests

All tests executed successfully!
.

Executed 9 tests

All tests executed successfully!
banach:Discovering rolvseehuus$ 

The output from the extra testrunner created and executed, are folded into the output of the "real" testrunner and clutters the communication of the reporting mechanism. I would like the testrunner to have some kind of "silent" mode, or maybe I can create a modification point for how it generates it's reports. As this is a matter of cosmetics, and considering that I'm very eager to roll the function map compile-time, I'm adding a note on the testrunner output on my todo-list.

11 Automatically generating the test-list

Now then, back to automatically generating the test-functions. How will I test this? I can create a testcase with a test, call the magic function that adds the test-functions, and check if it's called by the test-runner. This sounds like a little extension of the test of the test-runner, doesn't it? After adding the appropriate forward-declared mixin template I'll use to roll the test function map, it looks like this - a cut-and-paste job from testtestrunnercallstest.

...
mixin template TestBuilder(T){
  void collectTests(TestRunner tr){
  }
}
...
class EditorTest : TestCase {
  ...
  void test_test_creator_creates_called_test(){
    string callSignature = "callMe called";
    class LoggingTestCase : TestCase {
      mixin TestBuilder!LoggingTestCase;

      void test_callMe(){
        log ~= callSignature;
      }
      public string log;
    }

    LoggingTestCase tcase = new LoggingTestCase;
    TestRunner runner = new TestRunner(tcase);
    tcase.collectTests(runner);
    runner.runTests();
    assert(tcase.log == callSignature);
  }

The empty TestBuilder template is there only to make the things compile, and to implements the collectTests function we use to add the tests. As it is empty at the moment, the test fails as expected:

banach:Discovering rolvseehuus$ dmd -unittest editor.d && ./editor 
..

Executed 0 tests

All tests executed successfully!
F.......

Executed 1 tests

All tests executed successfully!
.

Executed 10 tests

There were errors!

Failed test: test_test_creator_creates_called_test

banach:Discovering rolvseehuus$ 

As previously noted, the junk is piling up when I'm having multiple test runners being executed during tests - but it is still obvious to me what test is failing as long as its only one.

Adapting the code to generate the test-function lookups turns out to be easy enough, and after creating a TestBuilder mixin as previously mentioned, the tests run okay. The code that allows me to automatically register and list the functions (adapted from this post looks like the following.

class TestEnlister(T, string prefix = "test"){
  static void collectTests(TestRunner tr){
    foreach(idx; 0..testNames.length){
      tr.addTestFunction(testNames[idx], testFunctions[idx]);
    }
  }

  static string buildStaticFunctionWrappers(string[] names){
    string res = "";
    foreach(name; names){
      res ~= "static " ~ name ~ "(Object o){ (cast(T)o)." ~name ~ "();}\n";
    }
    res ~= "static void function(Object)[] testFunctions = [";
    foreach(name; names){
      res ~= "&"~name~",";
    }
    res ~="];";
    return res;
  }

  static string[] enlistTestFunctions(){
    string[] res;
    foreach(fun; __traits(allMembers, T)){
      if(fun.indexOf(prefix) == 0){
        res ~= fun;
      }
    }
    return res;
  }

  mixin(buildStaticFunctionWrappers(enlistTestFunctions()));
  static string[] testNames = enlistTestFunctions();
}

mixin template TestBuilder(T){
  void collectTests(TestRunner tr){
    TestEnlister!(T).collectTests(tr);
  }
}

That means I can simply replace the collectTests function on the original test-case with the TestBuilder mixin, and everything should look like before. I replace 10-15 lines of code with one:

class EditorTest : TestCase {
  mixin TestBuilder!EditorTest;
  ...tests...
}

Now there is one simple item I'm not entirely happy with - I need to tag every test-class with the mixin template, thus needing to write the class name twice. I think I can live with that for the time being, so I don't bother adding that to my todo-list. I note though that "testtestrunnercallstest" and "testtestcreatorcreatescalledtest" are identical, save for the compile-time generated table so I remove the superfluous test.

But finally! I've removed the need to add a new obscure line of code every time I define a new test in the TestCase. I'm of course seriously noticing that we are on the brink of having something useful here: A nice little easy-to-use test library. Its being born from paying attention to duplication and communicating intent. The last todos on my list regarding the test-framework is enough for it to make the final leap over to the realm of usefulness. I add a todo on moving it out in a separate file.

12 Back on track

It's time to get cranking with some production code again - we've been refactoring our test-system for a long time. For reference, here is my current todo-list:

TODO:
x Add characters to a "document"
x undo empty
x undo with only one edit
x undo with multiple edits
x After performing any edit, the user should be able to undo the last edit.
x Redo without undo
x Redo all edits, after undoing all edits.
x Report from test
x Doing something else than a undo, resets redo-state
x The user also should have the ability to regret undoing, by performing a "redo".
x Test for runTests function
x Duplication w.r.t creating tests.
x Triplication! when adding test to the sut
- Multiple testcases in one test-runner.
- clearing or resetting the editor, and that this can be undone.
- Delete characters from the "document"
- The capability to perform a copy and paste will be very nice to have.
- Optimize storage for undo-stack.
- Needs to be able to modify the output from the TestRunner.
- Need to discern between failures and exceptions thrown during
  execution in test system.
- Move test-library to separate unit.

Deleting characters from the document looks like a feature that will bring me the feeling of progress I crave right now, and it might also be usable when creating copy and paste later on. I suspect cutting and then pasting also should be a thought, so we should allow the user to store the text deleted from the document. Something that looks like this would accomodate these needs:

void test_cutting_a_region_of_text(){
  string prefix = "This is a ";
  string suffix = " text";
  string infix = "DELETEME";

  addEditsToEditor([prefix, infix, suffix]);

  string res = ed.cutRange(prefix.length, infix.length);

  assert(res==infix);
  assert(ed.getDocument() == prefix ~ suffix);
}

In other word, I choose to specify from where to begin the cut, and how many characters to cut from the document.

This time, the new (failing, of course) test is automatically picked up and executed by the runner due to the code generation! Oh the joy. I'm feeling a deep sense of closure. During my little celebration I add a todo on handling out-of-bounds ranges when cutting.

I also may note that having two asserts in this test turned out to be a somewhat bad idea. I worked with one assert commented out for an extended period to have a grip on what was going on - shame on me! This might as well have been two separate tests - one for the returned result, and one for the resulting document. I guess I just got lazy on that one.

After the test run, I realize that the cutting test should be undoable as well. I add a failing test to demonstrate this behavior:

void test_cutting_a_region_of_text_can_be_undone(){
  string document = "The text Im adding to the editor is this";
  addEditsToEditor([document]);
  ed.cutRange(document.length/4, document.length/2);
  ed.undo();
  assert(ed.getDocument() == document);
}

Its easy to fix the cut-function, I just needed to push the document on the undo stack. After the fix it now looks like the following - I include the write function for a comparison.

class Editor {
  ...
  void write(string edit){
    undoStack ~= document;
    document ~= edit;
    redoStack = [];
  }
  ...
  string cutRange(size_t pos, size_t num){
    undoStack ~= document;
    string sub = document[pos..pos+num];
    document.replaceInPlace(pos, pos+num, "");
    return sub;
  }
  ...

Pushing the undo stack seems to be something I would like to do every time I edit the string, and the next feature - pasting a copied or cut range - will need the same thing. This leads to some duplication, can I resolve that in some way? I choose not to ponder that to much right now, and note my concerns in the todo-list (annotating the functions with push and pop aspects seems too easy.) Also, I decide that out-of-range as handled by the array-range suffice (it throws an exception), but still adds a test for it now that it has become so easy to add them. It passes - no surprise really… We can get more specific on the type of exception later on if I need it.

void test_cutting_a_region_of_text_out_of_range_throws(){
  string document = "The text Im adding to the editor is this";
  addEditsToEditor([document]);
  assertThrown!Throwable(ed.cutRange(document.length, 10));
}

13 Cleaning up the test reports

At this point I would like to remove the clutter when running a successful set of tests - that extra output from testing the testrunner. If the TestRunner got a report generator as an argument to it's constructor, to which it reported successes and failures, I could replace it with a null-op report generator and get a test-runner without the cluttered output. I add a TestReport interface with the abstract functions reportSuccess, reportFailure and generateReport and implement a default implementation that reproduces the current output.

class DefaultTestReport : TestReport {
  void reportSuccess(string name){
    testCount++;
    writef(".");
  }
  void reportFailure(string name){
    testCount++;
    writef("F");
    failedTests ~= name;
  }

  void generateReport(){
    writeln("\n\nExecuted ", testCount, " tests\n");
    if( !failedTests.length ){
      writeln("All tests executed successfully!");
    }else{
      writeln("There were errors!\n");
      foreach(fail; failedTests){
        writeln("Failed test: " ~= fail);
      }
      writeln("");
    }
  }

  private size_t testCount;
  private string[] failedTests;
}

class TestRunner {
  this(TestCase tCase){
    testCase = tCase;
    report = new DefaultTestReport;
  }
  ...
  void runTests(){
    foreach(name, func; testFunctions){
      bool passed = true;
      try{
        testCase.setUp();
        func(testCase);
      }catch(Throwable e){
        passed=false;
      }
      finally{
        passed ? report.reportSuccess(name) : report.reportFailure(name);
      }
    }
    report.generateReport();
  }
  ...

The runTests function became much cleaner, and the output is identical (as confirmed by visual inspection - that's a test too, no need to get all fanatic about this.) Now I can add a constructor to the TestRunner that takes a TestReport as an argument. I create a TestReport that just swallows the extra output, and use this in the tests that uses TestRunner. Here is the new improved output:

banach:Discovering rolvseehuus$ dmd -unittest editor.d && ./editor 
............

Executed 12 tests

All tests executed successfully!

Cleaning up the things I'm staring at all the time gives me a great sense of relief. This has been annoying me for such a long time now…

14 Support for multiple testcases in one TestRunner

At this point the testcase for the Editor is getting really big - and it still does a bunch of things that doesn't belong there - the name of the test communicate that it is the test for the editor, but it also tests the testrunner. I want to separate the two test-cases, so I add the test for multiple tests first so I can perform the separation safely.

void test_test_runner_can_handle_multiple_tests()
{
  class LoggingTestCase : TestCase {
    mixin TestBuilder!LoggingTestCase;

    this(string logmsg){
      logMessage = logmsg;
    }

    void test_callMe(){
      log ~= logMessage;
    }

    string logMessage;
    string log;
  }

  LoggingTestCase cOne = new LoggingTestCase("This is one test");
  LoggingTestCase cTwo = new LoggingTestCase("This is another test");
  TestRunner runner = new TestRunner(new EmptyTestReport);

  runner.addTest(cOne);
  runner.addTest(cTwo);
  runner.runTests();

  assert(cOne.log == cOne.logMessage);
  assert(cTwo.log == cTwo.logMessage);
}

It fails as expected of course, after adding the necessary parts to make it compile. Also, I notice that the collectTests function I created earlier, just in order to keep that functionality within the test-case, now proves to be very useful in the general case - I could call that, after adding a test case to the TestRunner. I just need to make it available in the TestCase abstract class!

I'm struggling a bit with the design choices here, though. Adding the collectTests-functions to TestCase introduces a circular dependency between TestCase and TestRunner. As TestCase is abstract and only an interface so far, I think I don't mind that too much though. And if I keep TestCase and TestRunner in the same file, it wouldn't pose any problems later on (and it can easily be yanked apart later by creating an additional TestCollector for that functionality.) Poking around a bit in the TestRunner though, makes me realize that I need something to keep track of what test-functions belongs to what TestCase. The TestCollector step forth and make itself a necessity even before the circular dependency between TestCase and TestRunner became established. Nice.

I perform a refactoring (in the green, after disabling the failing test to make the TDD-police happy) in two steps: Introduce the TestCollector and use it instead of the TestRunner for collecting tests, then add an internal class to the TestRunner that implements the TestCollector and collects the testfunctions with the testcases together in an object. Adding these to a container with the testcase-testfunction mappings, make my new test pass with flying colors.

class TestRunner {

  class TestCaseFunctions : TestCollector { 
    this(TestCase tc){
      tCase = tc;
    }

    void addTestFunction(string name, void function(Object o) func){
      testFunctions[name] = func;
    }

    public TestCase tCase;
    public void function(Object)[string] testFunctions;
  };

  this(TestReport treport){
    report = treport;
  }

  this(){
    report = new DefaultTestReport;
  }

  void addTest(TestCase tCase){
    TestCaseFunctions fns = new TestCaseFunctions(tCase);
    tCase.collectTests(fns);
    testCaseFunctions ~= fns;
  }

  void runTests(){
    foreach(tc; testCaseFunctions){
      foreach(name, func; tc.testFunctions){
        bool passed = true;
        try{
          tc.tCase.setUp();
          func(tc.tCase);
        }catch(Throwable e){
          passed=false;
        }
        finally{
          passed ? report.reportSuccess(name) : report.reportFailure(name);
        }
      }
    }
    report.generateReport();
  }

  private TestCaseFunctions[] testCaseFunctions;                   
  private TestReport report;
}

I don't like the double loop first over testCaseFunctions, and then over individual testfunctions. And it hits me like a ton of bricks: I've been wasting my time big-time. Feeling a little bit stupid, I move the inner loop in to the TestCaseFunctions, and add a runTests function to it - basically completing the extract class refactoring by coming full circle back to having something that looks exactly like the original One-test-case-only TestRunner for each test. There is one part that is different though - I send the reporter as an argument to the runTests function on the TestCase. I could have gotten to the same point with fewer edits, just by renaming TestRunner to TestCaseFunctions (I don't really like that name but I don't feel overly creative right now and add it to the todo-list instead), and created a new TestRunner that looks like the one below.

class TestCaseFunctions : TestCollector { 
  this(TestCase tc){
    tCase = tc;
  }

  void addTestFunction(string name, void function(Object o) func){
    testFunctions[name] = func;
  }

  void runTests(TestReport report){
    foreach(name, func; testFunctions){
      bool passed = true;
      try{
        tCase.setUp();
        func(tCase);
      }catch(Throwable e){
        passed=false;
      }
      finally{
        passed ? report.reportSuccess(name) : report.reportFailure(name);
      }
    }
  }
  public TestCase tCase;
  public void function(Object)[string] testFunctions;
}

class TestRunner {
  this(TestReport treport){
    report = treport;
  }

  this(){
    report = new DefaultTestReport;
  }

  void addTest(TestCase tCase){
    TestCaseFunctions fns = new TestCaseFunctions(tCase);
    tCase.collectTests(fns);
    testCaseFunctions ~= fns;
  }

  void runTests(){
    foreach(tc; testCaseFunctions){
      tc.runTests(report);
    }
    report.generateReport();
  }

  private TestCaseFunctions[] testCaseFunctions;                   
  private TestReport report;
}

Now that I have support for executing multiple testcases in the testrunner, I can separate the TestRunner's tests from the Editor's tests - the TestEditor and TestTestRunner names finally communicates their intent. Here is the resulting TestTestRunner:

class TestTestRunner : TestCase {
  mixin TestBuilder!TestTestRunner;

  class EmptyTestReport : TestReport {
    void reportSuccess(string name){}
    void reportFailure(string name){}
    void generateReport(){}
  }

  void test_test_builder_creates_called_test(){

    string callSignature = "callMe called";
    class LoggingTestCase : TestCase {
      mixin TestBuilder!LoggingTestCase;
      void test_callMe(){
        log ~= callSignature;
      }
      public string log;
    }

    LoggingTestCase tcase = new LoggingTestCase;
    TestRunner runner = new TestRunner(new EmptyTestReport);
    runner.addTest(tcase);
    runner.runTests();
    assert(tcase.log == callSignature);
  }

  void test_test_runner_can_handle_multiple_tests()
  {
    class LoggingTestCase : TestCase {
      mixin TestBuilder!LoggingTestCase;

      this(string logmsg){
        logMessage = logmsg;
      }

      void test_callMe(){
        log ~= logMessage;
      }

      string logMessage;
      string log;
    }

    LoggingTestCase cOne = new LoggingTestCase("This is one test");
    LoggingTestCase cTwo = new LoggingTestCase("This is another test");
    TestRunner runner = new TestRunner(new EmptyTestReport);

    runner.addTest(cOne);
    runner.addTest(cTwo);
    runner.runTests();

    assert(cOne.log == cOne.logMessage);
    assert(cTwo.log == cTwo.logMessage);
  }
}

Its some more duplication here, for example the two implementations of LoggingTestCase, and the within-test creation of the testrunner with one test. I feel compelled to remove the duplication, and while removing the duplication I realize that the test for multiple tests renders the test for calling a test function superfluous. After removing the duplication and the superfluous test, the rest looks like this - substantially less code to wrap my head around (which is a good thing, considering my short term memory… )

class TestTestRunner : TestCase {
  mixin TestBuilder!TestTestRunner;

  class EmptyTestReport : TestReport {
    void reportSuccess(string name){}
    void reportFailure(string name){}
    void generateReport(){}
  }

  class LoggingTestCase : TestCase {
    mixin TestBuilder!LoggingTestCase;

    this(string logmsg){
      logMessage = logmsg;
    }

    void test_callMe(){
      log ~= logMessage;
    }

    string logMessage;
    string log;
  }

  void setUp(){
    runner = new TestRunner(new EmptyTestReport);
  }

  void test_test_runner_can_handle_multiple_tests()
  {    
    LoggingTestCase cOne = new LoggingTestCase("This is one test");
    LoggingTestCase cTwo = new LoggingTestCase("This is another test");

    runner.addTest(cOne);
    runner.addTest(cTwo);
    runner.runTests();

    assert(cOne.log == cOne.logMessage);
    assert(cTwo.log == cTwo.logMessage);
  }

  private TestRunner runner;
}

15 A self-tested unit testing module

The size of my little test-library has become a substantial portion of my file - so substantial that my editor-module primarily looks like a testing framework at first glance (obscuring the intent of the editor-module.) I decide that it's time to move it to a separate module.

So far we've been cruising along, and the refactorings have been relatively low risk, because all source is contained in one source-file that (at times at least) have been relatively easy to read. Also, adding a failing test every now and then has posed as a stand in for the test of the TestRunner with a failing test. There have been, though, a couple of times where I've had this "uh-oh, I jumped plane without a parachute" feeling - prompting me to deliberately break some tests to check for firm ground beneath my feet. In other words, before moving it out of the editor-file, I finally need to ad the test that verify that a failing test behaves correctly. This will make the testing framework self contained, self tested and reusable.

Looking for a way to implement the failing test, I grab hold of the test-report - it can function as a mock (now how useful didn't that turn out to be?) A successful test will be followed by a reportSuccess-call, and a failing test will be followed by a reportFailure call. By making a TestReport that just log it's successes and failures, I can check that the report is correct in face of a failing test.

void test_a_failing_test(){
  class LoggingTestReport : TestReport {
    this(){}
    void reportSuccess(string testname){
      successes ~= testname;
    }
    void reportFailure(string testname){
      failures ~= testname;
    }

    void generateReport(){}

    string failures;
    string successes;
  }

  class CaseWithFailingTest : TestCase {
    mixin TestBuilder!CaseWithFailingTest;

    void test_failing_test(){
      assert(false);
    }

    void test_passing_test(){
    }
  }

  LoggingTestReport report = new LoggingTestReport;
  TestRunner trunner = new TestRunner(report);
  trunner.addTest( new CaseWithFailingTest);
  trunner.runTests();

  assert(report.failures == "test_failing_test");
  assert(report.successes == "test_passing_test");
}

Unsurprisingly, it works. But now I'm certain it will work in the future as well. I also notice that it doesn't quite fit in the TestTestRunner class - because it does not use the member variable yet (I had to create a new TestRunner in order to change the reporting object) That can wait - lets move this baby.

Moving the entire test-framework to the module dunit, removing the parts not needed by our editor and then importing it in the editor-file, takes two minutes. It does - again - change our output slightly though:

banach:Discovering rolvseehuus$ dmd -unittest editor.d dunit.d && ./editor 
...........

Executed 11 tests

All tests executed successfully!
..

Executed 2 tests

All tests executed successfully!
banach:Discovering rolvseehuus$ 

This change is caused by every unittest-block setting up and running a separate unittest. Originally I aimed for that single dotted line, with one dot for every test runned. If I wanted to keep it as is, it seems I would need to be able to register test-suites in some global test repository as part as the unittest-block - by all means doable. Choosing exactly how the output should look, is a matter of taste though, and I'm fine with keeping it this way for now - I don't have any customers telling me otherwise yet…

16 Was it worth it?

It's time to conclude this blog-post. The rest of the work will be one feature and the not-so-very-interesting cleanups and minor technicalities.

By paying attention to the four principles outlined at the beginning - writing tests that always run, paying attention to the intention of the pieces of the software and make sure it is communicated, removing duplication and seeking to keep the number of separate classes/functions and modules at a minimum - I've ended up with a useful reusable test-framework that I'm sure I'll use and evolve in the future. Without that being the plan at all - reusable useful bits and pieces emerge seemingly without a plan for them to do so, if you just follow the "rules" (okay - I admit it. I kind of knew that it would happen. But that's beside the point…)

I did two things with the tests: I added new tests, and I performed maintenance on the tests - that is, refactoring in order to meet the "rules" of simple design. And it was this maintenance that made the testing framework emerge. The maintenance I performed on the tests might, from reading this, seem like a big part of the development, and granted - it was. In hindsight - and I'm only guessing based on gut feel - It might amount to half of the total time spent coding. The rest, I expect was evenly divided between writing tests and writing the editor - slightly skewed towards writing tests.

Was it worth it? To keep track of and pick up those loose ends? If I were never to touch any of this code again, and nobody needed to perform maintenance on it - If this was a totally isolated development effort - then maybe I've wasted my and - worse - the customers time. But for most cases - I believe it would certainly be worth it. But that would fill another blog-post.

Finally, about the rear-view-only part. I wanted to write this while only paying attention to the next requirement and the source. For a small system like this, it is clearly a sub optimal strategy - some of the changes I did could clearly be anticipated with some forethought - building the needed designs from the start and thus I would have done less refactoring underway. But sometimes, working on a large-ish legacy system with history beyond the start-date of the most senior team member after changes in staffing during the life of the project, that's all you've got: The source, a window of knowledge about starts and ends that is very small compared to the entire life of the project, and the next handful of requirements. So pay attention to what happens in the rear view mirror as well.

17 The source for the final editor module

As I've presented you with segments here and there of the code, It is good form to display the final look of both the testing framework and the editor-module with it's unittest.

The testing framework can be found at bitbucket under the name dxunit (because dunit was taken for a Delphi framework.) The source for the editor is found below.

import std.array;
import std.exception;
import dunit;

class Editor {

  size_t size() {
    return document.length;
  }

  void write(string edit){
    undoStack ~= document;
    document ~= edit;
    redoStack = [];
  }

  void undo(){
    if(undoStack.length > 0){
      redoStack ~= document;
      document = undoStack.back();
      undoStack.popBack();
    }
  }

  void redo(){
    if(redoStack.length > 0){
      undoStack ~= document;
      document = redoStack.back();
      redoStack.popBack();
    }
  }

  string cutRange(size_t pos, size_t num){
    undoStack ~= document;
    string sub = document[pos..pos+num];
    document.replaceInPlace(pos, pos+num, "");
    return sub;
  }

  string getDocument(){
    return document;
  }

  private string document;
  private string[] undoStack;
  private string[] redoStack;
}


class EditorTest : TestCase {
  mixin TestBuilder!EditorTest;

  void setUp(){
    ed = new Editor;
  }

  string addEditsToEditor(string[] edits){
    string document;
    foreach(edit; edits){
      document ~= edit;
      ed.write(edit);
    }
    return document;
  }

  void test_ed_is_empty_after_initialization(){
    assert(ed.size() == 0);
  }

  void test_ed_contains_as_many_characters_added_after_a_write(){
    string text = addEditsToEditor(["some text we're adding"]);
    assert(ed.size() == text.length);
  }

  void test_undoing_one_edit_is_equivalent_with_resetting_the_state(){
    addEditsToEditor(["some text we're adding"]);
    ed.undo();
    assert(ed.size() == 0);
  }

  void test_undo_after_multiple_edits(){
    string document = addEditsToEditor(["one", "two", "three"]);
    ed.write("lasttoberemoved");
    ed.undo();

    assert(ed.getDocument() == document);
  }

  void test_undo_on_empty_does_not_raise_an_exception(){
    try{
      ed.undo();
    }catch(Throwable e){
      assert(false);
    }
  }

  void test_redo_without_undos(){
    string document = addEditsToEditor(["one", "two", "three"]);
    ed.redo();
    assert(ed.getDocument() == document);
  }

  void test_redo_with_undos(){
    string[] edits = ["one", "two", "three"];
    string document = addEditsToEditor(edits);

    foreach(count; 0..edits.length){
      ed.undo();
    }
    foreach(count; 0..edits.length){
      ed.redo();
    }
    assert(ed.getDocument() == document );
  }

  void test_edit_after_undoing_reset_redo_state(){
    addEditsToEditor(["one", "two", "three"]);
    ed.undo();
    ed.write("Unrelated");
    auto document = ed.getDocument();
    ed.redo();
    assert(ed.getDocument() == document);
  }

  void test_cutting_a_region_of_text(){
    string prefix = "This is a ";
    string suffix = " text";
    string infix = "DELETEME";

    addEditsToEditor([prefix, infix, suffix]);

    string res = ed.cutRange(prefix.length, infix.length);


    assert(ed.getDocument() == prefix ~ suffix);
    assert(res==infix);
  }

  void test_cutting_a_region_of_text_can_be_undone(){
    string document = "The text Im adding to the editor is this";
    addEditsToEditor([document]);
    ed.cutRange(document.length/4, document.length/2);
    ed.undo();
    assert(ed.getDocument() == document);
  }

  void test_cutting_a_region_of_text_out_of_range_throws(){
    string document = "The text Im adding to the editor is this";
    addEditsToEditor([document]);
    assertThrown!Throwable(ed.cutRange(document.length, 10));
  }

  private Editor ed;
}

unittest {
  TestRunner runner = new TestRunner;
  runner.addTest(new EditorTest);
  runner.runTests();
}