12. Is Scrum "GTD for teams?"

I've been using GTD for ten years plus, after a then colleague of mine sold me on the idea. He was very exited about David Allens book. I've been doing Agile in some way or another from before that after going all in on eXtreme Programming with a startup, and Scrum have been the go-to-process of choice for the last six years or so. A couple of weeks ago, I heard a pod-cast (I think it was with Daniel Pink) where a part of the show touched upon the similarity between Scrum and GTD. One of the statements made in that podcast was that "Scrum is GTD for teams." When listening, I agreed that this comparison was not entirely without merit. I've come to appreciate the similarities between them, and I pondered a similar idea my self a couple of years ago. This is a short stab at what's similar and what's different. Is Scrum really GTD for teams?

Scrum (as do other agile processes, such as eXtreme Programming) split the problem space of software development in to Stories, and device a plan for how to prioritize and track the translation of the Stories to actual software artefacts. GTD focuses on translating "stuff" that's coming your way in to Projects and next actions, and the efficient management of these through usage of contexts.

A Story maps almost directly to a Project. A story is a tiny description of a user interaction of the system, with a clear definition of the What and Why, and a description of the goal state through a recipe for how to demonstrate the complete story. A simplified example may be "User press the Stroke button in order to draw lines with strokes", where this should be demonstrated by performing a drawing action that does not cause strokes unless the Strokes-button is indeed pressed. In a similar way, GTD encourage you to think first about "Whats the Outcome, and what does doing look like" and then define your actions in terms of things you need to do in order to reach done. A Project in GTD-jargon is any goal-state that needs more than one physical action. In this regard, you may say that "Research BitCoin" may be an ill formed project. "Research what economists in general thinks about BitCoin" however, is better. "Research ten renowned economists opinions of BitCoin" however, clearly states what needs to be done much in the same way what a Scrum Story does. In some sense, Scrum and GTD have converged to very similar formats  for the actionable items. At least ideally, if not in form.

Both GTD and Scrum defines a kind of "morning ritual". Both asks you to figure out and state somewhat clearly what your goals for the day is. The GTD morning ritual is to look over the calendar for today, find the next actions you need to complete and perhaps group them in a "today" list. The Scrum morning ritual is to gather around the Scrum-board, state for each other what goals one have for the day and perhaps grab new work if one's done with the previous task.

But this is where the similarities ends. Scrum defines, in addition, a process, where you first plan a "sprint" lasting one, two, three or four weeks. The goal is to first plan the sprint, by building a "Sprint backlog". The sprint backlog contains all stories to complete with estimates. The stories are perhaps broken down in more details - tasks needed to build the stories. Progress is measured relatively to the sprint backlog. One goal is to be able to fill the sprint with exactly the amount of work one can complete within the Sprint. The Sprint Backlog is considered "holy", and should not be touched during the sprint. The items on the Sprint Backlog is then tracked and "burned down" towards completion during the execution of the sprint. Then there is an end-of-sprint ritual to evaluate its execution. Lather, rince, repeat.

GTD do not come with such a process attached. However, GTD advocates a weekly review, where one clear the deck, look over all open loops, check the calendar for the last two weeks and the next two weeks. This is done in order to be able to see if there are any required actions that one have failed to record any where. something crops up. It may look like a planning and retrospect meeting, but it lacks the concept of estimation and "locking in" any target scope of work. GTD recognized that all your priorities may change during a two minutes chat beside the water-cooler, or when your spouse call you and tell you she have crashed the car. Reacting on such priroity-changing events in Scrum, requires an act of "terminating the sprint" and starting over with a new planning session.

With the last paragraph in mind, the GTD process looks more like a personal Kanban process. It is a more lightweight process than Scrum. Kanban focuses on continuous evolutionary change and how work items flow through the organization from concept to finished feature. The concept of a sprint is gone, so is the concept of a Sprint Backlog. There is only the product backlog, in addition to issues that crop up while one move forward together. The center of a Kanban process is of course the Kanban, maintaining a living representation of what work is actually being done at any time. Spending some time every week reflecting over the work being done is still a good idea. So is standups.

Another thing that distinguishes GTD from Scrum, is that GTD moves beyond the mere execution of a plan. GTD defines some useful thinking tools, the different levels of focus, to guide how projects and actions are prioritized. Areas of focus and responsibility, short term goals, visions and intentions, and finally purpose and principles - all these levels of thinking unite in a constructive way to reflect around you and your life's direction. A team working towards delivering a product need to do the same, and Scrum provides very little guidance besides giving the key stakeholder a name.

In conclusion then. I'd say that the execution-part of GTD bares more resemblance to Kanban, than it does to Scrum. So one may say Kanban is GTD for teams. Or GTD is a one-man-Kanban. But this comparison on the execution-level only leave out the deeper more life-encompassing aspects of GTD. GTD is a different beast all together.

11. It's not the resources, it's the priorities.

"We don't have enough resources" - if I had an euro for every time I heard that statement, I would be rich. I've heard the sentence used everywhere, from the mouth of a one-man-team, and from leading developers in a seemingly "unlimited resources" situation. And every time, it kind of strikes me how... wrong... the statement is.

I remember participating in a project meeting once with our then most important customer. From the customer side, it included the main stakeholder - our key sponsor - and two of his colleagues, who participated in QA and follow up activities. From our part, it included the Project Manager for the project, who was a new hire, the CEO of our company, a developer who had participated in meeting this customer before, and me. It was the first time I visited, and I was quite new on the team. Due to several factors, such as key people moving on, lack of communication and a couple of very buggy binaries sent to the client, trust was suboptimal to say the least.

The meeting spanned three days. The first day was "get to know each other day". The second day was more of a feature discussion day, which turned out to be the day where the customer spent most of their time piling on new stuff they wanted to have delivered within the scope of the contract.

Ah - the contract. I need to talk about the contract. From the start, it was almost the agilists dream contract. It was fixed price, with a list of perhaps 20 or so vaguely described features they wanted, delivered over four increments. The contract had fixed seemingly non negotiable delivery-dates for each phase. We had delivered phase three, and the customer was still not happy with what we had delivered for phase two. So some place along the line we had failed to match - or control - the expectations of the customer. Fair enough. We were sitting there, we had not closed phase 2, we wanted to close phase 3 and we wanted to get started on the last phase. To add insult to injury, we had spent more time than scheduled on phase 3 - we didn't have the budget to complete phase 4 without taking a monetary loss (not to mention finishing it on time.) And still, the customer was piling on changes and fixes on features delivered for phase 1, 2 and 3.

I can rant on about what went wrong with that project, but I don't think that would be the interesting part. The interesting part is how we got out of it. We simply did a variation on the planning game. New and exiting for everybody but me.

On the evening of the second day, me and the PM sat down, wrote all requirements we had gathered during those two first days on A4-sized printer paper (For you purists: sorry - no index-cards in the grocery-store on the corner of our hotel.) One requirement on one piece of paper. There was even room for drawings. I committed a grave sin that day, by singlehandedly giving StoryPoint estimates on every item in the list (the PM was not able to participate.) And voila, we had a backlog.

The next day then turned out to be the planning and prioritization day. We started by presenting the key sponsor and his entourage with the following:

• Acknowledged that we had a fixed deadline
• Put forward the pile of paper the backlog
• Stated the sum of the StoryPoints (we called them apples) and how they came about
• That given the current velocity of the team, we could complete a third of the backlog within the deadline for the project (our team had an established velocity, and we used this to extrapolate under the assumption that my estimates was ballpark-ish.)

I must admit, that the first part of that meeting was difficult. But to cut the story short - when presented with what he asked for this way, the sponsor was able to agree to participate in prioritization of the backlog, remove two thirds of it's content, and negotiate a realistic scope for the deadline. On which we delivered successfully. Trust was reestablished, and we got a new contract with them.

Lack of resources was not an issue in the discussion. Priorities, on the other hand, was. Priorities, and clearly communicating what priorities the stakeholders have to make. What to keep, and what to remove. When this is communicated clearly, the resource-question solves it self. Either the project get's the amount of resources it needs, or the project gets shaffed. Both are wins, in some sense. In our case, it turned out to be more resources (the next contract.)

10. If it works, fix it anyway

I remember a day where I'd spent a couple of hours understanding a piece of code. There was no tests for this code, and it was hard to read. When I finally understood it, I thought of a way to change the design of these parts so that it would be easier both to test and to read for the next maintainer coming along. I aired my opinion to one of the teammates, and you have probably already guessed what he said.

"Why change that? It works. Don't fix it if it works."

And in some sense, that's a good advice. It's a good strategy for risk-mitigation, and my hunch is that it's correct 80% of the time. But in this particular case, I disagreed. We had touched upon that piece of code several times the last sprints, we had fixed several bugs in it, and chances were we would have to dig in to it again in the next sprints.

What definition of "work" do you follow, when it comes to your code? In our discussion above, my colleague was of the opinion that if the user experienced the feature as working, we should not risk breaking that state of "working". I think that's a bad definition for "work".

Any definition of "work" must include maintainability to some degree. Of course, the required level of maintainability may vary, but for most cases readability and testability not to mention the existence of adequate tests must come first. If the code isn't easy to understand, it doesn't work. If the code is difficult to test, it doesn't work. If the code is difficult to exclude from a test that don't need it, it doesn't work. If it isn't covered by a test, it definitely doesn't work. If it doesn't work, it needs to be fixed.

The code must stand the test of change in order to be flexible. Maybe the feature works, but the code doesn't. So go and fix it anyway.

8. A New War on Waterfall

"I'm concerned, because that looks very much like a waterfall!" a coworker of mine quipped. "We need to communicate across disciplines, and some times things discovered while implementing the solution might affect the final design of the product" he continued. The product manager had just finished outlining a sketch of a process he wanted us to follow when bringing new products or new versions of products to the market. My colleague (a very competent one, I might add) commented on what he perceived to be communicated as a linear step by step process. As he spoke, I felt an agonizing pain somewhere in my soul. As I've come to do ever so often lately.

The plan our product manager had outlined, contained steps of a data-driven process for product development. Where we every step of the way, from first concepts, then minimum value products (MVPs) and prototypes, could validate the product at different stages by interacting with the audience even before implementation and deployment. What my colleague failed to see was that the PM was picturing a "main-flow" - not a prescriptive "do this, then that, then finnish with this" type of plan. Colleague reacted as if saying that we want to do user studies and prototypes before we start working on the real thing, implies that lessons learned while implementing the product, or data from the live product, can not affect the direction the product takes. Which was obviously not what the PM had in mind. My colleague is not alone in making this mistake, though. An entire industry have made the same error for decades.

Even the person defining the waterfall process was clearly aware that any ventures in to software development would require feedback from parts lower "down" to various parts higher "up". The "waterfall process" is usually credited Winston W. Royce, although he did not use the term in the original paper. The figure he used to outline his basic process, or workflow, however, does indeed resemble a waterfall, hence the name. The waterfall is a flow of "steps" from System Requirements through analysis, design, coding and testing to deployment. The term Waterfall then, have come to mean a process where every activity in that list is performed to completion in isolation, where the result is handed over to the next box for processing. Below is the original canonical picture of the waterflow process.

From reading the paper of Dr. Royce, however, it becomes quite clear that this inflexible process is definitely not what he intended to communicate in his paper. On the general overview of the process he states, and I quote: "I believe in this concept, but the implementation described above is risky and invites failure." The paper is full of examples showing how, if applied blindly, such a "step-by-step" process might fail. He also state quite clearly, translated to more modern terminology, that it is usual to have iterations spanning multiple phases. And there is also modern sounding gold in the paper, like "Involve the customer - the involvement should be formal, in-depth, and continuing" - still a hallmark of good software development if you ask me! So the first paper that defined the Waterfall process, is actually the first loud critic of it. It actually talks about something that to a high degree resembles something closer to a modern agile iterative methodology. Unfortunately after Royce's paper, and contrary to his intentions, the term waterfall is cemented in to our minds to mean an inflexible step-by-step process that are doomed to fail due to lack of feedback loops. It is wrongfully used as a contrast by whoever have some new snake-oil a new process to sell. But that inflexible step-by-step process never existed.

This is the core of my pain: Most, if not all, people I meet in the software industry are (somewhat) grownup persons who are well aware that succeeding in software development require communication and feedback, and that things we learn underway almost invariably change what we end up building. Heck, it's those who are fresh out of school who tend to yell out the loudest about the need for feedback and communication. Do we forget this when we get older or something?

Could we please then, even if somebody for the sake of communication paints a simplified picture of a bunch of boxes with arrows between them take for granted that there are feedback-loops? And do that even if, taken literally, it resembles a waterfall? I'm not saying I believe that those feedback-loops will be popping up all by them selves. They must be nurtured and cared for as any other important aspect of the development process. But sketching a general flow from idea to finished product does not exclude it's existence!

It seems to me like yelling "waterfall!" has become the favorite guilt-by-association argument. Some feat, for a process that never really existed anyway! So then. Hereby, I declare war on the term waterfall it self. 

7. Test behavior, not methods

I recently revieved some code from one of my colleagues. A birds-eye view of his class looked like the following. There was nothing wrong with it - it looked perfectly fine to me.

// Foo.java
class Foo {
  public void Foo(Bar bar) {...}
  public void methodOne(String str){...}
  public String methodTwo(){...}
  public Integer methodFour(Boolean b){...}
}

However, I read the test for this class first, FooTest. Even though it looked like it tested what it should test, I had some immediate comments after just a quick glance in the file.

class FooTest {
  @Test public void testMethodOne(){
    ...code...
    assert...
    ..reusing and modifying state..
    assert...
    ...more code...
    assert..
  }
  @Test public void testMethodTwo(){..more of the same..}
  @Test public void testMethodThree(){..more of the same..}

As many others, he had written one test for every method in the system under test (SUT). The methods was long. State bled through from the start of the test-method to the end of it. Variables were reused and their state modified as the test went along. I usually prefer to test behaviors, not methods, and I felt being on safe grounds when I suggested for him to rearrange his test so that he had one test per behavior.

He thought what I said was a ridiculous idea. As an example he pointed out one of my own tests that follows this pattern and looked something like the following.

class FooTest {
  @Test public void methodOne_leaves_foo_with_clothes_on(){...}
  @Test public void methodTwo_calls_mom(){...}
  @Test public void methodTwo_writes_result_in_book(){...}
  @Test public void methodOne_after_methodTwo_lights_the_firecrackers(){...}
  ...more tests...

"I hear what you say, but I think it is ridiculous. What do you need all those damned tests for? You have twice the amount of code in your tests than what you have in the classes you write. Plus. You are violating our coding standard with those underscores! Think about maintainability, man! And after all, this is just the test-code. It doesn't need that much focus." Awh the pain! I managed to resist the temptation to slap the offending party in the face with a fresh cod I usually carry around for that exact purpose. Instead I resorted to a reasoned debate and conversation.

In some respects he's right. It becomes more code. Much more code. But I've come to realize that organizing tests by methods does not tell me anything about the behavior of the SUT. In order to understand the behavior of Foo from above, I need to read all the code in the tests. I probably need to read all code in Foo as well. And from that I might be able to deduce what it does or what it's supposed to do. But even if I get thus far, it's still very difficult for me to understand what aspects of that behavior actually have a test and which one have not.

One test per behavior, allows a quick glance of the behavior of the class or subsystem just by collapsing the body of the test-methods. The naming of the tests can read as a story. IMO the layout and cleanlyness of the test-code is more important than the production code it self. If we need more words to communicate, write more words - communicating the intents clearly is more valuable than writing less code.

As Einstein said, paraphrased, make it as simple as possible, but no simpler.

6. Efficient TDD: Start closer to the end user

Nobody gets it right every time. Especially not software design, where... well... everybody gets it wrong most of the time. Even when doing Test Driven Development (TDD). I remember a series of blogposts by Ron Jeffries where he tried to implement a Sudoku-solver using TDD and absolutely failed to solve the problem. If you read the posts from Ron Jeffries, notice how he jumps straight in and attack an internal design detail before even understanding the problem. He basically gets stuck there and end up without a solution.

I remember having had design-sessions that looks eerily similar. After listening to the product owner for two minutes outlining what's needed, we went on to discuss lots of internal design details. "We would need a Frobinizer." "Also, we need a FlexiPlug" "The Frobinizer and FlexiPlug cant talk to eachother directly - they need to post and read messages via the PlancCampfer." "But the semantics of a Planc dictates that it should not share component with the Camfer, they should be separated." At more than one situation, I've found my self preoccupied with the software components to a very high degree, and to a lesser degree about how it actually implements the features we wanted. And I do believe that the times I've ended up with a huge outburst of frustration and then scrapping everything and starting over, is strongly correlated with the times I've become to preoccupied with the internals of what I'm making.

The topic for this post is a strategy to approach design and implementation in a way that I experience steer me away from the problems Mr. Jeffries experienced. The key is to start with a test that is as close to the user as possible.

All applications originates with some user or some agent that interact with the application (not to much of a generalization, I presume.) If I write a GUI application (I've blogged about something slightly relevant previously)  I'm probably using a GUI framework, such as Qt, wxWidgets, GTK or Swing, so the actual view I'm putting together is composed of components from the framework. I'd like to avoid testing these framework-provided components. But the user clicks a button with the mouse, or he type some text in a text view, or he exercise a key-combo. I'd like to start as close to the user as possible, and for a GUI application this is where the user action leave the framework code - usually this involves implementing some kind of event handler. So the first test is for the event-handler that originates with the user interacting (and it's immediate surroundings.)

For another example, we can look at a web-service. When adding something new I like to start with writing a test for the network access-point. The test basically checks that the web-endpoint manages to get responses, and react appropriately to them. The endpoint is tested by relaying representations to an interface. The diagram below illustrates the  basic design of such a test.

I start by writing the WebServiceEndPointTest and the WebServiceEndPoint, create a interface that represent the real behavior, and translates calls to WebServiceEndPoint to something ServiceInterface can relate to (e.g. method calls and values.) The test then sets up the mock for the ServiceInterface, exercises the WebServiceEndPoint and validates that the interactions with the mock was correct. There is nothing revolutionary with this idea of course - the main point of this blogpost is that this is the right end to start working in. Get the user interactions defined first, before the design of the rest of the application get so set in stone that it can dictate how the user interaction can be.

You may view this as performing top-down design, as opposed to doing things bottom up. And you would be right in pointing out that this leads to a design that pretty much follows the Dependency Inversion Principle (DIP.) By following DIP, we get a design where "lover level" components and packages depend on abstractions exposed by "higher level" components, and that everything relates to each other via those abstractions. This in turn leads to a nicely decoupled and inherently testable codebase. The figure below is a caricature of the dependency inversion principle. If you let Foo in the picture below play the same role as WebServiceEndPoint above, notice how higher level components don't depend on lower level components, but that lower level components depends on abstractions from higher level components (higher translates to closer to the end user.)

It also turns out that the result of following this advice, becomes very similar to behavior driven development (BDD), without the domain language for specifying the tests. At the beginning I get an outer loop, where I write tests that closely capture the behavior and the interactions of the end user. These tests are also suitable for testing the overall behavior of the system, and can be 'wired' as integration tests (in addition to working as unit-tests for the outermost units.) I also get an inner loop, where I can refine and refactor the designs and tests of individual units that build the application. So from following a very simple advice - starting as close to the user as possible - we've discovered the dependency inversion principle and we've been doing BDD. Not bad for an afternoons work!

So, how would I do the sudoku thing then? When I solved sudoku by means of TDD, I started with a textual representation of a completed sudoku-board and wrote a test that verified that my solver recognized it as complete and valid. After that I progressed with leaving one, then two, then more fields blank. After a handful of tests, I had a solution that was not to dissimilar from this solution, based on search and constraint propagation. I felt satisfied with the solution, and I never needed any tests for the internal data structures. Starting closer to the user guided me in a good direction.

5. Ruthless with email

I'm using David Allens system for personal productivity called Getting Things Done. The system states some overarching principles, such as Inbox Zero - reduce the content in all inboxes to zero elements at least once every day, and a process in order to reach that state.

Roughly stated, the process for emptying all inboxes is as follows: For each item in the box, ask the question "Is it Actionable?" If yes, formulate a next action and place this in a system to keep track of it. If no, figure out if it's trash, if it's something you need for later, or something you want to keep for reference. Sounds like simplicity itself, right? See the picture below for details. There is one big problem with the application of this procedure. The first question - "Is it Actionable?" - generate too many false positives, and there are too many effing emails in this world.  If you open an email, and ask your self "Is this Actionable?" your brain will happily translate that to "Is there something I could possibly do with this thing?" and then happily continue to answer "Well, dude! Yeah, sure! Here's some random thing you could do!" "This commercial from the electronics store... You know, you should investigate a new stereo!" Bam. New useless project.

In the email-account I use for work, I get ~70 mails I need to relate to in some way or another pr. week. That's... 14 mails a day. There are spikes of perhaps 30 mails in one day. The actual number of mails that hits my mailbox is higher - maybe three times that number - but two thirds get sorted to designated folders I seldomly need to look at. I'm not on any mailing lists except those relevant at work. So 70 - that's the number of mails I need to open, read, and figure out if and how to act on. In addition I get ~30 mails a week for my private account. I don't know if 100 emails a week is a lot. It feels like a lot at times. And some times I get a bit on my heels, and they pile up. So how do I cope?

The solution to this email-problem is found in the parts of GTD that have not gained the same amount of attention as the mechanics around processing. Allen talk about six different horizons of focus:

1. Next Actions
2. Projects
3. Areas of focus and responsibility
4. Goals and intentions - things that should manifest in life in two to three years
5. Vision - how should life be
6. Purpose - principles, ideals and why I ultimately are here.

Most tools only handle 1, 2 and perhaps they pay lip-service to level 3. These are the high throughput artefacts. The things that should have turnaround-times in at most a few weeks. They pose interesting engineering challenges. They are easier to talk about. "Hey, what system do you use to track your next actions?" But in order to fix that mail problem, I find I need to play on more levels.

A more representational thought-procedure for my part might be something like the following - if I really spell it out.

Is it somewhat probable that this item does not relate to my current projects, my defined areas of focus or my goals?

If the answer is yes, I chuck it in the bin right away. This is not revolutionary, by the way. Judging how meaningful a piece of "stuff" is to you by aligning it with your areas, goals and vision is exactly the point of the GTD-system. This fact just seems to drown in the tool-mayhem.

I can intuitively answer that question quite fast. Often even without opening the mail. If I hesitate a little bit with deciding yes or no, I archive it instead of deleting it. Minor safety valve. Sure, I loose stuff. I quite possibly miss opportunities. I feel a bit like a ruthless bully when I munch through 20 mails in five minutes, and only two of them survived the ordeal. But I empty my inbox and feel good about it.

Didn't I get back to you on the email you sent? I guess you didn't align with my grand plan in life. I'm sorry. But my inbox is clean!