2013-03-30

8. A New War on Waterfall

"I'm concerned, because that looks very much like a waterfall!" a coworker of mine quipped. "We need to communicate across disciplines, and some times things discovered while implementing the solution might affect the final design of the product" he continued. The product manager had just finished outlining a sketch of a process he wanted us to follow when bringing new products or new versions of products to the market. My colleague (a very competent one, I might add) commented on what he perceived to be communicated as a linear step by step process. As he spoke, I felt an agonizing pain somewhere in my soul. As I've come to do ever so often lately.

The plan our product manager had outlined, contained steps of a data-driven process for product development. Where we every step of the way, from first concepts, then minimum value products (MVPs) and prototypes, could validate the product at different stages by interacting with the audience even before implementation and deployment. What my colleague failed to see was that the PM was picturing a "main-flow" - not a prescriptive "do this, then that, then finnish with this" type of plan. Colleague reacted as if saying that we want to do user studies and prototypes before we start working on the real thing, implies that lessons learned while implementing the product, or data from the live product, can not affect the direction the product takes. Which was obviously not what the PM had in mind. My colleague is not alone in making this mistake, though. An entire industry have made the same error for decades.

Even the person defining the waterfall process was clearly aware that any ventures in to software development would require feedback from parts lower "down" to various parts higher "up". The "waterfall process" is usually credited Winston W. Royce, although he did not use the term in the original paper. The figure he used to outline his basic process, or workflow, however, does indeed resemble a waterfall, hence the name. The waterfall is a flow of "steps" from System Requirements through analysis, design, coding and testing to deployment. The term Waterfall then, have come to mean a process where every activity in that list is performed to completion in isolation, where the result is handed over to the next box for processing. Below is the original canonical picture of the waterflow process.


From reading the paper of Dr. Royce, however, it becomes quite clear that this inflexible process is definitely not what he intended to communicate in his paper. On the general overview of the process he states, and I quote: "I believe in this concept, but the implementation described above is risky and invites failure." The paper is full of examples showing how, if applied blindly, such a "step-by-step" process might fail. He also state quite clearly, translated to more modern terminology, that it is usual to have iterations spanning multiple phases. And there is also modern sounding gold in the paper, like "Involve the customer - the involvement should be formal, in-depth, and continuing" - still a hallmark of good software development if you ask me! So the first paper that defined the Waterfall process, is actually the first loud critic of it. It actually talks about something that to a high degree resembles something closer to a modern agile iterative methodology. Unfortunately after Royce's paper, and contrary to his intentions, the term waterfall is cemented in to our minds to mean an inflexible step-by-step process that are doomed to fail due to lack of feedback loops. It is wrongfully used as a contrast by whoever have some new snake-oil a new process to sell. But that inflexible step-by-step process never existed.

This is the core of my pain: Most, if not all, people I meet in the software industry are (somewhat) grownup persons who are well aware that succeeding in software development require communication and feedback, and that things we learn underway almost invariably change what we end up building. Heck, it's those who are fresh out of school who tend to yell out the loudest about the need for feedback and communication. Do we forget this when we get older or something?

Could we please then, even if somebody for the sake of communication paints a simplified picture of a bunch of boxes with arrows between them take for granted that there are feedback-loops? And do that even if, taken literally, it resembles a waterfall? I'm not saying I believe that those feedback-loops will be popping up all by them selves. They must be nurtured and cared for as any other important aspect of the development process. But sketching a general flow from idea to finished product does not exclude it's existence!

It seems to me like yelling "waterfall!" has become the favorite guilt-by-association argument. Some feat, for a process that never really existed anyway! So then. Hereby, I declare war on the term waterfall it self. 

No comments:

Post a Comment