This principle is common to Lean Startup, Playbook Risk Management, and Knowledge Based Product Development as well as Agile and KANBAN Software Development methods. Spoiler Alert - it’s all about the batches….
Don Reinertsen showed us, initially in Managing the Design Factory, circa 1997, the meaning of a design batch and the costs associated with batching our requirements and tests.
Don points out that, if we design our product to do X, Y and Z before we test it, it will generally take us longer than if we focus on X first, then Y, and then Z. By batching these features (requirements) together we delay our learning, which results in longer projects, or projects where some requirements are not met, or both.
Returning to the example from Lean Startup and in the previous post, from Mark Cook at Kodak, there are four high-level requirements which must be met before we will have a profitable product.
1. Customers recognize that they have a problem
2. Customers will pay for a solution
3. Customers will buy the solution from us
4. We can build a solution
By waiting until we have number 4 satisfied before we test number 1-3, we create a big batch of requirements to test and we delay feedback. Feedback about number 1-3 impacts how we satisfy number 4, and whether we even spend our money and time trying. Receiving late information about 1-3 can easily result in very costly delays and lost profits.
Our visual depiction of this situation in simplified Learning Map form looks like this:
Figure 1: Learning Map
In our Learning Map we capture the objects that answer a question, mitigate a risk and/or enable the satisfaction of a requirement (which are all pretty much the same thing) in a green, lower-level node. The answers to 1-3 often impact our solution and how we achieve number 4.
For example, customers will only pay so much for a solution. The price they will pay often changes what our solution looks like -- a lot. By leaving these questions unanswered for a long time, we carry an ever-increasing risk exposure much longer. Often enough this increasing risk eventually costs us. Instead, by seeing the impact of each answer, we can determine which questions to answer first and burn down the risk more quickly. We meet our economic objectives far more often by taking this approach.
The earlier we burn the risk, the earlier we will be done, the higher quality and performance we will achieve, and the more profit we will make. This is the essence of Lean Product Development.
The product of product development is knowledge and information. (We will explore the relationship between knowledge and information in a future post.) When we develop products we are essentially generating, translating, and transferring knowledge and information -- answers to questions we either recognize beforehand or not. Almost every method we see in Lean Product Development attempts to generate, translate, or transfer information and knowledge faster.
As an example, we can develop quick-and-dirty prototypes to test or show customers. Sometimes our prototypes are just an artist’s rendition, or a mockup in PowerPoint. We get at least a little knowledge a lot faster. We spend days or weeks eliminating a significant portion of the risk rather than waiting months or years to eliminate all of it. Each piece of information (answer) has a potential impact. The combination of each answer’s impact and probability is the corresponding exposure of the risk that the unanswered question poses. The Risk Exposure is the measure of how valuable the information is.
Success in product development comes from generating more valuable information earlier.
Lean is all about getting correct answers to the right questions in the best order according to the impact of the answers, and doing so as quickly as possible. Lean is about burning down risk more quickly.
There is a standard process in product development which we follow whether we are Lean and learning quickly, or not Lean and learning slowly. The steps are the same – Design, Build, Test - repeat. In Lean New Product Development (NPD), we design, build, and test only as much as necessary to generate at least some of the answer. Perhaps it is my background in hardware development, but I see good Lean Design→Build→Test loops are the same as Eric Ries’s Build→Measure→Learn cycles.
Figure 2: Build Measure Learn and Design Build Test loops are similar.
Eric Ries includes Design in his Build step. Sometimes Design only amounts to designing the test -- what we will test and how we will measure the results. Sometimes we need to design at least some basic test articles, a quick-and-dirty prototype, or a test setup. Other times there is more traditional design and build preceding the test, but there is always at least a small design step. The object is to get through this loop quickly, while still achieving significant learning and/or confirmation of our design.
One important note - that not all tests produce good learning. With a Lean Startup approach to our loop, more correct learning is achieved via measurements from downstream (aka Validated Learning). We actually burn risk, rather than just think we did, and gain higher value in each iteration.
It is the combination of our speed and the amount of valid learning in each loop that we are trying to maximize – the risk burn down rate, which is the risk exposure reduced or answer value achieved per week.
Imagine a race where you pick up requirements and risks at the beginning of each lap and run, walk, or drag them around the track, as depicted in the figure below.
Figure 3: The Product Development Race
Each time you complete a lap, if you learned or confirmed something valuable, you can deposit in the bank some of what you carried on that lap. Each requirement (whose value is represented by the number of dollar signs), comes with some risk (represented by the number of question marks), and we must carry both. The number of question marks determines how many laps each requirement must be carried in order to deposit its full value. We can run a lot faster if we carry fewer risks and requirements.
In many cases, carrying twice as many requirements takes more than twice as long to complete a lap. In some cases, some added requirements only adds a little time to the lap. The number of risks and requirements we carry in each lap is our design→test→learning batch. The smaller our design batch, the faster we complete the cycle.
The object of the race is to deposit in the bank the most value over a given time period. Our choice is to either run fast and deposit often, carrying only a little at a time, or to walk slowly and try to carry a lot on each lap.
In this race analogy, our aerobic conditioning is good. We can breathe all we need to, and going more quickly doesn’t mean we will need to take more breaks or collapse at fewer laps. In fact, the more we carry, the more likely we are to need breaks, collapse from exhaustion, or trip over something in our path. As a result, the more likely it is that something will fall out of our bag.
One last important part to this analogy is that the more question marks we carry in each lap, the more we will leave in our bag for the next lap. In other words, if there are a few questions to learn answers to, we can answer all of them in the same lap. If there are many questions, some will be hidden from us and we’ll end up carrying them, and their associated exposure, for at least one more lap.
In Lean, each lap is very focused on a few specific requirements and risks. We load up a backpack with the most we can run with and run quickly around the track. On the other hand, in traditional, non-Lean product development, we do a lot more design before we build and test. We batch together and try to satisfy a lot of requirements with each Design→Build→Test loop.
In traditional product development, we typically attempt to complete this race in three or four laps. We carry over 80% of the requirements and risks with us on the first lap – all of the things the product is supposed to do and much of what it is supposed to look or feel like. The only requirements we leave behind are the one we expect to slow us down a lot - like ‘get parts from the production vendor’, ‘have production people assemble it’, etc.
Figure 4: Traditional Product Development
Because we try to move so many requirements at the same time, it becomes impossible to move them forward at any real pace. Instead, we move almost the whole pile a little bit further forward and make a new pile. Then we repeat this until the whole pile has made it around the track. It is slow, and we can only make a deposit once we get all of it to the bank.
It is the amount of work, and the number of requirements and questions we put into each Design, Build, Test iteration and therefore, the time it takes to get to Test, which either makes us Lean or not Lean.
Learning is valuable and becomes money-in-the-bank in the new product development race. Most learning happens during Test. In hardware development companies a little learning happens during Build. However, during Build in hardware development we are really testing whether our suppliers can make it and whether the parts fit and are easy to assemble. We only learn when we fail these tests. Very little learning happens during Design. In fact, quite the opposite. It is in Design that we make incorrect assumptions which we will learn about when we get to test. The more we design, the more incorrect assumptions we build into our design, and the more learning there is to do later.
In our example from Mark Cook and Lean Startup, by designing and building the solution before testing/answering our requirements/questions 1-3, we leave some high-impact uncertainties in place for a very long time. We create a large batch for our learning. We carry a lot of risk (a lot of uncertainty AND the large impact of that uncertainty), for a very long time. By debatching our requirements and questions (breaking them down into smaller iterations), and designing→building→testing them independently, we can get answers and gain certainty much earlier. This results in much faster, more successful projects and higher profits.
There is one more big benefit to debatching our requirements and questions and getting to a test earlier. We discover more of the unknown unknowns -- the questions we didn’t know we needed to ask earlier.
Where do we find the bugs which are hidden in our products and our 'hard-to-imagine-they-are-incorrect' assumptions? We find these unknown unknowns the same place we always do - downstream. For example when we get a product to verification & validation (V&V) testing, regulatory approvals, manufacturing, customers, service, and anyone else who resides downstream from development.
We find these unknowns earlier by getting downstream earlier. We get downstream earlier, in part, by designing and building less beforehand. We build a less-than-100%-complete product, get it downstream, and find many issues earlier than we otherwise would.
The common underlying principle between Lean Startup, Agile, Kanban, Flow, Knowledge Based Product Development, and PLAYBOOK Requirements and Learning Management is this – it is the breaking down of some stinking batches in order to burn risk and achieve value more quickly.
If you have seen Blazing Saddles, you know the voice to use when reading the title above. If not, or for a reminder and good chuckle, see a clip here. Or go here for the original, which Mel Brooks parodies. It’s pretty funny, too. Check your volume level first. Youtube puts some pretty obnoxious ads before the videos sometimes.
In every case, we break down the design→build→test batch into smaller pieces, and we learn and complete our projects much faster as a result. More specific to Lean Startup, Agile, Kanban, and PLAYBOOK Requirements & Learning Management is the recognition that there are unknown unknowns which we will draw out earlier by getting our incomplete designs through build and downstream earlier.
There are many in the manufacturing and hardware development world who think some of this is just plain madness. How can we possibly get done earlier every time by designing and testing X before we add Y and Z? How can we possibly put a partial product downstream into V&V , or into a regulatory approval process to draw out unknown unknowns earlier? My reply is this – you are right, sort of.
Hardware is different in many ways from software. When lead-time and part cost are a significant percentage of the design→build→test loop, the optimal amount of design we put into each batch does get bigger. To achieve project delivery at the fastest velocity possible, we need to understand when a design batch is too small and when it is too big. Fortunately, most hardware companies can safely reduce their batch sizes from where they currently are and they will improve.
Please continue with me to the next few posts, where we will look at a more interesting real-life example, from the origins of Lean – the Wright Brothers. It’s more interesting to me anyway, with my background in Aeronautical & Aerospace engineering. We will use this example to dive deeper into Lean Startup, PlaybookRequirements & Learning Maps, the differences between Hardware and Software development, and establish a common approach to debatching our product development and learning as quickly as possible in order to assure our success.
In the next post, we’ll review some quantifiable benefits of reducing the Design Build Test batch size, as presented in a research paper by Sang-Don Lee of General Motors and Ik-Sung Lim of Namseoul University.
----
Lean Startup Methodology and Product Development
Lean Startup Methodology and Risk Management
Lean Startup Methodology and Reducing Batch Sizes
Lean Startup Methodology and Early Learning
Lean startup principles
Small batch sizes
