In Chapter 4 of Lean Startup, entitled, "Experiment," Eric Ries shares an example from Mark Cook, the VP of Products for Kodak Gallery…. Mark explained:
“Traditionally, the product manager says, ‘I just want this.’ In response, the engineer says, ‘I’m going to build it.’"
Instead, I try to push my team to first answer four questions:
1. Do consumers recognize that they have a problem you are trying to solve?
2. If there was a solution, would they buy it?
3. Would they buy it from us?
4. Can we build a solution for that problem?
The common tendency of product development is to skip straight to the fourth question and build a solution before confirming that customers have the problem."
To me, this stands out as a perfect example of a common theme and the underlying principle of Lean Startup, Playbook’s style of Project Risk Management.
Earlier learning comes from an earlier recognition that there are questions we still need to answer. It comes from recognizing that models, analysis, and research are often good ways to generate answers early, but they are imperfect and very often much more learning remains. And earlier learning comes from Validated Learning, early and often, to counteract our natural biases and uncover the questions that remain hidden in our assumptions and the imperfections of our analyses.
Traditionally, when embarking on a new product development project, product managers and product development teams assume that the answers to questions 1-3 above are a confident ‘Yes’. In fact, confidence can be so great the questions may never be asked before building the solution. There are other similar high-level questions whose answer must be ‘Yes’ to achieve project success and profitability, but we will use these four questions for the purpose of discussion.
What we often find out too late -- after many of these traditional projects are complete -- is that our products don’t sell very well. We find out, too late, that the answer to one or more of questions 1-3 turned out to be ‘No’.
Late answers to these questions can cost us a lot. These costs include the profit we expected and the money we spent developing a product which doesn’t sell. The costs also include the cost of delay of a product which would sell. Early learning would have saved us a lot of time and energy, and made us a lot more money.
Wherever there are questions, whether recognized in advance or not, until we have confidence in the answers, those questions are risks. They represent uncertainties and potential reductions of our profits from the maximum profit achievable. I use the terms question and risk interchangeably.
To illustrate this discussion we’ll visualize the problem and solution using a simple Playbook Requirements and Learning Map. This is a simple, visual tool we have developed at Playbook to help teams establish clear requirements, see where the risks lie, and determine the best course of action. It is a modified version of a Future Reality Tree, from the Theory of Constraints. There is more to it than what I present here, and I’ll leave the details of the method for another day. For now, we’ll keep it simple.
Requirements and risks are the yin and yang of projects. Requirements produce risks, and many risks are eliminated with knowledge about lower level requirements. We intermingle requirements and risks in our Requirements and Learning Maps to create a complete single view of our future and current situation. In its simplest form, where a requirement is risky and we aren’t sure what we want or if we can easily get it, we put the requirement into the form of a question. This is described below.
Returning to Mark Cook’s example, in traditional product development the situation initially looks like this, in pure requirements form on the left, and in the form which recognizes the one question we traditionally see on the right.
Figure 1: Requirements Map (Left) and Learning Map (Right)
With a lack of recognition that customers may not buy our solution, we charge forward to determine how to build it. When we miss the market, we realize we missed a critical question.
Figure 2: Unknown Unknowns
Until these additional questions (risks) are recognized, they remain hidden among the Unknown Unknowns. Later in the project the answers to these questions emerge and they very often cost us.
What Mark Cook and Eric Ries recognized was that these questions still need to be answered in order to ensure profitability.
Figure 3: Known Unknowns
By drawing out the unrecognized unknowns and recognizing them, we can take deliberate actions to answer the questions earlier, leaving less to chance.
And so it goes with every question. There are underlying questions and often additional unknown unknowns. For example, underlying the ‘Can we build a solution?’ question are other questions like: Is it safe?, Will we pass regulatory approval?, Is it reliable?, Can we manufacture it for a low enough cost?, and many, many more.
Figure 4: Unknowns Everywhere
Often these questions also go without saying or with only loose controls over the answers until too late in a project. When these questions are answered late, like the customer acceptance questions, the impact on profitability can be extremely high. (See our complete guide to cost of delay to take a deeper dive into the impact on profitability.)
In most cases, failure to identify questions early enough is costly. Assuming an answer which turns out to be incorrect is costly. Getting an answer whenever we get it, with no effort put toward getting the answer earlier, is costly.
A reduction of these costs and, inversely, an increase in our profits comes from recognizing and answering important questions earlier. Knowledge Based Product Development is powerful because it focuses on answering important questions earlier. In most examples I have seen, Knowledge Based Product Development focuses on the questions which underlie how we build the solution, but I don’t think it needs to be limited to that.
Playbook Risk Management also covers questions about how to achieve a solution. It differs from traditional Project Risk Management, in part, because risk drivers are put into the form of a question. We dig down to the lower-level questions we really need to answer, prioritize them, and then focus on getting the most important answers quickly. Playbook Risk Management and Knowledge Based Product Development have this in common.
However, Playbook Project Risk Management also encompasses questions about market acceptance, product delivery and manufacturability, resource availability, and anything else potentially impacting the objectives of the project. The result is earlier learning about all types of important questions.
Lean Startup reminded us to recognize some very important, too often unrecognized questions. For example - Will our customers really like it? What will they pay for it? What features and attributes will they use and what won’t they use?
These questions are not new, though. Product managers and team members have been asking similar questions since the beginning of product development. When first capturing project risks, our clients very often identify market acceptance risks before we mention it.
However, project teams have traditionally lacked the ability to answer these questions early and reliably. The best methods (mitigations) available were market research, creating and showing customers our brochure before creating the product, beta testing, test markets, and other methods which are helpful but not entirely reliable or even possible early enough in a project. As a result these questions are often lumped back in with the other Unknown Unknowns we don’t spend any extra time trying to answer earlier.
Lean Startup presented better ways to answer these questions earlier, such as developing Minimum Viable Products (MVPs), using Build-Measure-Learn loops focused on learning about customers, and using Actionable Metrics rather than Vanity Metrics. In this recent post by Ramli John, he notes that MPVs are about answering specific questions. Essentially, Lean Startup identified better best-practice mitigations to our market acceptance risks. And these are big risks in dire need of better mitigations, even when our uncertainty isn't extreme. Thank you, Eric Ries.
Lean Startup, like Playbook Risk Management, has another great benefit. This is the ability to draw out more Unknown Unknowns earlier. How do they do this? We cover this topic in greater detail in the next post.
---
Lean Startup Methodology and Product Development
Lean Startup Methodology and Risk Management
Lean Startup Methodology and Reducing Batch Sizes
Lean Startup Methodology and Early Learning
Lean startup principles
Small batch sizes