Uncertainty in Product Development
Gaining a strategic advantage through statistics and mathematical analysis is changing the game of product development. We are learning every day how to build better mathematical and scientific models for our systems and how to drive our desired direction based on those models
When faced with decisions in new product development, and throughout life, there is uncertainty. We very often need to use at least some gut-feel in our decisions. Unfortunately, we are so accustomed to it, we too often rely only on gut-feel, even in cases where we would be better doing a little analysis, too. "Should we develop product X?", "Do we add feature Y?", "Do we spend more money on additional resources?", and many other questions are answered and decisions made too often using only gut-feel.In each case, in the back of decision makers' minds, or in their gut somewhere, they evaluate the benefit, cost, and risk, and estimate whether the resulting value seems high enough to make it worth doing. A qualitative analysis of value is performed internally, invisible to everyone else, sometimes even unconsciously. The value is assessed, and the decision is made, by gut-feel.
So what does new product development decision making have to do with Brad Pitt?
If you have seen the movie Moneyball, with Brad Pitt and Jonah Hill, or read the book, you have seen a great example where large improvements have been achieved by adding a little more mathematical analysis to what was once only gut-feel.
Some may disagree and assert that, "The (Moneyball) term is ... falsely associated with the use of advanced statistical models ... to build a club that could thrive. But the principle of Moneyball is about exploiting systems for maximum gain by acquiring talents that are undervalued by the rest of the industry." It seems to me that the principle of Moneyball is absolutely the mathematics. It is the quantitative analysis that makes it different, special, and better. The models really weren't that advanced, either.
It is All About How You Measure Value
For every team in the league, even the Yankees, baseball scouts were sitting around their team tables looking for good players they could get for a good enough price. Everyone was looking for undervalued players, where undervalued means that the value you expect is higher than the value you expect of other players you could get at the same cost, or the cost of the player is less than other players of the same expected value.
The difference was how they measured value, or didn't measure it, really. Every other team estimated value via a gut-feel, internal, qualitative analysis, performed by each scout separately. It involved parameters such as batting average and home runs, as well as how good the player looked in the batters box or in the team picture, or where they played in college, or whatever else each scout thought was an indicator of value.
But Peter Brand, and eventually Billy Bean, had a different approach. They first defined value quantitatively - in number of wins. Then they built a mathematical model to measure value (wins) as a function of specific, measurable parameters, such as home runs and batting average.
In so doing, they realized that On Base Percentage (OBP) and Slugging Percentage (SLG) were better indicators of real value (# of wins) than how someone looked in the batters box or how many home runs they hit. Then they made the OBP and SLG parameters weigh more heavily in their 'Expected Value'. Then they found the players who had a higher expected value per dollar (ROI), and they won more - a lot more.
Build Better Models and Measure Quantitatively
Mathematical analysis has changed the game of baseball in the same way it is changing the game of product development, politics, and a great many others. We are learning every day how to build better mathematical and scientific models for our systems and how to drive in our desired direction based on those models. And we are learning how to supplement the gut-feel of a few people with a more quantitative approach, and getting better results. For more examples, I suggest reading ‘The Signal and the Noise’ by Nate Silver and ‘The Wisdom of Crowds’ by James Surowiecki.
The same large improvements are available to product development companies willing to use a little more mathematical analysis in their economic decisions. Quantitative means for assessing value are also at the heart of the Lean Startup revolution and its Build-Measure-Learn loops.
While we cannot predict the future with 100% accuracy, and no analysis or model is perfect, the more data-driven we can be, the more profitable our decisions will be. Not every player chosen by the methods in Moneyball played well for the team and increased their wins. But, more players helped then didn't, so the average was improved and the team won.
We need only do the same in product development, where we are getting more data-driven every day. Because we execute many projects, and make project decisions multiple times a day, every day, we have the law of averages (or, more correctly, the law of large numbers), on our side. In fact, we live in an uncertain world, and in a world of uncertainty, we are governed by the laws of statistics.
Use Statistics to Gain a Strategic Advantage
Statistical laws are like the laws of physics. They are just as unbreakable, and if we understand them well, we can use them to our advantage. And, of course, if we want to get better at something, we have to practice. The more we practice, the better we get.
So continue with us through the rest of the series, where we dig deeper into how we measure value, build our mathematical models, and make more profitable decisions in the high-uncertainty world of product development. Stay tuned....
Related Posts
Introduction to Cost of Delay
What is Cost of Delay?
Cost of Delay: How to Calculate It
Cost of Delay and Project Modeling
Cost of Delay Project Model Examples
Cost of Delay Project Modeling Risk
Cost of Delay and Strategic Advantage
Cost of Delay: Project decisions based on profit
8 Ways to Decrease Risk in Project Decisions
14 Tips for Calculating Cost of Delay
WSJF and How to Calculate It
Don Reinertsen on cost of delay
Wikipedia on cost of delay
Guide to Cost of Delay
