Project Modeling Risk
In this post, we cover the impact of uncertainty in sales forecasts and other inputs as they pertain to building an economic model to drive product development project decisions.
Tackling risk in economic model inputs
Uncertainty is everywhere in product development. Some of it's visible and some of it is not.
Eliminating and exploiting uncertainty (i.e., generating profitable information), is in fact what product development is all about. It is at the heart of the Lean Startup movement, and Agile, Critical Chain, Flow and Next Generation product development, and it is at the heart of Playbook.
In future posts we will discuss risk in product development and how to leverage and eliminate it. Not by avoiding it, but by generating information more quickly. In this post we will focus on uncertainty in the inputs used in project economic models.
Reducing the uncertainty economic model inputs
First, we will eliminate uncertainty in our sales forecasts, or at least recognize the impact of this uncertainty is less than it seems. We also reduce the impact of resulting uncertainties in our Cost of Delay and Sensitivities to Sales Volume, Unit Cost, Sales Price and Project Expenses.
In the subsequent post, we will discuss how to accommodate uncertainty in our Impact Variables. For example, how much will launch date, sales volume, unit cost, sales price, and project expenses change with a particular decision? For continuity, we will be referring to the project scenarios we used as examples in previous posts, parts 1 - 5.
(You can also view our complete Guide to Cost of Delay.)
Uncertainties in sales forecasts
The first required input to an economic model is a forecast of expected sales. Executives and project team members often recognize that sales volume forecasts are inaccurate – and often overestimated. We agree. The tendency to overestimate sales forecasts is chronic, especially in traditional product development companies where project durations and expenses are often much higher than predicted.
Surprisingly, overestimating sales volume does not invalidate the economic model.
As it turns out, the cost of delay, the sensitivity to sales volume and the sensitivity to unit cost are all directly proportional to sales volume. So, if our actual sales volume is 30% less than we predicted, then the cost of a one-month delay, the profit impact of a 1% change in volume, and the impact of a 1% change to Unit Cost are each 30% less as well.
Figure 1: Impact of Error in Sales Volume Estimates
Therefore, when making tradeoffs between unit cost, sales volume, and schedule changes, we will often reach the same conclusion, regardless of our original volume forecasts.
This can easily be demonstrated using spreadsheets which calculate these sensitivities. Simply reduce your forecasts by 30% and compare the resulting sensitivities. They are each 30% lower. However, I’d like to share a quick mathematical proof of this, which also provides us another easy way to calculate our economic sensitivities.
For this, let’s think back and remember a little of our freshman calculus class, specifically derivatives. For a simple introduction to derivatives, or a refresher, see Math is Fun. In summary, from Wikipedia, “The derivative measures the sensitivity to change of one quantity (a function or dependent variable) to another quantity (the independent variable).”
The economic sensitivities (i.e., change in profit due to a change in each economic variable), are the derivatives of this formula for profit:
Profit (P) = Volume (V) * Margin (M)
= Volume (V) * (Sales Price (S) – Unit Cost (C))
P = V * M
= V * (S - C)
In our example, product V is 7200 units, M is $2000, S is $4000, and C is $2000.
So, P = 7200 * $2000 = $14.4M
(If you have questions, please don't hesitate to comment below. This is a lot to digest we know!)
Sales volume sensitivity and cost of delay
The Sales Volume Sensitivity (SVS) is the change in profit from a change in sales volume. It can be determined by taking the derivative of sales volume.
SVS = ∆P / ∆V
= ∆V * M
= x% * V * M
In the case of Sales Volume Sensitivity (SVS), the ∆V is 1% of V (.01 * V). In the case of a delay to launch (COD), it is also expressed in % of forecasted volume, but x is typically more than 1 (x = % of total units that are in the lost month + % peak reduction). In both cases it is directly proportional to Margin and Volume.
SVS = 1% * V * M
COD = x% * V * M
In our example:
SVS = 1% * 7200 * $2000= $144K (rounded to $150K in previous posts, for simplicity)
Unit cost sensitivity and sales price sensitivity
The Unit Cost Sensitivity (UCS) is the change in profit from a change in unit cost. It can be determined by taking the derivative of unit cost:
P = V * S + V * C
UCS = ∆P / ∆C
= V * ∆C
= V * 1% * C
UCS = 1% * 7200 * $2000 = $144K (Rounded to $150k in earlier posts, for simplicity.)
Comparing a 1% sales change (SVS) to a 1% unit cost change (UCS):
SVS / UCS = (1% * V * M) / (V * 1% * C)
= M / C
= $2000/$2000
= 1 in our example
The result is similar for sales price sensitivity (SPS) which we’ve left out of this example thus far.
SPS = ∆P/∆S
= V * ∆S
= V * 1% * S
SVS / SPS = (1% * V * M) / (V * 1% * C)
= M / S
SVS = 1% * 7200 * $2000
= $144K (rounded to $150k in earlier posts, for simplicity)
Implications and break-even values
The value of each economic sensitivity, including the cost of delay, is directly proportional to sales volume. When comparing them on a percentage basis, volume drops out of the equation. Therefore, even if the original volume forecasts are exaggerated, we often arrive at the same conclusion. We are confident in the output, regardless of the accuracy of the original marketing forecast.
For example, on our project where SVS is $150K and UCS is $150K, it will be profitable to increase unit costs 2% if that will increase sales volume more than 2%, even if the original forecasts are not correct. Only the margin and unit cost need to be close to correct to have confidence in this decision. Similarly, every 2.6% of unit cost we can drive out is worth 1 month of delay to us. ($400K per mo. delay / $150K per 1% of unit cost = 2.6% unit cost per mo. delay.) If we can get more than a 2.6% cost reduction per month of delay, it is profitable to do. (Unless, of course, you can effectively drive the cost out later and not delay the launch at all.)
This illustrates the break-even values we can use to help drive our decisions quickly, and with confidence. They help provide us with terms we can use to know when to stop doing things like reducing cost and adding features to increase sales. Each variable (volume, unit cost, etc.), has a break-even value with each other variable, and we can have high confidence in these break-even values, even when we don’t have great confidence in our sales forecasts.
While the absolute sizes of the sensitivities are heavily dependent on volume, their relative sizes are dependent only on margin and the actual unit costs, in which we typically have more certainty.
If actual unit cost ends up significantly higher than expected, we could consume enough margin to make the product unprofitable and the whole project a colossal waste of time! (Sound familiar?) In future posts we will discuss how to eliminate this particular uncertainty early (or rather, learn quickly).
Sometimes volume matters
Having just said that sales forecasts don’t need to be accurate, there are some cases where the accuracy of predicted volume does matter.
Note, as shown in the image above, project expenses are not a function of sales volume error. This comes as no surprise, I’m sure. If we are overestimating sales volume, we will still spend what we spend on the project. We don’t get to ‘not spend it’ if we happen to sell fewer units, like if it were unit-cost.
It is more important to have sales volume forecasts in the right ballpark when project expenses are significantly different in a trade-off decision. If the project expense difference is small between options A and B, we can compare the gain of A to the gain of B and confidently choose whichever is greater.
For example, say we must choose between spending $50K in additional resources to accelerate the project 1 month ($400K gain, $350K value, 7-to-1 ROI), or $60K on additional advertising to increase sales 5% ($750K gain, $690K value, 11.5-to-1 ROI). In this case, expenses are about the same, so we need only evaluate and compare the gains.
This example illustrates another case where sales forecast accuracy matters – when funds are limited, you are considering the ROI of your decision, and you are on the bubble. In this example, let's say we have chosen not to invest any funds which have less than a 10-to-1 ROI (Minimum Decision ROI = 10). If volume is overestimated by 15%, the actual gain from a 5% volume increase will be 15% lower. (~$640K gain, $580K value, 10-to-1 ROI)
However, when the ROI is high enough, we can have confidence in our decision, even when our confidence in the original forecast isn’t high. If our minimum decision ROI is 5-to-1, our sales forecasts need to be overestimated by 100% (actual SSV is 50% of estimated), before the decision to spend the $60K on advertising didn’t have an ROI that makes it worth doing.
Build your model mean
At Playbook, we recommend building a mean model. For sales forecasts, use the values which give a 50% chance that actual volumes will exceed your forecasts, and 50% chance that volumes will fall short. Use similar 50/50 estimates (50/50s) for expected average sales price, unit cost, and even project expenses and launch date.
There are many details supporting why 50/50s are the best approach, most of which I will spare you for now because I know that not everyone likes statistics as much as I do.
-----
Ready to create you project model, calculate the cost of delay and make project decision tradeoffs? Download this excel spreadsheet for creating your own project model.
Related Posts
Introduction to Cost of Delay
What is Cost of Delay?
Cost of Delay: How to Calculate It
Cost of Delay and Project Modeling
Cost of Delay Project Model Examples
Cost of Delay Project Modeling Risk
Cost of Delay and Strategic Advantage
Cost of Delay: Project decisions based on profit
8 Ways to Decrease Risk in Project Decisions
14 Tips for Calculating Cost of Delay
WSJF and How to Calculate It
Don Reinertsen on cost of delay
Wikipedia on cost of delay
Guide to Cost of Delay
Editor's note: This post was originally published in 2015 and has been updated for accuracy and comprehensiveness.
