Vviewpoints
DOI: 10.1145/2347736.2347745
the Business of software
the goldilocks estimate
Ben WAs HUrrYing to his next meeting when his boss topped him in the hallway. “Ben, I’m heading up to the CEO’s office for the budgeting meeting,” he said. “Remember the
system upgrade we talked about the
other day? Could you give me a quick
ballpark dollar figure that I can take to
the boss just as an example? You won’t
be held to it, of course…”
the Drive-By Estimate
A drive-by estimate occurs when a senior manager corners a subordinate
and demands an immediate answer to
the estimation questions: “When can
we get this project done?” “How much
will it cost?” and “How many people do
we need?” Depending on how much
pressure is applied, the unhappy estimator must produce some numbers
and do it quickly, usually without much
research. Estimates derived this way are
normally of low quality and making any
kind of critical business decision based
on them is dangerous and often costly.
the never-Ending Estimate
At the opposite extreme, sometimes estimation can be a process that goes on
and on and on. To make an estimate
“safer” and more “accurate,” organizations may try to remove all the uncertainty in the project and the data they
use to create the estimate. One way to
reduce uncertainty is simply to spend
more time and effort in analyzing the
situation—spending more time and effort on something almost always produces a better outcome. However, when
estimating a project, the work we have to
do to remove the uncertainty in the esti-
Cost versus the time and effort to produce an estimate.
Spending too little time or
effort on an estimate results
in a high risk of variance—the
estimate is not "accurate"
Spending too much time and
effort may "overengineer" the
project, plus time is passing
and the project is not running
overall Cost
Cost
Cost of Project to Produce estimate
Ideally, we would spend
just enough time and
effort to produce the
most valuable estimate in
the time available
time and effort to produce an estimate
Cost of Project of Variance in estimate
mate is pretty much the same work we
have to do to actually run the project. I
have seen companies where, in order to
decide if they should run a project and
what resources it will need, they de facto
run the project and use up the resources.
Ideally, an estimation process would
aim for the Goldilocks “sweet spot”
where we spend enough time and effort
on the estimate to be sufficiently confident in the result, but not so much that
we overengineer the estimate and actually start working the project.
inverse Exponential value
Most uncertainty reduction processes
(such as inspections, testing, and es-
timation) follow an exponential decay
curve. Initially there is a lot of low-hang-
ing fruit—items that are quite clear and
easy to find and fix. As we progressively
clean up the more obvious issues, the
remaining ones tend to be more ob-
scure, ambiguous, and difficult to find
and it takes us longer and longer to re-
solve the next uncertain item.
Linear Cost
If we allocate a certain number of people to produce an estimate and let them
work until we are “satisfied” with the
result (whatever that might mean), the
cost profile will be linear and ascending
with respect to time and effort. The combination of these two graphs produces a
U-shaped profile, as shown in the figure
here. Too far to the left and the likely
cost of a poor estimate will be high. Too
far to the right and the work done in producing the estimate is not balanced by
