functory job of it when they do. But that
is a topic for another day. This simple
arithmetic could be called a “
straight-line ROI.” It is simply the expected returned value divided by the expected
cost. It is a simple calculation, easy to
compute and to understand. But, in
most cases, it is also wrong.
figure 1. straight-line Roi.
figure 2. Gaussian probability distribution.
figure 3. Cost profiles.
figure 4. Value profiles.
the Role of Risk
The reason why straight-line ROI is
wrong for most projects is simply that
it does not account for risk. The return
computed using the above straight-line ROI calculation will be true only if
˲ ˲ 100% guarantee of cost containment at $1 million—that is, the project
has no cost risk, the project cannot/will
not run over or under in budget.
˲ ˲ 100% guarantee of value returned
at $1.1 million—the project has no return risk, the return is fixed and invariable no matter what happens to the
These are the necessary conditions
for the calculation to be valid. There
are conditions where the cost risk and
value risk cancel out and the calculated
return ends up at 10%. For instance, if
a project overruns on cost but is able
to recoup more value than expected,
it may cancel out the budget overrun.
Note, this does not mean the calculation is correct, simply that the project
was “lucky”b in that the inaccuracies
happened to be equal and opposite.
The moment we introduce risk, the
straight-line ROI calculation does not
work. If we only have a 20% probability
of cost containment at $1 million, given a typical set of project conditions,
the ROI is not a positive 10%, it is more
like a negative 18% (!)
stocks and Bonds
This is true in other disciplines. Equities typically carry more risk than
government, municipal, or corporate
bonds so we expect higher return to
compensate us for the risk. Bonds are
safer, so we are content with less income because the downside is more
controlled. A less sophisticated financial consultant may show a customer
what savings might be achieved over
time based on the average returns of
the stock market. The U.S. stock market has realized approximately 9% average annual gains over the last 100
years or so (depending on the index
used and whether returns are compounded). Showing a potential investor a nice straight line of ever-increas-
b This is similar to the “lucky” (as opposed to
“accurate”) estimate described in P.G. Armour, “Truth and Confidence,” Crosstalk
(Apr. 2008), 27.
ing wealth is more a sales gimmick
than a realistic prediction of return
since it does not account for risk. Similarly, calculating the likely return for
a software project without accounting
for risk is bogus.
The financial services company I
mentioned would not dream of using
a straight-line return calculation for
its investments but, like the insurance
company that did not calculate cost of
risk, it blithely performed the wrong
calculation on its internal projects.
And it wondered why it got blindsided
by failure to achieve returns on most of
To more correctly calculate the true
likely return we must incorporate the
cost of risk and its counterpart—what
we could call the value at risk. Recently, financial markets have shown
what failure to properly account for
risk does to one’s investment, either
through not including risk in the calculation or having the risk hidden inside complex derivatives. It is very important that we learn from this in the
business of software by performing
the right calculation.
There are six elements to computing return at risk:
˲ ˲ The expected cost of the project
˲ ˲ The likelihood of achieving that
˲ ˲ The risk profile or “shape” of the
cost risk distribution
˲ ˲ The expected returned value of the
˲ ˲ The likelihood of achieving that
˲ ˲ The value profile or “shape” of the
the reason why
is wrong for most
projects is simply
that it does not
account for risk.