0
25,000
50,000
75,000
100,000
125,000
150,000
175,000
200,000
225,000
250,000
275,000
300,000
325,000
350,000
Jan
2010
Mar
2010
May
2010
Jul
2010
Sep
2010
Nov
2010
Jan
2011
Mar
2011
May
2011
Jul
2011
Figure 2. measuring lines of code in two different ways.
Lines of code
Lines
0
25,000
50,000
75,000
100,000
125,000
150,000
175,000
200,000
225,000
250,000
275,000
300,000
325,000
350,000
375,000
400,000
Jan
2010
Mar
2010
May
2010
Jul
2010
Sep
2010
Nov
2010
Jan
2011
Mar
2011
May
2011
Jul
2011
Figure 3. measuring number of files used.
Nr. of files
Jan
2010
Even though it might seem simple,
this technique can be applied incorrectly in a number of subtle ways. For
example, imagine a situation in which
customers are unhappy because they
report problems in a product that are
not solved in a timely manner. To improve customer satisfaction, the project team tracks the average resolution
time for issues in a release, following
the reasoning that a lower average resolution time results in higher customer satisfaction.
Unfortunately, reality is not so
simple. To start, solving issues faster
might lead to unwanted side effects—
for example, a quick fix now could result in longer fix times later because of
incurred technical debt. Second, solving an issue within days does not help
the customer if these fixes are released
only once a year. Finally, customers
are undoubtedly more satisfied when
no fix is required at all—that is, issues
do not end up in the product in the
first place.
Thus, using a metric allows you
to steer toward a goal, which can be
either a high-level business proposition (“the costs of maintaining this
system should not exceed $100,000
per year”) or more technically oriented (“all pages should load within
10 seconds”). Unfortunately, using
metrics can also prevent you from
reaching the desired goal, depending on the pitfalls encountered. In the
remainder of this article, we discuss
some of the pitfalls we frequently encountered and explain how they can
be recognized and avoided.
What Does the metric mean?
Software metrics can be measured on
different views of a software system.
This article focuses on metrics calculated on a particular version of the code
base of a system, but the pitfalls also apply to metrics calculated on other views.
Assuming the code base contains
only the code of the current project,
software product metrics establish
a ground truth. Calculating only the
metrics is not enough, however. Two
more actions are needed to interpret
the value of the metric: adding context;
and establishing the relationship with
the goal.
To illustrate these points, we use
the LOC (lines of code) metric to pro-
