viewpointsV

DOI: 10.1145/2184319.2184329

the Business of software
a Measure of Control

YoU Can’t Control what you can’t measure.” So says Tom DeMarco at the beginning of his seminal book Controlling Software Projects. 3 DeMarco
was echoing Lord Kelvin who, a century
or so earlier, had said “If you cannot
measure it, you cannot improve it.”
These statements seem rather intui-
tive, and both quotes have been used
over the years to justify the develop-
ment and application of metrics pro-
grams in the business of software. Cer-
tainly it would be difficult to determine
if you have actually improved some-
thing if there were no way to quantify
the change—but there are other con-
siderations in measurement.

 

Watts, on first

In the mid-1990s I attended a presentation by the late Watts Humphrey given to an audience of telecommunications executives where he gently chided them for investing so heavily in a measurement program that was, to his mind, rather ineffectual. Humphrey’s point was that this particular company was, at the time, an SEI CMM Level 1 organization. Such an organization was characterized by the Software Engineering Institute’s Capability Maturity Model as Initial or

should the control
chicken come before
the measurement egg
or should it be the
other way around?

“ad hoc” meaning the processes used to build systems were recreated each time for each system and each project. In Humphrey’s view this meant that, to some extent, this company’s development process was out of control; or perhaps more appropriately, nothing could be inferred from measurement on this project since the next project would likely use a quite different process. As Humphrey put it, the only thing measurement would truly show was that the process was out of control and they already knew that. Therefore, he reasoned, the company should invest first in stabilizing their processes before they expended much effort in developing measurement programs.

This message received a cold response. The executives and the company had already invested heavily in their metrics program and they were proud of it. They believed it was instrumental in their technological success irrespective of their SEI level. So who was right? Should the control chicken come before the measurement egg or should it be the other way around?

 

hawthorne Would

In purely metric terms, Humphrey had a point—if the engine in your car keeps surging uncontrollably it is more important to fix it than to install a new speedometer. However, this company had a long track record of using measurements to generate change. They intentionally employed a form of Hawthorne Effecta; measurement was sometimes intended to catalyze improvement rather than to measure. At one point this company instituted a large metrics program that collected data from regular feed- a Named after a study done at Western Electric’s plant in Cicero, Illinois between 1924 and 1932 where significant productivity increases were measured after apparently minor or placebo changes in the workplace environment. The reasoning was that simply measuring work activities tends to improve them.