Dear kV,

After spending the last year buying and deploying a set of monitoring systems in my company’s production network, we hit our first bug in the manufacturer’s system software. After we reported the bug, the manufacturer asked us to upgrade the systems to the latest release. It turns out the upgrade process requires us to reset the devices to their factory defaults, losing all our configuration information on each system and requiring a person to reenter all of the configuration data after the upgrade.

At first, we thought this might be something required only for this particular upgrade, which crosses a major revision, but we have since learned we must reenter our configuration data each and every time we upgrade the software on these systems. I suppose we should have asked this question before we bought the systems, but it just never occurred to us that anyone would sell a box purporting to act as an appliance that could not be easily upgraded in the field. One of the other members of my group suggested we just return the boxes and ask for our money back, but, depressingly enough, these are the best systems we have found for network monitoring.

Down on upgrades
Dear Down,

Even in the worst-designed products—and I have used enough of those—there is usually some bit of perl that takes the old configuration data and turns it into something that is mostly valid for the latest revision.

As a matter of fact, many years ago I worked on a networking switch project and the first thing the systems team (which was responsible for getting a reasonable operating system and applications onto the box) did was to come up with a way to field-upgrade the system. Anyone who has ever configured a switch or router knows you do not just toss out the configuration on an upgrade.

The sad part is that doing this correctly just is not that difficult. Most embedded systems now use stripped-down Unix systems such as Linux or the BSDs, all of which store their configurations in well-known files. Granted, this is not the nicest way to store configuration data, because it tends to be a bit scattered, but it is not that difficult to write a script that can handle differences between the versions and reconcile them. On FreeBSD there is etcupdate, and Linux has etc-update and dispatch-conf. In a properly designed system the configuration would likely be stored in a simple database or an XML file, both of which are field-upgradable with fairly simple scripts.

These sorts of issues are what differentiate an appliance that can be deployed and maintained with little human interference from a system that has to be diddled constantly to keep it happy. It is a sad fact that many programmers and engineers do not think much of appliances and are more likely to think their users should “man up” and spend their time making up for the original designer’s lack of forethought.

I still remember one of the first all-
digital stereo systems I ever saw. It was
designed around a Sun workstation
and cost on the order of $15,000. It was
obvious from the moment you looked
at the controls that little or no thought
had been put into what people really
wanted out of a sound system. What
most people want out of a stereo is
good-quality sound with a minimum of
button pushing to get what they want.
What the system I saw presented was
a lot of button pushing for about the
same quality of sound I could get out
of an amplifier and a CD player. If the
user interface was horrific, it was noth-
ing compared with the system perfor-
mance. The box crashed three times be-
fore I finally walked out of the store. The
one thing I got from that experience was
an understanding that systems and
products are not the same thing.

Related articles on queue.acm.org

The Seven Deadly Sins of Linux Security

Bob Toxen

http://queue.acm.org/detail.cfm?id=1255423

A Conversation with Chris DiBona

http://queue.acm.org/detail.cfm?id=945130

Closed Source Fights Back

Greg Lehey

http://queue.acm.org/detail.cfm?id=945126

George V. neville-neil ( [e-mail]) is the proprietor of neville-neil Consulting and a member of the ACM Queue editorial board. he works on networking and operating systems code for fun and profit, teaches courses on various programming-related subjects, and encourages your comments, quips, and code snips pertaining to his Communications column.