DoI: 10.1145/1965724.1965741

Article development led by queue.acm.org

A discussion with Nico Kicillof,
Wolfgang Grieskamp, and Bob Binder.

ACM CASE STuDY Microsoft’s Protocol Documentation Program: Interoperability Testing at Scale

in 2002, miCrOsOFt began the difficult process of verifying much of the technical documentation for its Windows communication protocols. The undertaking came about as a consequence of a consent decree Microsoft entered into with the U.S. Department of Justice and several state attorneys general that called

for the company to make available cer-
tain client-server communication pro-
tocols for third-party licensees. A series
of RFC-like technical documents were
then written for the relevant Windows
client-server and server-server commu-
nication protocols, but to ensure in-
teroperability Microsoft needed to ver-
ify the accuracy and completeness of
those documents. From the start, it was
clear this wouldn’t be a typical quality
assurance (QA) project. First and fore-
most, a team would be required to test
documentation, not software, which is
an inversion of the normal QA process;
and the documentation in question
was extensive, consisting of more than
250 documents— 30,000 pages in all.
In addition, the compliance deadlines
were tight. To succeed, the Microsoft
team would have to find an efficient
testing methodology, identify the ap-
propriate technology, and train an
army of testers—all within a very short
period of time.