WoLFGAnG GRIESKAMP
one of the
challenges for
our project was
to make sure
the functions
performed by
Windows servers
could also be
performed by
other servers.

team arrived at an approach to that enormous testing challenge. More specifically, it focuses on one of the testing methodologies used—model-based testing—and the primary challenges that have emerged in adopting that approach for a very large-scale project. Two lead engineers from the Microsoft team and an engineer who played a role in reviewing the Microsoft effort tell the story.

Now with Google, Wolfgang Grieskamp at the time of this project was part of Microsoft’s Windows Server and Cloud Interoperability Group (Winterop), the group charged with testing Microsoft’s protocol documentation and, more generally, with ensuring that Microsoft’s platforms are interoperable with software from the world beyond Microsoft. Previously, Grieskamp was a researcher at Microsoft Research, where he was involved in efforts to develop model-based testing capabilities.

Nico Kicillof, who worked with Grieskamp at Microsoft Research to develop a model-based testing tool called Spec Explorer, continues to guide testing efforts as part of the Winterop group.

Bob Binder is an expert on matters related to the testing of communication protocols. He too has been involved with the Microsoft testing project, having served as a test methodology consultant who also reviewed work performed by teams of testers in China and India.

For this case study, Binder spoke with Kicillof and Grieskamp regarding some of the key challenges they’ve faced over the course of their large-scale testing effort.

BoB BInDER: When you first got involved with the Winterop Team [the group responsible for driving the creation, publication, and QA of the Windows communication protocols], what were some of the key challenges?

nICo KICILLoF: The single greatest challenge was that we were faced with testing protocol documentation rather than protocol software. We had prior expertise in testing software, but this project called for us to define some new processes we could use to test more than 30,000 pages of documentation against existing software implementations already released to the world at large, even in some cases where the original developers were no longer with Microsoft. And that meant the software itself would be the gold standard we would be measuring the documentation against, rather than the other way around. That represented a huge change of perspective.

WoLFGAnG GRIESKAMP: What was needed was a new methodology for doing that testing. What’s more, it was a new methodology we needed to apply to a very large set of documents in relatively short order. When you put all that together, it added up to a really big challenge. I mean, coming up with something new is one thing. But then to be faced with immediately ap-

IllustratIon Base D on a Photogra Ph courtesy of Wolfgang grIeskaMP