the reviewers were interested in, it was
the comments that added to the value
of the review. On the other hand, some
comments just increase the burden of
the code review and slow down the development process. So then we wanted
to know what kinds of comments they
found most useful, since we could then
start thinking about how to encourage
and lend greater emphasis to those.
JC: Just as this interesting question
of usefulness led to practical implications later on, the same might be said
of the work that was done to look into
other process-related questions. For
example, how many people should
you include in a code review? Is there
a number beyond which it becomes
counterproductive? We all intuitively
feel that smaller reviews are better, but
where exactly to draw that line? And
what’s the optimal amount of time to
allow for a review?
MG: Another interesting thing we
found is that, while the popular notion is that code reviews are mostly
about finding bugs, only a very small
percentage of the code-review comments we studied actually had anything to do with bugs at all. In fact,
most of the comments were about
structural issues and style problems.
Sometimes they were even about really minor issues, like spelling. Basically, what we found was that many
reviewers were using their commenting platform to discuss these issues
and share their knowledge.
We found it very enlightening to
categorize these comments and do
LP: Which problems did you decide to
attack first?
JC: Most of the issues we chose to
focus on were process oriented. The
tool itself is quite flexible and adaptable to practically any process. We
spent a lot of time trying to understand the benefits of code review and
what was getting in the way of achieving those advantages. Also, we wanted
to understand how the existing code-review tool was being used. We were
interested in learning more about the
costs and the turnaround times in
hopes we would be better able to see
what the drivers were.
MG: Also, one of the issues we looked
at was how to create a reviewer recommender since programmers had been
complaining to us about how difficult
it was to find the right people to look
over their code. Chris started working
on a tool that would deliver a listing
of people with the expertise to match
the sorts of problems addressed by
your code, along with suggestions as to
which of these people you might want
to add to a review.
Something else Chris and I studied
for a while was code-review usefulness.
That wasn’t a problem we were trying
to solve, of course, but we did want to
understand which aspects of code reviews tend to be most valued by engineers—that is, by both reviewers and
programmers. What did they see as being most useful? It didn’t take us long
to conclude that it was not the mere decision to accept or rework the code that
CHRISTIAN BIRD
The code-review
process we now
have at Microsoft
has more or less
grown organically—
through
experimentation—
from the grassroots.
