otherwise be using to create code. If
developers are rewarded only for adding functionality, that’s going to end
up crippling the code-review process,
which in turn will almost certainly have
an adverse effect on the maintainability of the code that’s generated.
CB: One thing I would like to add is
that the code-review process we now
have at Microsoft has more or less
grown organically—through experimentation—from the grassroots. I
mention this only because I think it
might also work well for smaller companies, instead of having some process
that’s mandated from the top down.
Also, each product group at Microsoft does code reviews a little differently, with each group using its own
set of policies that have essentially
come together organically. While this
probably won’t come as a ground-breaking revelation, it can definitely
be said that there is no one-size-fits-all solution for code reviews. This
only serves to reinforce the importance of being willing to let your approach evolve organically such that it
ends up fitting in with your work processes with the least amount of friction while putting the lightest burden
possible on your developers.
Another important point is something Michaela talked about earlier,
which is that treating code review as a
first-class citizen—just as many companies are likely to treat testing—is
probably the best way to get the most
bang for your buck. If, instead, it becomes something you are just expected to do, like flossing your teeth daily,
then you’ll find people aren’t going to
embrace it. But if you say this is important and so will be tracked and evaluated, then people are likely to respond
to that. Certainly, that’s how it has
worked out here.
And then the other thing I would
add is that it’s instructive to think in
some depth about what it is you’re really looking to get out of code reviews.
Then, of course, you should also think
about how you can go about measuring that. To the degree that you can
track those metrics and set targets,
you’re always going to achieve more.
© 2019 ACM 0001-0782/19/2 $15.00.
to workflow. Still, there are times when
two unrelated concerns end up getting
crammed into a single review, so we
are hoping we will be able to untangle
some reviews by automatically splitting those concerns into two smaller
reviews. On average, that ought to lead
to better turnaround times, as well as
LP: Have you taken any steps to get
development teams to focus their
code-review time on correctness and
content versus style? Have any tool
changes or process changes been implemented toward that end?
JC: We haven’t done a proper study
of that, but there is a team here that’s
done something along those lines.
This is something that had to do with
some factoring changes they considered to be low-risk—such as the renaming of methods or local variables.
For example, this might involve putting a special tag on a review to say,
“We don’t really need to have two
people look at this. One is enough
since it’s very unlikely we’ll have any
functionality issues here.” Modest as
that might seem, it can also prove profound since it turns out there are many
changes like this floating through a
legacy system—clogging the system.
The thing to remember is that it’s
not just about making one change go
faster, since what you’re dealing with
here is a pipeline of changes—
meaning that any change you can redirect to
a lighter-weight path is going to lower
the load on your key people and get it
out of the way of other changes waiting to be reviewed. That’s just the sort
of thing that makes for a more efficient
system all the way around.
TC: With an eye to the people outside of Microsoft that don’t have your
tooling, do you have any recommendations from your experience that might
JC: I would say the one thing to
recognize is that comments related
to maintainability are primarily what
you are going to get out of the code-
review process. Contrary to popular
opinion, locating bugs is not the pri-
mary outcome. The other important
thing to bear in mind is that the small-
er a review is, the better it’s going to
be. In our case, we’ve found that if a
review contains more than 20 files,
it’s too big already. In fact, from our
study of all the data at our disposal,
we’ve concluded that for more than
20 files the density and usefulness
of comments degrades significantly.
This is actually more a rule of thumb
than a precise limit, but it is useful to
keep in mind.
Also, if your organization has data
from past reviews, I would suggest in-
vesting in a recommender system that
can help make some of the administra-
tive steps a little less tedious. You can
even use these systems to automatical-
ly address some of your maintainabil-
ity issues, which is something we’re
starting to get into these days. That is,
you can imagine that some of these
maintainability issues are essentially
things that might be autodiscovered
and flagged, which means you then
don’t have to expend any human re-
sources to get this accomplished.
Another thing, as we just dis-
cussed, is the idea that two signoffs
on every change might be too many.
If you look at the distribution of com-
ments made by either the first or the
second reviewer, you’ll find that your
first reviewer typically discovers the
most egregious problems. In many
cases, waiting for a second reviewer
to corroborate those findings before
allowing the commit into the main
source tree might be less efficient.
MG: My biggest takeaway from the
survey is to always make the burden of
code reviews just as small as you possibly can. Part of that comes down to
having a good code-review process that
enables and encourages comments
that can be easily reviewed.
Another important consideration
has to do with supporting the reviewers themselves by giving them advance
notice about any reviews that might be
coming up and giving them enough
context so they will be able to dive
right into a review without having to
figure all that out for themselves. Doing what you can to reduce the size of
reviews can also be helpful. But I think
what is really important is to make the
reviews just as uncomplicated as possible, since, otherwise, you may end up
with reviewers who have no clue about
where even to start.
Also, organizations need to show
they recognize the value of code reviews
since there’s no question that they take
away from the time developers could