This quarterly publication is a peer-reviewed and archival journal that
covers reconfigurable technology,
systems, and applications on reconfigurable computers. Topics include
all levels of reconfigurable system
abstractions and all aspects of reconfigurable technology including platforms, programming environments
and application successes.
We believe the
as a whole desires
such a system.
however, we also
realize such a
system can also be
are perhaps inherently conflicted and
therefore should not be reviewing that
paper. Of course, there is a danger that
public reviews will be too polite, but
this will no doubt sort itself out over
time. The advantage of using true identities (verinyms) is that this handles R1,
R2, and R3. Alternatively, reviews could
be signed with pseudonyms, where
the pseudonyms could persist across
conferences. Nonce pseudonyms will
protect the nervous but prevent building reputation. There is a fundamental
balance between anonymity and credibility that we cannot hope to solve.
a Grand unified mechanism
A deeper examination of the incentive
structure suggests that perhaps the real
problem is that too much of the work
of submitting and selecting papers is
hidden. What if the entire process were
made open, transparent, and centralized? The goal would be to have a standard way for members of the community to review and rank papers and authors
both before and after publication, in a
sense adding eBay-style reputations to
Google Scholar or arXiv. All papers and
reviews would be public and signed,
with either pseudonyms or verinyms.
This system, would, in one fell swoop
achieve many simultaneous goals:
˲ Readers can draw their own conclusions (and tell the world) about the
quality of papers published by an author. This would encourage authors not
to submit bad papers (achieving A1).
˲ Community members who publish often and review rarely would be
exposed, achieving A2.
˲ We would see the reviews and the
names of the reviewers alongside the
paper, addressing R1, R2, and R3.
˲ We get to see whose opinions correlate well with our own to help decide
what papers to read.
˲ There is a good chance that very
good papers that end up as technical
reports or in smaller, less well known
conferences, are raised to the top by
˲ The system would allow continued
discussion and feedback about papers
even after they have been published
( 1) to help others (busy senior people,
and new people not knowing where to
start), and ( 2) to provide an opportunity
for others to participate in the discussion and debate.
We believe the academic community
as a whole desires such a system. However, we also realize such a system can also
be subverted. As with e-cash, the hardening of reputation systems to resist collusion and other attacks is well known,
and we merely need to import the appropriate machinery and techniques.
We have identified the underlying incentive structure in the paper publishing process and shown where these incentives lead to poor outcomes. These
insights allow us to propose several
mechanisms that give incentives to
authors, reviewers, and the community to do the “right thing.” We accept
that there has been much altruism in
the past, but in today’s resource-scarce
world, it may not be fair to rely on this
any longer. We recognize our work is
preliminary and leaves out many important details but nevertheless hope
these ideas will serve as the foundation of a fundamental rethinking of
the process. We hope at least some of
our proposals will make their way into
future conferences, workshops, and
Jon Crowcroft ( firstname.lastname@example.org) is the
marconi Professor of communications systems at the
university of cambridge, cambridge, u.K.
S. Keshav ( email@example.com) is a professor and
canada research chair in tetherless computing in
the cheriton school of computer science university of
waterloo, ontario, canada.
Nick McKeown ( firstname.lastname@example.org) is an associate
professor of electrical engineering and computer science
at stanford university, stanford, ca.
an earlier version of this material was published in
Proceedings of the Workshop on Organizing Workshops,
Conferences, and Symposia for Computer Systems