novel and would need experimentation and experience.
Author Incentives. Our first mechanism addresses A1 using peer pressure.
It requires the conference to publish
not only the list of accepted papers, but
also, for each author, the author’s acceptance rate for that conference. For
example, if an author were to submit
two papers and none were accepted,
the conference would report an acceptance rate of 0, and if one was accepted,
the author would have an acceptance
rate of 0.5. Because no author would
like to be perceived to have a low acceptance ratio, we think this peer pressure
will enforce A1.
Our second mechanism addresses
A2 by raising the prestige of reviewing.
For example, conferences can have a
best reviewer award for the reviewer
with the best review scoreb or give them
a discount in the registration fee.
A more radical step would be to solve
A1 and A2 simultaneously by means of
a virtual economy, where tokens are
paid for reviews, and spent to allow
submission of papers.c Specifically, assuming each paper requires three reviews on average, reviewers are granted
one token per review, independent of
the conference, and the authors of a
paper together pay three tokens to submit each paper. We recognize that this
assumes all conferences expect the
same level of reviewing: one could pervert this scheme by appropriate choice
of reviewing venues. We ignore this fact
for now, in the interests of simplicity.
Continuing with our scheme, authors
of accepted papers would be refunded
one, two, or all their tokens depending on their review score. Authors of
the top papers would therefore incur
no cost, whereas authors of rejected
papers would have spent all three of
their tokens. Clearly, this scheme forces authors to become reviewers, and
to be careful in using the tokens thus
earned, solving A1 and A2.
We note that we obviously need to
make tokens non-forgeable, non-repli-cable, and perhaps transferable. E-cash
systems for achieving these goals are
b See the subsection Reviewer Incentives for details on review scoring.
c We have been informed that this scheme was
first suggested by Jim Gray, though we cannot
find a citation to this work.
the goal would be to
have a standard way
for members of the
community to review
and rank papers and
authors both before
and after publication.
well knownd—they merely need to be
adapted to a non-traditional purpose.
We recognize that regulating the economy is not trivial. Over-damping the
system would lead to conferences with
too few papers, or too few reviewers.
Underestimating the value of tokens
would only slightly mitigate the current problems, but would add a lot of
expensive overhead in the form of these
mechanisms. Moreover, it is not clear
how this system can be implemented.
Indeed, even if it was, it would not be
obvious how it can be bootstrapped,
or whether it would have unintended
consequences. One possible technique
would be to start by publishing signed
reviews and rely on technologies such
as Citeseer and Google Scholar as we
describe here in more detail.
Reviewer Incentives. We first discuss
dealing with R1 and R3. We propose that
authors should rate the reviews they receive for their papers, while preserving
reviewer confidentiality. Average (
non-anonymized) reviewer scores would
then be circulated among the PC. No
PC member wants to look bad in front
of his or her peers, so peer pressure
should enforce R1 and R3 (PC collusion
will damage the conference reputation).
Note that we expect most authors to rate
detailed but unfavorable reviews highly.
An even more radical alternative
is for reviews to be openly published
with the name of the reviewer. The
idea is that reviewers who are not willing to publish a review about a paper
d For example, David Chaum’s seminal work
“Blind signatures for untraceable payments,”
Advances in Cryptology Crypto ’ 82, Springer-
Verlag (1983), 199–203.
the 3rd International
conference on ubiquitous
contact: Won Kim,
the 36th annual
Symposium on principles of
Savannah, ga, uSa
contact: Zhong Shao,
on agents and
contact: Joaquim B. Filipe,
asia and South pacific design
contact: Yutaka tamiya,
the eleventh australasian
Wellington, New Zealand
contact: Margaret hamilton,
on advances in computing,
communication and control
contact: Srija unnikrishnan,
6th International conference
on computer graphics, virtual
reality, visualization and
Interaction in africa
pretoria, South africa
contact: alexandre hardy,