�����
This quarterly publication is a peer-reviewed and archival journal that covers reconfigurable technology, systems, and applications on reconfigurable computers. Topics include all levels of reconfigurable system abstractions and all aspects of reconfigurable technology including platforms, programming environments and application successes.
�����
are perhaps inherently conflicted and therefore should not be reviewing that paper. Of course, there is a danger that public reviews will be too polite, but this will no doubt sort itself out over time. The advantage of using true identities (verinyms) is that this handles R1, R2, and R3. Alternatively, reviews could be signed with pseudonyms, where the pseudonyms could persist across conferences. Nonce pseudonyms will protect the nervous but prevent building reputation. There is a fundamental balance between anonymity and credibility that we cannot hope to solve.
A deeper examination of the incentive structure suggests that perhaps the real problem is that too much of the work of submitting and selecting papers is hidden. What if the entire process were made open, transparent, and centralized? The goal would be to have a standard way for members of the community to review and rank papers and authors both before and after publication, in a sense adding eBay-style reputations to Google Scholar or arXiv. All papers and reviews would be public and signed, with either pseudonyms or verinyms. This system, would, in one fell swoop achieve many simultaneous goals:
˲ Readers can draw their own conclusions (and tell the world) about the quality of papers published by an author. This would encourage authors not to submit bad papers (achieving A1).
˲ Community members who publish often and review rarely would be exposed, achieving A2.
˲ We would see the reviews and the
names of the reviewers alongside the paper, addressing R1, R2, and R3.
˲ We get to see whose opinions correlate well with our own to help decide what papers to read.
˲ There is a good chance that very good papers that end up as technical reports or in smaller, less well known conferences, are raised to the top by popular acclaim.
˲ The system would allow continued discussion and feedback about papers even after they have been published ( 1) to help others (busy senior people, and new people not knowing where to start), and ( 2) to provide an opportunity for others to participate in the discussion and debate.
We believe the academic community as a whole desires such a system. However, we also realize such a system can also be subverted. As with e-cash, the hardening of reputation systems to resist collusion and other attacks is well known, and we merely need to import the appropriate machinery and techniques.
We have identified the underlying incentive structure in the paper publishing process and shown where these incentives lead to poor outcomes. These insights allow us to propose several mechanisms that give incentives to authors, reviewers, and the community to do the “right thing.” We accept that there has been much altruism in the past, but in today’s resource-scarce world, it may not be fair to rely on this any longer. We recognize our work is preliminary and leaves out many important details but nevertheless hope these ideas will serve as the foundation of a fundamental rethinking of the process. We hope at least some of our proposals will make their way into future conferences, workshops, and publications.
Jon Crowcroft ( jon.crowcroft@cl.cam.ac.uk) is the marconi Professor of communications systems at the university of cambridge, cambridge, u.K. S. Keshav ( keshav@uwaterloo.ca) is a professor and canada research chair in tetherless computing in the cheriton school of computer science university of waterloo, ontario, canada. Nick McKeown ( nickm@stanford.edu) is an associate professor of electrical engineering and computer science at stanford university, stanford, ca. an earlier version of this material was published in Proceedings of the Workshop on Organizing Workshops, Conferences, and Symposia for Computer Systems (WOWCS 2008).
References:
Archives