field in two ways: how to assess the importance of conference publication, particularly compared to journal publication, and how to manage conferences to maximize the impact of the papers they publish. “Impact factor” (average citation rate) is the commonly used measure of the influence of a journal on its field. While nearly all computer scientists have strong intuition about the link between conference acceptance rate and a paper’s impact, we are aware of no systematic studies examining that link or comparing conference and journal papers in terms of impact.

This article addresses three main questions: How does a conference’s acceptance rate correlate with the impact of its papers? How much impact do conference papers have compared to journal papers? To what extent does the impact of a highly selective conference derive from filtering (the selectivity of the review process) vs. signaling (the message the conference sends to both authors and readers by being selective)? Our results offer guidance to conference organizers, since acceptance rate is one of the few parameters they can control to maximize the impact of their conferences. In addition, our results inform the process of evaluating researchers, since we know that computer scientists often defend the primary publication of results in conferences, particularly when being evaluated by those outside the field (such as in tenure evaluations). 2 Finally, we hope these results will help guide individual researchers in understanding the expected impact of publishing their papers in the various venues.

 

Data and methodology We based our study on ACM Digital Library metadata for all ACM conference and journal papers as of May 2007, as well as on selected other papers in the ACM Guide to Computing Literature for which metadata was available. Since there is no established metric for measuring the scientific influence of published papers, we chose to estimate a paper’s influence as the number of times it was cited in the two years following publication, referred to as citation count or simply as impact. We excluded from this count “self-citation” in subsequent papers by the authors

overall, the
conference papers
had an average
two-year citation
count of 2. 15, and
the journal papers
had an average
two-year citation
count of 1. 53.

of the original work. Using citations as a measure of scientific influence has a long tradition, including the journal-impact factor. 1 We chose two years as a compromise between measuring long-term impact and the practical importance of measuring impact of more recent work. 1 Less than two years might be too short for the field to recognize the worth of a paper and cite it. More than two years would have excluded more recently published papers from our analysis due to insufficient time after publication, so would not have allowed us to include the current era of widespread use of the Digital Library.a Finally, since our data source was limited to the metadata in the ACM Guide, our analysis considered only citations from within that collection and ignored all citations from conferences and journals outside of it; this was a pragmatic constraint because, in part, other indexing services do not comprehensively index conference proceedings. While it means that all our numbers were underestimates and that the nature of the underestimates varied by field (we expected to significantly underestimate artificial intelligence and numerical-computation papers due to the large number of papers published by SIAM and AAAI outside our collection), such underestimates were not biased toward any particular acceptance rate in our data set.b a To ensure that the two-year citation count was reasonable, we repeated this analysis using four- and eight-year citation counts; the distributions and graphs were similar, and the conclusions were unchanged.