Figure 4 shows the percentages of papers within each group where citation count was above a certain threshold. The bottom bands (reflecting papers cited more than 10, 15, or 20 times in the following two years) show high-acceptance-rate conferences have few papers with high impact. Also notable is the fact that about 75% of papers published in >55%-acceptance-rate conferences were not cited at all in the following two years.

Addressing the second question—
on how much impact conference
papers have compared to journal pa-
pers—in Figures 3 and 4, we found that
overall, journals did not outperform
conferences in terms of citation count;
they were, in fact, similar to conferenc-
es with acceptance rates around 30%,
far behind conferences with accep-
tance rates below 25% (T-test, T[7603]
= 24. 8, p<.001). Similarly, journals pub-
lished as many papers receiving no cita-
tions in the next two years as conferenc-
es accepting 35%–40% of submissions,
a much higher low-impact percentage
than for highly selective conferences.

figure 3. average citation count by acceptance rate within two years of publication.

Avg. number of Citation for Conferences Avg. number of Citation for Journals

4

3. 5

3

citation count

2. 5

2

1. 5

1 0.5 0

10%–15% 15%–20% 20%–25% 25%–30% 30%–35% 35%–40% 40%–45% 45%–50% 50%–55% 55%–60%

acceptance Rate

figure 4. citation count distribution by acceptance rate within two years of publication.

1+

6+

11+

16+ 21+

0.8

0.7

0.6

fraction of Papers

0.5

0.4

0.3

0.2

0.1

10%–15% 15%–20% 20%–25% Journal 25%–30% 30%–35% 35%–40%40%–45% 50%–55% 55%–60% 45%–50% 0

acceptance Rate/Venue type

the third question—on the extent the impact of a highly selective conference derives from filtering vs. signaling—as this correlation can be attributed to two mechanisms:

Filtering. A selective review process
filters out low-quality papers from the
submission pool, lowering the accep-
tance rate and increasing the average
impact of published papers; and
We performed this separation by
normalizing the selectivity of filtering to
the same level for different conferences.
For example, for a conference accepting
90 papers at a 30% acceptance rate, the
best potential average citation count
the conference could have achieved
by lowering the acceptance rate to,
say, 10% for the same submission pool
would be the average citation count of
the top 30 most-cited papers of the 90
accepted (presumably the 30 best pa-
pers of the original 300 submitted). We
treated these 30 papers as the top 10%
best submissions in the pool; other sub-
missions were either filtered out during
the actual review or later received fewer
citations. Their citation count was thus
an upperbound estimate of what might
be achieved through stricter filtering,
assuming conference program com-
mittees were able to pick exactly the
submissions that would ultimately be
the most highly cited. Using this nor-
malization, we compared the same top
portions of submission pools of all con-
ferences and evaluated the effect of sig-
naling without the influence of filtering.
We normalized all ACM conferences in
Figure 3 to a 10% acceptance rate and
compared the citation counts of their
top 10% best submissions; Figure 5
(same format as Figure 3) outlines the
results. We excluded transactions and
journals, as we were unable to get actual