Noonan, worked to allow some processing of email reviews.
• Ratings and reviewer comments would be compiled
(manually for most symposia), papers ranked and selected,
and comments returned to authors via the postal system.
Overall, the paper submission and reviewing process was
dictated, largely, by paper-based technology, with papers and
reviews transmitted by postal mail.
• SIGCSE 2000–SIGCSE 2008: In 1999 when planning
SIGCSE 2000, Symposium Co-chairs, Nell Dale and Lillian
“Boots” Cassel, asked Program Chair, Henry Walker,
about the possibility of allowing the online submission
and reviewing of papers. In response, over the summer
1999, Walker worked with two students, Weichao Ma and
Dorene Mboya, and two technical consultants, Wayne
Twitchell and Theresa P. Walker, to develop a system that
allowed either online or paper submissions. (At this stage,
electronic submission was encouraged, but paper served
as an alternative for those accustomed to the traditional
approach or those with limited internet access.) The next
year, almost all papers were submitted electronically, and
with SIGCSE 2002, electronic submission was required.
Within a few years, John Dooley joined the collaborative
team, and this system was used extensively for over a
decade. Pragmatically, the introduction of an online system
had several anticipated and important consequences.
Logistics of submission, paper handling, reviewer
assignment, paper distribution to reviewers, recording of
reviews, paper selection, and feedback to reviewers were
The number of reviewers per paper could be expanded,
since paper duplication and mailing were no longer required.
After reviews were completed, they could be distributed
easily to authors, allowing efficient refining of accepted
papers based on feedback.
posium were evaluated and selected through a peer-review process, and, as previously reported, only about one-third of submissions have been accepted for almost three decades. Details of the
review and selection process, however, have evolved substantially
through this period, following at least two challenging principles.
• Program committees have encouraged all interested
SIGCSE members to collaborate in the reviewing process.
In contrast to many conferences, in which reviewers are
screened by conference leaders, SIGCSE has valued the
wide range of perspectives of its full membership.
• Program committees, of course, sought to accept the papers
most appropriate, based both on overall quality and on the
desire to organize a wide-ranging program. For example, if
numerous papers discussed CS1/CS2 and few considered
operating systems, a balanced program might include the
best of both CS1/CS2 and operating systems, even if overall
ratings for one area might be higher or lower than those for
papers in the other area.
As an example of the challenges of paper selection, SIGCSE
2000 received 219 submissions, and each paper was distributed
to at least four of 482 reviewers. With the scope of this process,
attention needed to be paid to different perspectives and priorities among the many reviews received—altogether a challenging task for symposium leadership.
As might be expected, details of the submission and reviewing
process have evolved substantially over the years, often reflecting practical opportunities and limitations of technology as it
evolved. Although program committees refined the submission
and review process regularly over the years, logistics, policies,
and procedures may be divided into roughly three main phases.
• Reviewing in the 1990s: Throughout the 1990s, paper
submission and review primarily utilized physical paper.
• An author would print n copies of a paper and mail the
copies to the program chair. The submission deadline might
be based on when the mail was postmarked, when the
submission was received, or some combination.
• The program chair would retain one copy for later
processing, assign reviewers, and mail the remaining n − 1
copies to reviewers.
• Reviewers would send reviews to the program chair.
Originally all correspondence was done through the postal
system, although SIGCSE 1999 Program Chair, Robert
Program committees have
encouraged all interested SIGCSE
members to collaborate in
the reviewing process. In
contrast to many conferences,
in which reviewers are
screened by conference leaders,
SIGCSE has valued the wide
range of perspectives of its
Figure 4: Paper Acceptance Rates