This quarterly publication is a peer-reviewed and archival journal that
covers reconfigurable technology,
systems, and applications on reconfigurable computers. Topics include
all levels of reconfigurable system
abstractions and all aspects of reconfigurable technology including platforms, programming environments
and application successes.
part of the discipline’s attraction, but
also complicates evaluation.
Across these variants, CS research
exhibits distinctive characteristics,
captured by seminal concepts: algorithm, computability, complexity,
specification/implementation duality, recursion, fixpoint, scale, function/
data duality, static/dynamic duality,
modeling, interaction…Not all scientists from other disciplines realize the
existence of this corpus. Computer
scientists are responsible for enforcing its role as basis for evaluation:
1. Computer science is an original
discipline combining science and engineering. Researcher evaluation must be
adapted to its specificity.
The CS Publication Culture
In the computer science publication
culture, prestigious conferences are
a favorite tool for presenting original
research—unlike disciplines where the
prestige goes to journals and conferences are for raw initial results. Acceptance
rates at selective CS conferences hover
between 10% and 20%; in 2007–2008:
˲ ICSE (software engineering): 13%
˲ OOPSLA (object technology): 19%
˲ POPL (programming languages): 18%
Journals have their role, often to
publish deeper versions of papers
already presented at conferences.
While many researchers use this op-
portunity, others have a successful
career based largely on conference
papers. It is important not to use
journals as the only yardsticks for
Books, which some disciplines do
not consider important scientific contributions, can be a primary vehicle in
CS. Asked to name the most influential publication ever, many computer
relies on assessment
of researchers by
scientists will cite Knuth’s The Art of
Computer Programming. Seminal concepts such as Design Patterns first became known through books.
2. A distinctive feature of CS publication is the importance of selective
conferences and books. Journals do not
necessarily carry more prestige.
Publications are not the only scientific contributions. Sometimes
the best way to demonstrate value is
through software or other artifacts.
The Google success story involves a
fixpoint algorithm: Page Rank, which
determines the popularity of a Web
page from the number of links to it.
Before Google was commercial it was
research, whose outcome included a
paper on Page Rank and the Google
site. The site had—beyond its future
commercial value—a research value
that the paper could not convey: demonstrating scalability. Had the authors
continued as researchers and come up
for evaluation, the software would have
been as significant as the paper.
Assessing such contributions is delicate: a million downloads do not prove
scientific value. Publication, with its
peer review, provides more easily de-codable evaluation grids. In assessing
CS and especially Systems research,
however, publications do not suffice:
3. To assess impact, artifacts such as software can be as important as publications.
Another issue is assessing individual contributions to multi-author work.
Disciplines have different practices
˲ Nature over a year: maximum coauthors per article 22, average 7.3
˲ American Mathematical Monthly: 6, 2
˲ OOSPLA and POPL: 7, 2.7
Disciplines where many coauthors
are the norm use elaborate name-or-dering conventions to reflect individual
contributions. This is not the standard
culture in CS (except for such common
practices as listing a Ph.D. student first
in a joint paper with the advisor).
4. The order in which a CS publication lists authors is generally not significant. In the absence of specific indications, it should not serve as a factor in
In assessment discussions, numbers
typically beat no numbers; hence the
temptation to reduce evaluations to
32 COMMUNICATIONS OF THE ACM | APRIL 2009 | VOL. 52 | NO. 4