Communications - April 2008

ACADEMIC INSTITUTIONS TO BENCHMARK THEIR RESEARCH PERFORMANCE AND TO IDENTIFY TOP PERFORMERS FOR POTENTIAL COLLABORATION.

should be of interest to academics and professionals alike. The reported results will allow firms and academic institutions to benchmark their research performance and to identify top performers for potential collaboration.

This study will also provide researchers with important indicators of IS research, for example, overall productivity and impact, evolution over the last 10 years, internationalization, concentration, and the level of industry involvement.

Productivity refers to the total IS research publications output. Consistent with prior studies we measure it with the adjusted count (fraction based on the number of co-authors) of research articles published by IS researchers in top journals in IS and referent disciplines. In addition, we also account for the impact of the research output, that is, the level of dissemination, which is measured with the adjusted impact scores (yearly impact ratios of the journal where a research article gets published). The impact ratios are reported by the Science Citation Index and the Social Sciences Citation Index.

Although more “objective” than perceptions, adjusted counts and impact ratios do not fully account for the rigor and prestige of the journals. Additional objective measures (for example, acceptance ratios), however, are not readily available and subjective measures (for example, journal rankings based on perceptions) are usually controversial. A quick Web search reveals that except for very few top journals, institutional rankings of IS journals differ significantly. We therefore opted not to mix objective and subjective measures, while acknowledging the limitations of our approach.

The journal selection is based on the most recent

citation-based ranking [ 1], with minor differences. We chose a cut-off of 0.1 for the impact ratio, removing

Journal of Computer Information Systems (only 0.034 when listed). We also removed IEEE Computer,

Asia as it was not clear which

journal/magazine the authors meant. We could identify several journals/ magazines with such a name,

but all having impact ratios different from the one reported in [ 1]. Furthermore, most previous rankings did not include such a journal, but listed instead the IEEE Transactions [ 3, 4]. We therefore consistently included IEEE Transactions on Engineering Management, IEEE Transactions on Systems, Man, and Cybernetics, and IEEE Transactions on Software Engineering [ 3, 4]. We also added the Journal of the Association of Information System (JAIS). Although this journal has no impact ratio due to its relatively short history, it is generally regarded as a rising top-ranked journal. Indeed, several recent studies have included JAIS as one of the important IS journals, for example, [ 2-4]. Without an impact ratio, this journal counts for productivity calculation only. We ended up with 25 IS journals. For referent disciplines, we included the top 11 journals from the original ranking.

The collection of information about all articles published in the selected 36 journals during the last decade (1995–2004) took 40 person-months. To prevent errors, we incorporated several validity checks within the data entry system. We also assigned three individuals to check all entries and reconcile discrep-ancies. The resulting database consists of 18,711 research articles written by 24,517 authors from 4, 111 institutions. The identification of research articles is based on the ISI classification. An article is included in the analysis if it has at least one IS co-author.