Communications of the ACM

editor’s letter
PERIODIC REVIEW OF academic units is a standard practice in academia. The goal of such review is to contribute to quality enhancement by
providing the unit and the institution
with a clear assessment of the unit’s
strengths and weaknesses. Typically,
such a review consists of a thorough
self-study by the reviewed unit, which
is followed with a visit by a commit-
tee of experienced academicians. Af-
ter the visit, the committee submits
a report, which combines an incisive
analysis with sage advice for unit im-
provements. Then what? Usually, not
much, I am afraid. It is rare to see such
a review resulting in a truly significant
quality enhancement.

I recently had the opportunity to be a member of a committee that reviewed all 17 computer science programs in Israel. The 17 programs range from graduate programs at world-famous research institutions such as the Weizmann Institute of Science, to small undergraduate programs at small colleges such as Tel-Hai College at the Northern Galilee. The review project was carried out over a period of a year and a half, included three visits to Israel, and was, overall, a considerable effort. (The reports are available at http://che.org.il/?page_id=34025.)

Then what? Not much, I am afraid.

Why is it that academic-unit reviews accomplish so little in spite of the significant effort both by the reviewed units and reviewing committees? And why do we continue to conduct these reviews in spite of their meager results?

The answer to the second question
is clear, I believe. Unlike businesses,
which can be measured by their bot-
tom line, academic institutions have
multiple, difficult to measure, and of-
ten conflicting goals. They are unified
by the nebulous concept of academic
excellence. The pursuit of excellence
must be visible, and academic-unit re-
views serve this purpose. But why do
they accomplish so little? There are
three main reasons, I believe.

Cultural Barriers: Culture is defined as “the way of life, especially the general customs and beliefs, of a particular group of people at a particular time.” Each academic unit has its own unique culture. Culture usually creates social cohesion by means of shared expectations, but it can also be a barrier to change. Quality enhancement requires an academic unit to change the way it runs its business, but culture is persistent and change is difficult. Israel has become famous as the “Start-Up Nation,” with a thriving high-tech sector. The computer science programs in Israel create the educated work-force that underpins the Israel high-tech sector. Yet computer science units in Israel are generally spin-offs from mathematics units. As a result they tend to be highly theoretical in their research focus. The Review Committee’s report called for a better balance between theoretical and experimental research, but such a change runs against an entrenched culture.

Institutional Barriers: Academic-unit reviews typically focus on low-level units, such as departments.

But departments do not operate in a vacuum. Departments typically belong to schools, and the operations of a department must be considered in the context of the school in which it is housed. But reviews almost always focus narrowly on departmental operations, missing the bigger context.

Furthermore, academic units have
multiple stakeholders—academic
staff members, administrative staff
members, students, and higher-level
academic administrators—with di-
verse sets of interests. Almost every
proposed quality-enhancing change
runs against the interests of some
stakeholder. Thus, there are institu-
tional barriers to change. It takes very
strong institutional support for quality
enhancement—as well as committed
leadership at all levels—to implement
recommended changes. For example,
in one institution in Israel we found
five distinct computer science pro-
grams and called for an institutional
reorganization to increase synergy and
efficiency. But such reorganization
runs against several institutional in-
terests and is unlikely to happen.

Follow-Up Process: The reviews produced by review committees often offer insightful diagnoses of the weaknesses discovered. They usually also offer recommendations for improvements, but they usually do not offer detailed action plans, as reviewers lack the intimate institutional knowledge required to develop such plans. But most institutions do not have a robust follow-up process to ensure that a detailed action plan to deal with the recommendations is developed by department and the school, is carried out to completion, and then re-reviewed to ensure actual improvement. Thus, academic-unit review reports mostly gather dust rather than affect change.

So what is to be done? Should we abandon the ritual of academic-unit review? I view the pursuit of academic excellence as a hallowed academic value, and reviews can serve an important role in such a pursuit, but they should not be undertaken without utmost commitment to the process and fortitude to carry it out.