is multidisciplinary by nature and already deals both with quantitative and qualitative research. Moreover, developing-region users differ in their employment and adoption of technology and thus comprise an important research direction for HCI professionals. In fact, HCI is arguably the easiest discipline within CS in which to work on ICTD research. Other areas with some inherent compatibility include systems, networking, databases, and AI. For example, in systems and networking, which are not as multidisciplinary as HCI and more quantitative in character, ICTD work is less natural, but it can still fit well when technology innovation and novel usage are involved. Examples from top-tier conferences include work on delay-tolerant networking, distributed storage, and novel MAC-layer protocols for long-distance WiFi. In these kinds of approaches to ICTD research, there must be a core technical nugget in addition to real-world deployments.

However, research requires a great deal of effort per published report, given the challenges of deployments; over the long term, ICTD researchers must aim to produce papers that are fewer in number but of higher impact. Moreover, it must be noted that ICTD tends to be driven by the solving of a problem rather than by technological innovation (often, in search of a problem), which means that many ICTD projects may not have a core technical nugget after all. Such problems, although highly satisfying to solve, are harder to claim as CS research.

For most projects, the real research is in actually discovering the specification of the problem via repeated fieldwork and deployments, which is similar in feel to iterative design in HCI. Although HCI is an exception, CS does not generally value problem discovery, especially if the end solution is simple (had we known to apply it). Researcher Matthew Kam went through such iteration to create effective educational games on cellphones:

3

• Evaluating 35 existing games for PCs with village students.

• Creating 10 test games for English as a second language (ESL) and testing them with 47 students.

•Studying 28 traditional village games to make the games more intui-

tive (compared to Western games).

• Implementing a new set of games.

•Leading an ongoing multiyear study on the educational value of these games.

Overall, this process has taken over four years and continues to this day.

ICTD is also developing its own community values over time. The clearest values so far are novelty and on-the-ground empirical results, both quantitative and qualitative. Less clear are the values surrounding repeatability, rigor, and generalizability, and least clear is how to merge the values of qualitative fields such as anthropology or ethnography with those of CS. Consider generalizability: CS values generalizable results as an indicator of potential impact, while qualitative researchers often emphasize the differences in groups or users and aim to broaden the dialogue. This leads to placing value on reusable technology frameworks, such as HCI toolkits, that can be customized and easily localized. We discuss one such framework here for mixed paper/phone applications. ICTD is also creating its own scholarly forums for discussing and disseminating this work. The International Conference on Information and Communication Technologies and Development and the International Conference on Social Implications of Computers in Developing Countries are two examples.

What about sustainability?

Long-term impact requires that ICTD projects be self-sustaining. First, after the researchers leave and the money stops flowing, does the project continue? Second, can it be replicated in other contexts?

Sustainability is challenging to define, and researchers disagree on the details. Most agree on financial sustainability as a key element: the deployment must produce enough income to at least cover its costs. In this view, philanthropy is acceptable for “kick starting” a project, but not for supporting routine operational costs. Similarly, while projects typically need not be wildly profitable, they should at least be cash-flow positive, as credit can be challenging.

The operating-cost issues add significant constraints to ICTD solutions.

They include not just the cost of the technology but also availability (uptime), power requirements, potential for theft, and logistics. One common approach to financial sustainability is to commercialize a solution; this has worked well for mobile phones and treadle pumps, for example. Even if a for-profit venture is not the purpose, researchers must essentially address the same issues of costs, cash flow, awareness (marketing), and ongoing support.

Operational sustainability is the capacity of the permanent staff to keep the project going technically (without the researchers). In theory, financial sustainability enables operational sustainability (by paying for it), but in practice it cannot do so all by itself. This is because of limits on local skills, supplies, and logistics. Solutions must be not only easy to use, but also amenable to straightforward diagnosis and repair with limited training.

Training costs are actually underrated. ICTD projects, particularly in rural areas, cannot view training as a one-time activity needed only when the project starts. Once trained, IT workers are often tempted to leave for better jobs in urban areas or other countries. Thus training is a recurring cost, and it must be short and effective.

These kinds of sustainability are fundamental to scaling a successful pilot project. Unfortunately, devel-opment-work pilots rarely turn into large-scale self-sustaining successes. Typically the pilot is small enough and has enough researchers involved (with their own support) that the financial and operational issues do not really hinder it. Thus the pilot is mostly useful to validate prototypes and assess community reactions. The understanding of financial sustainability requires a longer trial with detailed accounting and no hidden subsidies ( unless they are expected to continue at scale); it also requires dealing with replacement costs and expected equipment lifetimes. Operational sustainability must be evaluated via detailed tracking of problems and how and by whom they were solved. In both cases, the system evolves to reduce costs or simplify operation.

Finally, replication is the process of moving a successful project to a new environment. As developing regions