FORUM EVALUATION AND USABILITY

This forum addresses conceptual, methodological, and professional issues that arise in the UX field’s continuing effort to contribute robust information about users to product planning and design. — David Siegel and Susan Dray, Editors

their later opinions of technologies.

Therefore, there is much enthusiasm, and there are many good reasons, to work directly with children in both usability and user experience testing.

However, the involvement of children in usability studies and in evaluation studies is not without some difficulties. There are practical concerns around arranging studies and recruiting children, there are methodological concerns in terms of ensuring that children can contribute in meaningful ways, and there are ethical concerns around the meaning of the children’s participation.

The first two of these have been reasonably well covered in the literature. In practical terms, the classic workaround is to carry out evaluations in schools or in after-school clubs. It is true that developing relationships with schools takes time, and a research group has to invest in that school and ensure the work done does not “get in the way” of the curriculum, and also that it fits around the school’s schedule. Our group has a set of guidelines for working with schools (see http://www.chici.org/ schools). These guidelines include common sense things such as ensuring there is a back-up activity for children who finish early, making sure there are adequate electrical sockets (many schools are woefully short of these), and making sure the Internet connection can be used if needed.

Methodologically, the CCI community has put considerable effort into designing and modifying methods to be used with children in usability and evaluation studies. Many

The HCI community has long advocated the use of user studies to test and evaluate interactive systems. There is much to be learned by watching users interact with systems, both as novices and as experienced users.

It is generally thought that an expert inspection of a system is a poor substitute for user testing. One main argument for having users test systems is that they will typically do things that experts might not predict will happen. The expert inspector “guesses” what might happen based on his or her knowledge of the product and the intended users. This guesswork requires quite a mature understanding of the context of the system—its users, its uses, and its use.

The child computer interaction (CCI) community has a history of promoting children as active participants in research, design, and evaluation. The motivation for the involvement of children as evaluators has been that there is a considerable distance between children and any expert “guessers” evaluating the system on their behalf.

In our work in the ChiCI lab, we have engaged with children both as participants in design and as participants in evaluation. While a clear view on how children can contribute to design is still forming, we firmly believe there is no substitute in usability testing and user studies to having a product tested and experienced by a real child.

This belief comes not only from having
seen some of the crazy and wonderful
things children do with technologies, but
also from a logical analysis of how one
might test something. In our practice,
we have seen children act in ways that
could not have been predicted by an
expert. For instance, we’ve seen children
deliberately choosing to work with a
tablet (pre-rotating screens) held upside
down and children shaking digital pens
(for more digital ink!) when an ink trace
didn’t appear on the digital surface. The
logical case for user involvement is that,
provided there is no “risk,” it makes
sense to have at least some users try out
technologies and systems. It should be
noted, however, that children do not
unearth everything there is to be known
about a system— especially when they
are young—so there is still a need for a
parallel expert evaluation.

While watching children in natural interactions with technology can teach us much, a further reason to involve children in evaluation is to gather their opinions of products and features, especially when variations in these might affect motivation, performance, or fun. Again, this is not something that can easily be predicted by an expert, and our own work on measuring fun has shown several interesting things about the expectations of children and

Insights
→ Involving children in research studies
comes with a responsibility to let them
know what the work is about.

→ The excuses we use when thinking about who to work with need to be examined and challenged in an honest way.

→ It is hard to justify “using” children simply to get a paper written to gather academic citations.

Janet Read, University of Central Lancashire

Children as Participants
in Design and Evaluation