FORUM EVALUATION AND USABILITY
This forum addresses conceptual, methodological, and professional issues that arise
in the UX field’s continuing effort to contribute robust information about users to product
planning and design. — David Siegel and Susan Dray, Editors
their later opinions of technologies.
Therefore, there is much enthusiasm,
and there are many good reasons, to
work directly with children in both
usability and user experience testing.
However, the involvement of
children in usability studies and in
evaluation studies is not without
some difficulties. There are practical
concerns around arranging studies
and recruiting children, there are
methodological concerns in terms of
ensuring that children can contribute in
meaningful ways, and there are ethical
concerns around the meaning of the
children’s participation.
The first two of these have been
reasonably well covered in the
literature. In practical terms, the
classic workaround is to carry out
evaluations in schools or in after-school clubs. It is true that developing
relationships with schools takes time,
and a research group has to invest
in that school and ensure the work
done does not “get in the way” of the
curriculum, and also that it fits around
the school’s schedule. Our group has
a set of guidelines for working with
schools (see http://www.chici.org/
schools). These guidelines include
common sense things such as ensuring
there is a back-up activity for children
who finish early, making sure there
are adequate electrical sockets (many
schools are woefully short of these), and
making sure the Internet connection
can be used if needed.
Methodologically, the CCI
community has put considerable
effort into designing and modifying
methods to be used with children in
usability and evaluation studies. Many
The HCI community has long advocated the use of user studies to test and evaluate interactive systems. There is much to be learned by watching users interact
with systems, both as novices and as
experienced users.
It is generally thought that an
expert inspection of a system is a poor
substitute for user testing. One main
argument for having users test systems
is that they will typically do things
that experts might not predict will
happen. The expert inspector “guesses”
what might happen based on his or
her knowledge of the product and the
intended users. This guesswork requires
quite a mature understanding of the
context of the system—its users, its
uses, and its use.
The child computer interaction (CCI)
community has a history of promoting
children as active participants in
research, design, and evaluation. The
motivation for the involvement of
children as evaluators has been that
there is a considerable distance between
children and any expert “guessers”
evaluating the system on their behalf.
In our work in the ChiCI lab, we
have engaged with children both
as participants in design and as
participants in evaluation. While a clear
view on how children can contribute to
design is still forming, we firmly believe
there is no substitute in usability testing
and user studies to having a product
tested and experienced by a real child.
This belief comes not only from having
seen some of the crazy and wonderful
things children do with technologies, but
also from a logical analysis of how one
might test something. In our practice,
we have seen children act in ways that
could not have been predicted by an
expert. For instance, we’ve seen children
deliberately choosing to work with a
tablet (pre-rotating screens) held upside
down and children shaking digital pens
(for more digital ink!) when an ink trace
didn’t appear on the digital surface. The
logical case for user involvement is that,
provided there is no “risk,” it makes
sense to have at least some users try out
technologies and systems. It should be
noted, however, that children do not
unearth everything there is to be known
about a system— especially when they
are young—so there is still a need for a
parallel expert evaluation.
While watching children in natural
interactions with technology can teach
us much, a further reason to involve
children in evaluation is to gather their
opinions of products and features,
especially when variations in these
might affect motivation, performance,
or fun. Again, this is not something that
can easily be predicted by an expert,
and our own work on measuring fun
has shown several interesting things
about the expectations of children and
Insights
→ Involving children in research studies
comes with a responsibility to let them
know what the work is about.
→ The excuses we use when thinking about
who to work with need to be examined
and challenged in an honest way.
→ It is hard to justify “using” children
simply to get a paper written to
gather academic citations.
Janet Read, University of Central Lancashire
Children as Participants
in Design and Evaluation