Mobile phones remove
the need for special

clear way to model a client’s outside world. In order to overcome this, we have used peer users—teenagers who are not suffering from mental health problems. In the early stages, they participate in relatively brief usability trials, and at a later stage in more involved evaluations. We generally recruit participants from inner-city schools and after-school clubs, as this matches the catchment area of the services we deal with and provides peer users who share similar socio-economic backgrounds as the ultimate end users.

Usability evaluations usually involve between five and eight young people and last around 30 minutes. When testing mobile systems, we have the teenagers use their own mobile devices. Running usability trials of initial prototypes with peer users has helped us to identify usability problems and provided feedback on designs through direct observation and dialogue. For us, using peer users to evaluate usability has been a critical solution because there is a significant question as to whether it is ethically sound to evaluate software, which might have significant usability problems, on clients who have come solely for treatment.

We have also used peer trials at a later stage of development to help evaluate the suitability, viability, and appeal of systems in realistic contexts. These trials are generally more involved, lasting between one and two weeks.

One benefit of this type of evaluation is that it can provide evidence, at an advanced stage of design, that the system

is engaging and is suitable for clinical use. Peer evaluations of Mobile Mood Diary compared existing methods of recording mood information with a new mobile phone system. A large and statistically significant effect was found for improved compliance, providing a strong case for proceeding to clinical evaluation [ 4].

One issue with peer user evaluations is that some applications and content may not be meaningful or appropriate for a more general teenage population. For peer-user evaluations of My Mobile Story, a therapeutic storytelling system based on a series of template stories, we provided peer users with a more generic “Mobile Reporter” template, which they used to create nontherapeutic stories. We thought that the existing formats, such as a “thoughts, feelings and behaviors” template, would not have much meaning for peer users.

Peer evaluations can increase confidence in the value of a system and its appropriateness for clinical evaluation. Therapists are more likely to accept and use a system that has strong supporting evidence.

Role playing. Role playing has been used in the design process to numerous ends including: to generate new concepts, to test design ideas on potential users, and as a technique for designers to help them “imagine better…to empathize better” [ 5].

Role playing is particularly suited to use in mental health care. Therapists are extremely comfortable with role plays. Most trainee therapists are required to practice role playing therapeutic sessions in their training.

equipment: Users
are already familiar
with their phone,
and using it in public
places would not
attract extra attention.
Most important, as
most teenagers
carry their phones
for most of the
day, it could provide
them with near-
constant support
when and where
they need it.

November + December 2009