John [not his real name] was a 15-year old male from a single-parent family, and considered a potential suicide risk. He agreed to use the Mobile Mood Diary for two weeks, but continued using it for several months. He recorded 85 entries over 100 days, and opened the diary many times—possibly to review and reflect on previous entries. John’s parent was finding it very difficult to cope with his son’s struggles. The therapists suggested that he use the diary “to see if there was a correspondence between the child’s and parent’s moods.”
The parent recorded 68 mood entries over 75 days. This allowed discussion of how the client and the parent influenced each other, providing new insight to both of them, and was an unforeseen use of the system.
Before meetings with the psychiatrist, the client and therapist would login to the website and print out the chart for each person. This was used to support discussions and decisions regarding risk factors (suicide) and medication.
November + December 2009
[ 5] Burns, C., Dishman,
E., Verplank, W.,
and Lassiter, B.
“Actors, Hairdos,
and Videotape.” CHI
Conference Companion,
ACM Press (1995):
119-120.
interactions
In order to simulate the use of therapeutic systems, we provide therapists with simple role cards, which contain basic character details. For the therapist role, this includes information such as experience of technology, years of experience, and their attitude toward technology. For the client roles, it can include elements such as their name, age, mental health difficulty, current mood, and attitude to technology. The client role cards are based on realistic client scenarios and, where possible, actual client cases.
We have used role playing primarily to evaluate the suitability of our systems to the work practice of therapists. It has also proved useful in providing therapists with a safe environment in which to get to know the technology—a setting where they can make mistakes, ask “silly” questions, and rehearse important scenarios like introducing the technology to a teenager. Moreover, the use of role playing has helped to “debug” the clinical system, assisting in identifying practical problems prior to clinical use.
Observing role-plays has provided us with insights into the therapists’ existing work practices, and has been invaluable in helping to understand how therapists would instinctively use the tool.
Therapist reviews. Just before clinical pilots of a system, we complete a therapist review process. This involves therapists, from a wide range of theoretical backgrounds, evaluating the system. They are asked to imagine themselves in the place of a client and use the system for at least one week. These reviews provide feedback from a therapeutic perspective on the suitability of the system for teenage clients and the protocols for clinical usage. These reviews are different from early-stage expert reviews with HCI specialists and therapists.
In our experience, therapist reviews and role plays had numerous benefits, including assessment of the therapeutic validity of the system, understanding how the system could be used in practice, and increasing therapists’ confidence in using the system, which helps
in building up a network of evaluators.
Clinical pilots. While peer evaluations and other approaches have their benefits, some aspects of systems can be evaluated only in clinical settings. For example, privacy is more important to clients than to other teenagers, and the sense of stigma is heightened.
Formal clinical trials in mental health generally take the form of randomized, controlled trials (RCTs). These are extremely resource and time intensive. Formal trials are conducted in more tightly constrained situations than everyday practice, with many exclusions ( symptoms too mild or too severe) and minimal comorbidity. Consequently, many findings from mental health studies never find their way into clinical practice.
From an evaluation perspective, there are issues regarding the use of formal clinical trials (e.g., RCT) as a primary source of design information regarding the usefulness of mental health technology. Although formal trials measuring outcomes are important once a final system exists, we decided to evaluate our systems initially with practicing therapists in their day-to-day clinics. While this raises logistical issues, the benefit is that we have a greater chance of generating useful design inputs, which have a degree of real-world validity. During these clinical pilots, therapists act as proxies for the designer in their interaction with clients.
Logging. Therapist feedback from clinical sessions tends to focus on the impact of the technology from a therapeutic
References:
Archives