FORUM HEALTH MATTERS

interventions are widely regarded as complex interventions within healthcare. One widely cited framework for the development and evaluation of complex interventions is that of Craig et al. [ 6], comprising four key stages: development, feasibility and piloting, evaluation, and implementation. In this framework, feasibility and piloting and evaluation involve smaller and larger clinical trials, while implementation refers to the rollout of the intervention at scale (e.g., regionally or nationally).

All digital-tool development is located within the development phase, but even there, scant attention is paid to it: It is just assumed to happen.

In recent years, a few papers giving
guidance on how to design digital
health interventions have emerged
(e.g., [ 7]). These typically start
from the assumption that the main
source of expertise for digital health
interventions is clinicians or other
subject-matter experts (such as those
behaviors. These formative
evaluations are typically cost-effective
but do not address the questions that
matter to a health audience—namely,
about long-term engagement and
health outcomes.

A concern, even with health research on digital interventions, is a tendency toward “pilotitis,” where interventions are evaluated as being effective enough—and safe enough and economic enough—but there is no clear strategy for scaling them up and making them sustainable. Arguably, HCI research is even more prone to pilotitis than health research. This is exacerbated by current funding regimes and publication cultures.

Thus, the long-term impact of both HCI and health research can be limited because there is no pipeline of evaluation measures (Figure 1).

COMPLEMENTARY METHODS

With their focus on clinical effectiveness, digital health

ACM Conference
Proceedings
Now Available via
Print-on-Demand!

Did you know that you can
now order many popular
ACM conference proceedings
via print-on-demand?

Institutions, libraries and individuals can choose from more than 100 titles on a continually updated list through Amazon, Barnes & Noble, Baker & Taylor, Ingram and NACSCORP: CHI, KDD, Multimedia, SIGIR, SIGCOMM, SIGCSE, SIGMOD/PODS, and many more.

For available titles and ordering info, visit: librarians.acm.org/pod

ACM Conference

Proceedings
Now Available via
Print-on-Demand!

Figure 2. Person-based approach to digital health intervention development [ 7].

Planning Design Development and evaluation Implementation and trialing

Identify key behavioral issues, needs, challenges

• Synthesize
prior evidence
• Conduct primary
qualitation
research

Create guiding principles to guide designers in intervention development

• Key intervention
design objectives
• Key intervention
features to achieve
objectives

Evaluate and optimize intervention components

• Elicit, observe,
analyze user
reactions to
every intervention
element
• Iteratively modify
to optimize
• Longitudinal mixed
methods case
studies to optimize
independent usage

Intervention evaluated in real-life contexts

• Use mixed methods to improve acceptability, feasibility, effectiveness

 

Figure 3. A design lifecycle for digital health based on the principles and practices of HCI.
Identifying
the problem
Conceptual
design
Detail
design
Implementation
and testing

• What’s the problem to be addressed?

• Who are the intended users, and how might they engage with the intervention?

• Qualitative studies with intended users to deliver...

• Personas based
on evidence
• Scenarios of use
• What are
possible design
solutions/approaches?
• Design patterns
• Concepts to be
implemented
• Task structures
• Information
architecture
• Prototype and
test key features
of the design
• Make sufficiently
informed decisions
about non-critical
implementation
features
• Test iteratively
• Testing “in the wild”
• Refining
the implementation