FORUM HEALTH MATTERS
interventions are widely regarded
as complex interventions within
healthcare. One widely cited
framework for the development and
evaluation of complex interventions is
that of Craig et al. [ 6], comprising four
key stages: development, feasibility
and piloting, evaluation, and
implementation. In this framework,
feasibility and piloting and evaluation
involve smaller and larger clinical
trials, while implementation refers
to the rollout of the intervention at
scale (e.g., regionally or nationally).
All digital-tool development is located
within the development phase, but
even there, scant attention is paid
to it: It is just assumed to happen.
In recent years, a few papers giving
guidance on how to design digital
health interventions have emerged
(e.g., [ 7]). These typically start
from the assumption that the main
source of expertise for digital health
interventions is clinicians or other
subject-matter experts (such as those
behaviors. These formative
evaluations are typically cost-effective
but do not address the questions that
matter to a health audience—namely,
about long-term engagement and
health outcomes.
A concern, even with health
research on digital interventions, is
a tendency toward “pilotitis,” where
interventions are evaluated as being
effective enough—and safe enough
and economic enough—but there is no
clear strategy for scaling them up and
making them sustainable. Arguably,
HCI research is even more prone to
pilotitis than health research. This
is exacerbated by current funding
regimes and publication cultures.
Thus, the long-term impact of both
HCI and health research can be
limited because there is no pipeline of
evaluation measures (Figure 1).
COMPLEMENTARY METHODS
With their focus on clinical
effectiveness, digital health
ACM Conference
Proceedings
Now Available via
Print-on-Demand!
Did you know that you can
now order many popular
ACM conference proceedings
via print-on-demand?
Institutions, libraries and
individuals can choose
from more than 100 titles
on a continually updated
list through Amazon, Barnes
& Noble, Baker & Taylor,
Ingram and NACSCORP:
CHI, KDD, Multimedia,
SIGIR, SIGCOMM, SIGCSE,
SIGMOD/PODS,
and many more.
For available titles and
ordering info, visit:
librarians.acm.org/pod
ACM Conference
Proceedings
Now Available via
Print-on-Demand!
Figure 2. Person-based approach to digital health intervention development [ 7].
Planning Design Development and evaluation Implementation and trialing
Identify key
behavioral issues,
needs, challenges
• Synthesize
prior evidence
• Conduct primary
qualitation
research
Create guiding
principles to guide
designers in
intervention
development
• Key intervention
design objectives
• Key intervention
features to achieve
objectives
Evaluate and optimize
intervention
components
• Elicit, observe,
analyze user
reactions to
every intervention
element
• Iteratively modify
to optimize
• Longitudinal mixed
methods case
studies to optimize
independent usage
Intervention evaluated
in real-life contexts
• Use mixed methods
to improve
acceptability,
feasibility,
effectiveness
Figure 3. A design lifecycle for digital health based on the principles and practices of HCI.
Identifying
the problem
Conceptual
design
Detail
design
Implementation
and testing
• What’s the problem
to be addressed?
• Who are the intended
users, and how might
they engage with
the intervention?
• Qualitative studies
with intended
users to deliver...
• Personas based
on evidence
• Scenarios of use
• What are
possible design
solutions/approaches?
• Design patterns
• Concepts to be
implemented
• Task structures
• Information
architecture
• Prototype and
test key features
of the design
• Make sufficiently
informed decisions
about non-critical
implementation
features
• Test iteratively
• Testing “in the wild”
• Refining
the implementation