Usability
Testing EHRs
Correction
In Art Swanson’s and Scott Lind’s
article, “Usability Testing EHRs,”
from our November + December
2011 issue, the authors refer to
a proposed NIST protocol as the
“Usability Evaluation Protocol
(UEP).” The correct name of the pro-
tocol is the EHR Usability Protocol
(EUP). A draft document describing
the rationale for the protocol can
be found at http://www.nist.gov/
healthcare/usability/upload/Draft_
EUP_09_ 28_ 11.pdf
The article provoked many com-
ments from readers, some of which
are presented here in edited form.
innovations in interactive computing in flight decks. It may be the
most important usability issue for
the EHR industry is that a similar
culture of quality and safety is
still evolving for information systems that support clinical healthcare and safety-critical decisions.
Keith Butler
University of Washington
National Center for Cognitive
Informatics and Decision Making
March + April 2012
interactions
Comments
In the November + December
2011 issue of interactions, Swanson
and Lind describe industry objections to a NIST plan for summative usability testing of electronic
health record systems (EHRs).
They begin by citing reliability issues of formative tests, but
then imply those problems apply
equally to summative tests. This is
a little like ignoring your house’s
water meter because it won’t tell
you which faucets are leaky.
The authors assert that EHRs
and their usage are too complex
to allow comparison, describing
seven challenges to justify their
objection. In terms of measure-
ment theory and statistics, their
challenges fall into the well-
established factors of validity
and reliability, and the generality
of samples. These same factors
helped drive the ISO standard for
summative usability testing that is
the basis of NIST’s plan, which is
based on an adaptation of the ISO
standard for reporting summative
usability tests called the Common
Industry Format (CIF). The objec-
tives of the CIF are to allow usabil-
ity to be included in customers’
purchasing decisions, to replicate
a test if needed, and to judge how
well software products match the
customer’s own users, tasks, work-
flows, and context.
