INTERACTIONS.ACM.ORG 74 INTERACTIONS JULY–AUGUST2014
FORUM EVALUATION AND USABILITY
Kent are funded under grants from
the Engineering and Physical Sciences
Research Council (EPSRC). We would
like to thank our colleagues at Oxford
for supplying the model used in the
visualizations and for their constructive
comments on visualization prototypes.
We would also like to thank our
colleagues at Bath and Oxford for their
work on the use cases.
Jean Scholtz has been working in
user-centered evaluation for 25 years.
The technology domains she has applied
user-centered evaluation to include video
conferencing systems, robotics, information
retrieval, massive data, and visual analytics
environments. She has a Ph.D. in computer
science from the University of Nebraska.
Oriana Love is a user researcher and
designer within the domains of identity,
social media, multimedia analytics, and
large graphs. She has filed 14 patents in the
areas of collaboration, calendaring, identity
management, UI, and RFID. She has a master’s
degree in computer science from the Georgia
Institute of Technology.
William Pike leads the Visual Analytics
group at Pacific Northwest National Laboratory
( http://vis.pnnl.gov) and PNNL’s Analysis in
Motion initiative ( http://aim.pnnl.gov). His
research interests include mixed initiative
systems, visualization as an aid to knowledge
capture, and active machine learning. He holds
a Ph.D. in geography, with an emphasis on
geographic information science, from Penn
Joseph Bruce is a researcher in visual
analytics and has been contributing to a variety
of analytic tools for 10 years. He specializes in
visualization in Web environments. Other areas
of research include collaborative filtering and
automated user assistants. He has a B.S. in
computer science and in mathematics from
George Fox University.
Dee Kim is a multi-disciplinary designer
with an emphasis in UX and visual design.
She has worked in studio environments,
think tanks, non-profit organizations, and the
government sector. She has an M.F. A. in media
design from Art Center College of Design.
Arthur McBain is involved in visual
analytics-centered development of applications
and interactive displays to aid in user-driven
analysis of large sets of data. He has a B.Sc.
in computer science from the University of
Wisconsin at Eau Claire.
certainty to see the identity attributes
that are determined along the way.
Figure 6 shows an example of an
explanation. In this case the explanation
is why Twitter images can be used to
derive the location. The explanation
given is that many images are tagged
with a geolocation, and this provides the
latitude/longitude that can be used to
generate a city name.
At this point the use cases have been
extremely helpful in determining what
transformations between domains
are useful. Of course, many of these
transformations already exist and can
be added to the model immediately
(fingerprint recognition, face
recognition, etc.). Some transformations
exist in private databases, such as
obtaining owner names and addresses
from license plate information.
Many new transformations are being
explored by SID researchers, such
as swipe gestures on mobile phones.
Some early visualizations of the
transformations showed that there
were few links between other domains
and the psychological domains. SID
research has been looking at different
personality traits and how these
show up in cyber behavior. Certainty
levels for identification were flagged
as a necessity in the use cases. Early
visualizations of the model helped
researchers identify the number of
transformations between domains and
focus their research accordingly. The
visualization of the different paths for
identification has been useful for the
researchers to identify critical nodes
and missing nodes. Additionally, the
combination of a use case and the
visualization of paths for identification
have been extremely valuable to show
the project stakeholders the progress
of the work. The next step in PNNL’s
work is to conduct an evaluation
with the communities who helped
contribute to the use cases. Potential
users will be shown an example use case
and the path visualizations and asked to
To date, the UCD work has proven
even more useful than anticipated in
the SID research. Both the interviews
and subsequent use case development
have been helpful in focusing the
research and the model development.
The visualization work has been
useful in illustrating the underlying
model and therefore has also helped
show deficiencies in the model, as
well as the utility and progress of the
research work. As the stakeholders
have responded positively to the utility
of the model, it is anticipated that the
upcoming user evaluations will be
successful as well.
We urge researchers to consider
whether their projects would benefit
from UCD work, primarily the
development of use cases to guide the
directions of the research and to aid in
The work at PNNL is performed
under funding from the Department
of Homeland Security, Science and
Technology (DHS S&T) Reference
Number HSHQPM-11-X-00014. The
U.K. universities of Southampton,
Leicester, Oxford, Dundee, Bath, and
indicators of identity:
indicators of identity:
Legal representation Southampton
SuperIdentity Annual Reports and links to
Hodges, D., Creese, S., and Goldsmith, M.
A model for identity in the cyber and natural
universes. Proc. of Intelligence and Security
Informatics Conference (EISIC), 2012 European.
IEEE, 2012, 115–122; DOI: 10.1109/EISIC.2012.43
Saxby S. and Knight, A.M. SuperIdentity
framework. Proc. of 8th International Conference
on Legal Security and Privacy Issues in IT Law