DOI: 10.1145/2833093 COP YRIGHT HELD BY AUTHOR
best. And we must not set up or pay too
much attention to win/lose studies that
merely provide existence proofs.
Second, top HCI outlets focus on
novelty and originality. I wrote a paper
with some colleagues on replications
and discussed how journals require
originality (e.g., that submissions must
be “original in some way,” HCI Journal )
[ 5]. The insistence on originality
contrasts with incremental research;
the latter is sometimes used to reject
papers at program committee meetings.
This focus works against building on
earlier work and against replications (we
found three percent in a sample of 891
HCI papers). It means that HCI rarely
challenges earlier results.
As a way for ward, I think HCI
researchers, and HCI outlets as well,
need to emphasize incremental research
more and need to build on what we
already know to a much larger extent.
Being novel is overrated compared with
being wrong in a daring study about a
fundamental question in HCI.
With this column I want to call for us
to be more ambitious, both as individual
researchers and as a community. I
believe we can set up research in HCI
to be wrong more frequently and in
more informative ways: We can change
our outlets to value negative results,
and we can try to do empirical studies
that stand a chance of failing in an
interesting way. It is all about attitude:
Let’s try to be more willing to be wrong
in HCI research.
1. Kostakos, V. The big hole in HCI research.
Interactions 22, 2 (2015), 48–51.
2. Greenberg, S. and Buxton, B. 2008.
Usability evaluation considered harmful
(some of the time). Proc. of CHI 2008,
3. Platt, J. R. Strong inference. Science 146,
3642 (1964), 347–353.
4. Hornbaek, K. Some whys and hows
of experiments in human-computer
interaction. Foundations and Trends in
Human–Computer Interaction 5, 4 (2013),
5. Hornbaek, K., Sander, S. S., Bargas-Avila,
J., and Simonsen, J.G. Is once enough? On
the extent and content of replications in
human-computer interaction. Proc. of CHI
2014 , 3523–3532.
Kasper Hornbaek is a professor in
computer science at the University of
Copenhagen. He works on user experience,
shape-changing interfaces, large displays, and
body-based interaction. He is also interested
in the methodology of HCI, including the role of
replications, measures of usability, and solid
NOVEMBER–DECEMBER 2015 IN TERAC TIONS 21 INTERAC TIONS. ACM.ORG