Interactions - November/December 2011

BLoGPoST
At CHI and related
conferences there

standard experimental psychology study might need to have 30 subjects to show the effect being studied to a reasonable level of doubt. (There’s some math about why that is the case, but that’s not important right now.) The number of subjects in a study is referred to as n, so in our example n= 30. n has come to be an important mark of quality in psychological experiments. The results of a study in experimental psychology with n= 12 will be seen as much less reliable or valid— and thus less publishable —than a study with n= 30. A suitable n is recognized as one of the marks of quality in experimental psychology.

Let us compare that notion of
validity in an experimental psy-
chology experiment to an ethno-
graphic investigation of a single
extended family. In a paper for
CSCW, my co-author Elisa Oreglia
and I studied how members of
three families in northern China
gave each other cell phones as
gifts [ 4]. I would argue—and our
reviews tend to agree, I’m happy to
say—that our paper is a “good eth-
nography”: By the epistemological
standards of ethnographic work in
CSCW, it is seen to be a valid cre-
ation of knowledge. Importantly, it
is not a ”bad experiment” because
n= 12 rather than n= 30. That mark
of quality is specific to a different
epistemic culture.
And it’s precisely that distinc-
tion—between a good ethnogra-
phy and a bad experiment—that
makes it important to be explicit
about epistemic cultures in HCI.
In other, more established and less
inclusive fields, it is possible to
assume that everyone shares the
same epistemic culture as you: the
reviewers, the other readers, the
editor of the journal. For better or
for worse—and I stand solidly on
the side of better—HCI is a won-
derfully mixed bag of epistemic
cultures. We must be rigorous
about keeping research to episte-
mologically appropriate marks of
quality—watching six different
people for a few hours each is not
an ethnography—but we need to
recognize there are many different
ways to do good work in HCI. There
is an unfortunate tendency to see
one’s own epistemic culture as
the default and right way to make
knowledge, and my personal expe-
rience suggests this is particularly
apparent among those who have
been trained solely or primarily in
positivist science traditions. We
must recognize that such assump-
tions are not acceptable within the
CHI community. As we see over
and over again at CHI, at CSCW, at
DIS, and at our other conferences
and publication venues, HCI has
a huge variety of ways to create
knowledge—design, ethnographic
investigation, deployment of work-
ing systems in the real world, and
many more—and each are them-
selves valid ways of making new
knowledge, with standards of qual-
ity and of validity.

is often the sense
that whichever
intellectual discipline
one identifies with,
one feels like an
intellectual minority,
unappreciated and
drowned out
by louder, more
numerous voices.

cultures from committee descriptions, and particularly hard if one is part of the more than 50 percent new authors at the conference every year. Neither of these solutions is perfect, but both are steps toward helping the HCI community grow and incorporate new ways of knowing and new ways of making knowledge.

endnotes:

1. There is an open question about the degree to which CHI incorporates other forms of diversity, but that’s an issue for another article. 2. Landay, J. A. I give up on CHI/UIST. Dub for the Future blog (Nov. 7, 2009); http://dubfuture. blogspot.com/2009/11/i-give-up-on-chiuist.html 3. Knorr Cetina, K. Epistemic Cultures. Harvard University Press, Cambridge, MA, 2012. 4. Oreglia, E. and Kaye, J.J. A gift from the city: mobile phones in rural China. Proc. CSCW’ 12 (forthcoming 2012).