participant sitting at an individual
workstation was shown an interface
with a map with home ranges and
fixes of a kite for one of the weeks,
as well as two blogs, one written
manually and one computer-generated, without any information about
their provenance. Participants said
what blog they preferred (or expressed no preference) and also rated
each blog on how informative, fluent,
and engaging they found it on a sev-en-point Likert scale. Each participant evaluated three pairs of blogs.
We designed the study to test three
specific hypotheses:
H1. Computer-generated blogs are
preferred to human-written blogs;
H2. Computer-generated blogs are
more informative, fluent, and engag-
ing than human-written blogs; and
H3. The differences in ratings for
computer-generated and human-written blogs are conditional on the movement pattern of an individual bird C1,
C2, or C3, as in Figure 2.
Comparison with baseline. To directly evaluate whether communicating ecological insights through the
blogs is important to readers, we compared Blogging Birds to a computer-generated baseline that blogs about
the movement patterns without reference to ecological concepts; see Table
2 for an example. These baseline blogs
were entirely factual and reported behaviors only directly observed in the
data but that otherwise followed the
same format as the full-system blogs.
An additional 27 undergraduate students enrolled in the digital society
course, but who had not participated
in the earlier experiment, evaluated
the full vs. the baseline system using
the same methodology and interface
as before. We designed this study to
test two specific hypotheses:
H4. Computer-generated blogs with
ecological insights are preferred to
baseline computer-generated blogs
without ecological insights; and
H5. Computer-generated blogs
with ecological insights are more informative and engaging than baseline
computer-generated blogs without
ecological insights, while their fluency
is comparable.
the respective maps and described
on the information sheet. These 36
manually written blogs were compared to computer-generated blogs
for the same weeks in the evaluation.
As our goal was to investigate
Blogging Birds not just as a tool for
those with an interest in nature conservation but as a resource to engage
those interested in new technologies. We ran evaluations with two
distinct groups of participants: 93
undergraduate biology students enrolled in a second year “community
ecology” course and 49 first- and sec-ond-year undergraduates from across
disciplines enrolled in a course entitled “digital society,” both at the University of Aberdeen. In each trial, a
Figure 5. Preferences for human-written and for computer-generated blogs by movement
condition, as in Figure 2: C1 is movement within a home range; C2 is a round trip; and C3 is
movement between home ranges.
1.0
0.8
0.6
0.4
Pref
er
en
ce
(
pr
o
po
r
ti
o
n)
Community Ecology Students
C1 C2
Either Human-written Computer-generated
C3 C1 C2 C3
Digital Society Students
0.2
0.0
Figure 6. Average ratings for human-written and for computer-generated blogs by movement condition, as in Figure 2: C1 is movement
within a home range; C2 is a round trip; and C3 is movement between home ranges.
C1 C2 C3
Community
Ecology
0
1
2
3
4
5
6
7 (a) Informativeness
C1 C2 C3
Digital
Society
Community
Ecology
Digital
Society
Community
Ecology
Digital
Society
C1 C2 C3
0
1
2
3
4
5
6
7 (b) Engagingness
C1 C2 C3 C1 C2 C3
0
1
2
3
4
5
6
7 (c) Fluency
C1 C2 C3
Human-written Computer-generated