sense facts. However, the Open Mind
approach involves two drawbacks: reliance on the willingness of unpaid
volunteers to donate their time and no
guarantee that the information they
enter is correct. GWAPs differ from
Open Mind in that they are designed
to be enjoyable while ensuring that the
data they collect is free from error.
Interactive machine learning.
Another area leveraging human abilities
to train computers is “interactive machine learning” 4 in which a user provides examples to a machine-learning
system and is given real-time feedback
as to how well an algorithm is learning. Based on the feedback, the user is
able to determine what new examples
should be given to the program. Some
instances of this approach have utilized human perceptual skills to train
computer-vision algorithms to recognize specific objects.
Making work fun. Over the past 30
years, human-computer-interaction
researchers have recognized and written about the importance of enjoyment
and fun in user interfaces. 16, 26 For example, systems (such as the StyleCam)
aim to use gamelike interaction to increase enjoyment and engagement
with the software. 21 Many researchers have suggested that incorporating
gamelike elements into user interfaces
could increase user motivation and the
playfulness of work activities. 16, 26 Some
projects have taken this notion further,
turning the user interface itself into
a game. For instance, PSDoom provides a first-person-shooter-style interface for system-administrator-related
tasks. 2, 3 The idea of turning work tasks
into games is increasingly being applied in children’s learning activities. 12
Researchers note, as we do here, that it
is important to not simply slap a gamelike interface onto work activities but
to integrate the required activities into
the game itself; there must be tight interplay between the game interaction
and the work to be accomplished.
Desire to Be entertained
The GWAP approach is characterized
by three motivating factors: an increasing proportion of the world’s population has access to the Internet; certain
tasks are impossible for computers but
easy for humans; and people spend lots
of time playing games on computers.
In contrast to other work that has attempted to use distributed collections
of individuals to perform tasks, the
paradigm we describe here does not
rely on altruism or financial incentives
to entice people to perform certain actions; rather, they rely on the human
desire to be entertained. A G WAP, then,
is a game in which the players perform
a useful computation as a side effect
of enjoyable game play. Every GWAP
should be associated with a computational problem and therefore generate
an input-output behavior.
A game can be fully specified through
a goal players try to achieve (the winning condition) and a set of rules that
determines what players can and cannot do during the game. A GWAP’s
rules should encourage players to correctly perform the necessary steps to
solve the computational problem and,
if possible, involve a probabilistic guarantee that the game’s output is correct,
even if the players do not want it to be
correct.
The key property of games is that
people want to play them. We therefore
sidestep any philosophical discussions
about “fun” and “enjoyable,” defining
a game as “successful” if enough human-hours are spent playing it.
We advocate a transformative process whereby a problem is turned into
a GWAP. Given a problem that is easy
for humans but difficult or impossible
for computers, the process of turning
the problem into a GWAP consists of
first creating a game so that its structure (such as rules and winning condition) encourages computation and correctness of the output. Having created
many GWAPs, including the ESP Game,
Peekaboom, Phetch, and Verbosity, we
explore three game-structure templates
that generalize successful instances of
human computation games: output-agreement games, inversion-problem
games, and input-agreement games.
Output-agreement games.
Output-agreement games (see Figure 1) are a
generalization of the ESP Game (see
the sidebar “The ESP Game and Verbosity” on page 65) to its fundamental
input-output behavior:
Initial setup. Two strangers are randomly chosen by the game itself from
among all potential players;
Rules. In each round, both are given
the same input and must produce out-
figure 1: in this output-agreement game,
players are given the same input and
must agree on an appropriate output.
Player 1
INPUT
Player 2
INPUT
(t ) output
1, 1 1, 1
(t ) output
1, 2 1, 2
(t ) output
1,n 1,n
(t ) output
2, 1 2, 1
(t ) output
2, 2 2, 2
(t ) output
2,m 2,m
Players win if/when output = output
1,i 2,j
figure 2: in this output-agreement game,
the partners are agreeing on a label.
Player 1
(0:03) dog
(0:07) puppy
(0: 10) cute
Player 2
(0:06) animal
(0: 11) dog
puts based on the input. Game instructions indicate that players should try to
produce the same output as their partners. Players cannot see one another’s
outputs or communicate with one another; and
Winning condition. Both players
must produce the same output; they
do not have to produce it at the same
time but must produce it at some point
while the input is displayed onscreen.
When the input is an image and the
outputs are keyword descriptions of
the image, this template becomes the
ESP Game (see Figure 2).
Since the two players cannot communicate and know nothing about
each other, the easiest way for both to
produce the same output is by entering something related to the common
input. Note, however, that the game
rules do not directly tell the players
to enter a correct output for the given