of the game. For example, an hourly
high-score list gives players a specific
point total to aim for to get onto the
list, as well as relatively quick feedback
(within the hour) about their progress
toward it. A daily high-score list and
all-time high-score list define goals of
increasing difficulty. These multi-level
goals, varying in difficulty, provide
strong, positive motivation for extended game play—and related data generation.
Randomness. GWAPs should also incorporate randomness. For example,
inputs for a particular game session
are typically selected at random from
the set of all possible inputs, and players are randomly paired to prevent
cheating.
Because inputs are randomly selected, their difficulty varies, thus keeping
the game interesting and engaging
for expert and novice players alike. 11, 12
It also means that every game session
involves uncertainty about whether all
inputs will be completed within the
time limit, adding to the challenge experienced by players. 11, 12
Random partner assignment also
ensures the uniqueness of each game
session. Anecdotal evidence from the
ESP Game22 suggests that during each
game session players develop a sense
of their partners’ relative skill, a perception that affects their joint performance. The feeling of connection that
players can get from these games is one
of the factors that motivates repeated
play. 18, 20
output accuracy
Additional mechanisms must be added
to GWAPs beyond the basic template
structure to ensure output correctness
and counter player collusion. For example, players of the ESP Game might try
to circumvent the game’s built-in verification mechanism by agreeing prior to
the game that for every image they will
always type the letter “a”; in this case,
they would always match each other,
and incorrect data would therefore be
entered into the system. We describe
generally applicable mechanisms in
the following sections that have proved
successful in guarding against player
collusion and guaranteeing the correctness of the computation across all
game templates.
Random matching. GWAPs are
the real measure
of utility for a
GWaP is therefore
a combination of
throughput and
enjoyability.
meant to be played by hundreds, if not
thousands, of people at once, most in
distributed locations. Players paired
or grouped randomly have no way of
knowing their partner’s identity so have
no easy way to agree ahead of time on
any cheating strategy. Thus, under random matching, the probability of two or
more cheaters using the same strategy
being paired together should be low.
Player testing. Games may randomly
present players inputs for which all
possible correct outputs are already
known. For them, if the output produced by a particular player does not
match the known correct outputs, the
players should be considered suspicious, and none of their results should
be trusted. Depending on the number
of “test” inputs presented to players,
this strategy can guarantee with high
probability that the output is correct.
To illustrate, assume half of the inputs
given to a player are test inputs. The
probability is thus that a new output
by the player is correct, given of course
that the player is correct on all the test
inputs at least 50% of the time, a probability that can be increased through
repetition.
Repetition. A game should be designed so it does not consider an output correct until a certain number of
players have entered it. This strategy for
determining correctness enables any
GWAP to guarantee correct output with
arbitrarily high probability. As an example, consider an output-agreement
game; if for a given input the game accepts an output as correct only after
n pairs have entered it, and the game
itself knows that each of these n pairs
entered a correct output with at least
50% probability (as a result of player
testing), then the output is correct with
probability of at least (I–½n).
Taboo outputs. For problems in
which many different outputs can be
associated with one input (such as labeling images with words), ensuring
sufficient coverage of the output space
is an important consideration. The use
of “taboo,” or off-limits, outputs provides some guarantee that a larger proportion of all possible outputs will be
entered by all players. Taboo outputs
are known correct outputs displayed
onscreen during game sessions that
players are not allowed to enter. They
can be taken from correct outputs gen-