we have designed so far have focused
on problems that are easily divided
into subtasks. The “bite-size” nature
of these games adds to their popularity and appeal to casual gamers in particular, since such players typically go
for games they can play “just one more
time” without having to make too much
of a time commitment.
The GWAP approach represents a
promising opportunity for everyone to
contribute to the progress of AI. By leveraging the human time spent playing
games online, GWAP game developers
are able to capture large sets of training data that express uniquely human
perceptual capabilities. This data can
contribute to the goal of developing
computer programs and automated
systems with advanced perceptual or
We would like to thank Manuel and
Lenore Blum, Mike Crawford, Shiry
Ginosar, Severin Hacker, Susan Hrish-enko, Mihir Kedia, Edith Law, Bryant
Lee, and Roy Liu for their help in this
1. Carroll, J.M. and Thomas, J. M. Fun. ACM SIGCHI
Bulletin 19, 3 (Jan. 1988), 21–24.
2. Chao, D. Computer games as interfaces. Interactions
11, 5 (Sept.–Oct. 2004), 71–72.
3. Chao, D. Doom as an interface for process
management. In Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems
(Seattle, Mar. 31–Apr. 5). ACM Press, New York, 2001,
4. Fails, J.A. and Olsen, D.R. Interactive machine
learning. In Proceedings of the Eighth International
Conference on Intelligent User Interfaces (Miami,
Jan. 12–15). ACM Press, New York, 2003, 39–45.
5. Federoff, M. Heuristics and Usability Guidelines for
the Creation and Evaluation of Fun in Video Games.
Unpublished thesis, Indiana University, Bloomington;
6. Giles, J. Internet encyclopaedias go head to head.
nature 438 (Dec. 15, 2005), 900–901.
7. Hassenzahl, M., Beu, A., and Burmeister, M.
Engineering joy. IEEE Software 18, 1 (Jan.–Feb. 2001),
8. Laurel, B.K. Interface as mimesis. In User–Centered
System Design: new Perspectives on Human–
Computer Interaction, D.A. Norman and S. W. Draper,
Eds. Lawrence Erlbaum Associates, Inc., Mahwah, NJ,
9. Law, E.L. M., von Ahn, L., Dannenberg, R. B., and
Crawford, M. TagATune: A game for music and sound
annotation. In Proceedings of the Eighth International
Conference on Music Information Retrieval (Vienna,
Austria, Sept. 23–30). Austrian Computer Society,
Vienna, Austria, 2007, 361–364.
VISUALIZA TION B Y BEN FR Y
10. Locke, E. A. and Latham, G. P. A Theory of Goal Setting
and Task Performance. Prentice Hall, Englewood
Cliffs, NJ, 1990.
11. Malone, T. M. Heuristics for designing enjoyable
user interfaces: Lessons from computer games. In
Proceedings of the Conference on Human Factors in
Computing Systems (Gaithersburg, MD, Mar. 15–17).
ACM Press, New York, 1982, 63–68.
12. Malone, T. M. What makes things fun to learn?
Heuristics for designing instructional computer
games. In Proceedings of the Third ACM SIGSMALL
Symposium and the First SIGPC Symposium on Small
Systems (Palo Alto, CA, Sept. 18–19). ACM Press,
New York, 1980, 162–169.
13. Mokka, S., Väätänen, A., Heinilä, J., and Välkkynen,
P. Fitness computer game with a bodily user
interface. In Proceedings of the Second International
Conference on Entertainment Computing (Pittsburgh,
PA, May 8–10). Carnegie Mellon University, Pittsburgh,
PA, 2003, 1–3.
14. Pagulayan, R., Keeker, K., Wixon, D., Romero, R.,
and Fuller, T. User–centered design in games. In
The Human–Computer Interaction Handbook:
Fundamentals, Evolving Techniques and Emerging
Applications, J. A. Jacko and A. Sears, Eds. Lawrence
Erlbaum Associates, Mahwah, NJ, 2003, 883–905.
15. Richards, C. Teach the world to twitch: An interview
with Marc Prensky, CEO and founder Games2train.
com. Futurelab (Dec. 2003); www.futurelab.org.uk/
16. Shneiderman, B. Designing for fun: How can we design
user interfaces to be more fun? Interactions 11, 5
(Sept.–Oct. 2004), 48–50.
17. Steiner, I. Group Process and Productivity. Academic
Press, New York, 1972.
18. Stork, D. G. and Lam C. P. Open mind animals: Ensuring
the quality of data openly contributed over the World
Wide Web. In Learning from Imbalanced Data Sets:
Papers from the AAAI Workshop ( Technical Report
WS-00-05). (Austin, TX, July 30–Aug. 1). American
Association for Artificial Intelligence, Menlo Park, CA,
19. Stork, D.G. The Open Mind Initiative. IEEE Intelligent
Systems & Their Applications 14, 3 (May–June 1999),
20. Sweetser, P. and Wyeth, P. GameFlow: A model
for evaluating player enjoyment in games. ACM
Computers in Entertainment 3, 3 (July 2005), 3.
21. Tsang, M., Fitzmaurice, G., Kurtenbach, G., and Khan,
A. Game–like navigation and responsiveness in
non–game applications. Commun. ACM 46, 7 (July
22. von Ahn, L. and Dabbish, L. Labeling images with
a computer game. In Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems
(Vienna, Austria, Apr. 24–2). ACM Press, New York,
23. von Ahn, L., Ginosar, S., Kedia, M., and Blum, M.
Improving image search with Phetch. In Proceedings
of the IEEE International Conference on Acoustics,
Speech, and Signal Processing (Honolulu, Apr. 15–20).
IEEE Press, New York, 2007, IV–1209–IV–1212.
24. von Ahn, L., Kedia, M., and Blum, M. Verbosity: A
game for collecting common-sense knowledge. In
Proceedings of the SIGCHI Conference on Human
Factors in Computing Systems (Montreal, Apr. 22–27).
ACM Press, 2007, 75–78.
25. von Ahn, L., Liu, R., and Blum, M. Peekaboom: A Game
for locating objects in images. In Proceedings of the
SIGCHI Conference on Human Factors in Computing
Systems (Montreal, Apr. 22–27). ACM Press, New York,
26. Webster, J. Making computer tasks at work more
playful: Implications for systems analysts and
designers. In Proceedings of the SIGCPR Conference
on Management of Information Systems Personnel
(College Park, MD, Apr. 7–8). ACM Press, New York,
This work was partially supported by National
Science Foundation grants CCR-0122581 and
CCR-0085982 (ALADDIN) and generous gifts
from Google, Inc., and the Heinz Endowment.
Luis von Ahn was partially supported by
a Microsoft Research Graduate Fellowship,
a Microsoft Research New Faculty Fellowship,
and a MacArthur Fellowship.
Luis von Ahn ( firstname.lastname@example.org) is an assistant
professor in the Computer Science Department at
Carnegie Mellon University, Pittsburgh, PA.
Laura Dabbish ( email@example.com) is an
assistant professor of information technology and
organizations in the Heinz School, with a joint appointment
in the Human-Computer Interaction Institute in
the School of Computer Science at Carnegie Mellon
University, Pittsburgh, PA.
© 2008 ACM 0001-0782/08/0800 $5.00