we have designed so far have focused on problems that are easily divided into subtasks. The “bite-size” nature of these games adds to their popularity and appeal to casual gamers in particular, since such players typically go for games they can play “just one more time” without having to make too much of a time commitment.

The GWAP approach represents a promising opportunity for everyone to contribute to the progress of AI. By leveraging the human time spent playing games online, GWAP game developers are able to capture large sets of training data that express uniquely human perceptual capabilities. This data can contribute to the goal of developing computer programs and automated systems with advanced perceptual or intelligence skills.

acknowledgment

We would like to thank Manuel and Lenore Blum, Mike Crawford, Shiry Ginosar, Severin Hacker, Susan Hrish-enko, Mihir Kedia, Edith Law, Bryant Lee, and Roy Liu for their help in this research.

 

References

1. Carroll, J.M. and Thomas, J. M. Fun. ACM SIGCHI Bulletin 19, 3 (Jan. 1988), 21–24.

2. Chao, D. Computer games as interfaces. Interactions 11, 5 (Sept.–Oct. 2004), 71–72.

3. Chao, D. Doom as an interface for process management. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Seattle, Mar. 31–Apr. 5). ACM Press, New York, 2001, 152–157.

4. Fails, J.A. and Olsen, D.R. Interactive machine learning. In Proceedings of the Eighth International Conference on Intelligent User Interfaces (Miami, Jan. 12–15). ACM Press, New York, 2003, 39–45.

5. Federoff, M. Heuristics and Usability Guidelines for the Creation and Evaluation of Fun in Video Games. Unpublished thesis, Indiana University, Bloomington; www.melissafederoff.com/thesis.html.

6. Giles, J. Internet encyclopaedias go head to head. nature 438 (Dec. 15, 2005), 900–901.

7. Hassenzahl, M., Beu, A., and Burmeister, M. Engineering joy. IEEE Software 18, 1 (Jan.–Feb. 2001), 70–76.

8. Laurel, B.K. Interface as mimesis. In User–Centered System Design: new Perspectives on Human– Computer Interaction, D.A. Norman and S. W. Draper, Eds. Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 1986, 67–85.

9. Law, E.L. M., von Ahn, L., Dannenberg, R. B., and Crawford, M. TagATune: A game for music and sound annotation. In Proceedings of the Eighth International Conference on Music Information Retrieval (Vienna, Austria, Sept. 23–30). Austrian Computer Society, Vienna, Austria, 2007, 361–364.

VISUALIZA TION B Y BEN FR Y

10. Locke, E. A. and Latham, G. P. A Theory of Goal Setting and Task Performance. Prentice Hall, Englewood Cliffs, NJ, 1990.

11. Malone, T. M. Heuristics for designing enjoyable user interfaces: Lessons from computer games. In Proceedings of the Conference on Human Factors in Computing Systems (Gaithersburg, MD, Mar. 15–17). ACM Press, New York, 1982, 63–68.

12. Malone, T. M. What makes things fun to learn? Heuristics for designing instructional computer games. In Proceedings of the Third ACM SIGSMALL

Symposium and the First SIGPC Symposium on Small Systems (Palo Alto, CA, Sept. 18–19). ACM Press, New York, 1980, 162–169.

13. Mokka, S., Väätänen, A., Heinilä, J., and Välkkynen, P. Fitness computer game with a bodily user interface. In Proceedings of the Second International Conference on Entertainment Computing (Pittsburgh, PA, May 8–10). Carnegie Mellon University, Pittsburgh, PA, 2003, 1–3.

14. Pagulayan, R., Keeker, K., Wixon, D., Romero, R., and Fuller, T. User–centered design in games. In The Human–Computer Interaction Handbook: Fundamentals, Evolving Techniques and Emerging Applications, J. A. Jacko and A. Sears, Eds. Lawrence Erlbaum Associates, Mahwah, NJ, 2003, 883–905.

15. Richards, C. Teach the world to twitch: An interview with Marc Prensky, CEO and founder Games2train. com. Futurelab (Dec. 2003); www.futurelab.org.uk/ resources/publications_reports_articles/web_articles/ Web_Article578.

16. Shneiderman, B. Designing for fun: How can we design user interfaces to be more fun? Interactions 11, 5 (Sept.–Oct. 2004), 48–50.

17. Steiner, I. Group Process and Productivity. Academic Press, New York, 1972.

18. Stork, D. G. and Lam C. P. Open mind animals: Ensuring the quality of data openly contributed over the World Wide Web. In Learning from Imbalanced Data Sets: Papers from the AAAI Workshop ( Technical Report WS-00-05). (Austin, TX, July 30–Aug. 1). American Association for Artificial Intelligence, Menlo Park, CA, 2000, 4–9.

19. Stork, D.G. The Open Mind Initiative. IEEE Intelligent Systems & Their Applications 14, 3 (May–June 1999), 19–20.

20. Sweetser, P. and Wyeth, P. GameFlow: A model for evaluating player enjoyment in games. ACM Computers in Entertainment 3, 3 (July 2005), 3.

21. Tsang, M., Fitzmaurice, G., Kurtenbach, G., and Khan, A. Game–like navigation and responsiveness in non–game applications. Commun. ACM 46, 7 (July 2003), 57–61.

22. von Ahn, L. and Dabbish, L. Labeling images with a computer game. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vienna, Austria, Apr. 24–2). ACM Press, New York, 2004, 319–326.

23. von Ahn, L., Ginosar, S., Kedia, M., and Blum, M. Improving image search with Phetch. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (Honolulu, Apr. 15–20). IEEE Press, New York, 2007, IV–1209–IV–1212.

24. von Ahn, L., Kedia, M., and Blum, M. Verbosity: A game for collecting common-sense knowledge. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Montreal, Apr. 22–27). ACM Press, 2007, 75–78.

25. von Ahn, L., Liu, R., and Blum, M. Peekaboom: A Game for locating objects in images. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Montreal, Apr. 22–27). ACM Press, New York, 2006, 55–64.

26. Webster, J. Making computer tasks at work more playful: Implications for systems analysts and designers. In Proceedings of the SIGCPR Conference on Management of Information Systems Personnel (College Park, MD, Apr. 7–8). ACM Press, New York, 1988, 78–87.

This work was partially supported by National Science Foundation grants CCR-0122581 and CCR-0085982 (ALADDIN) and generous gifts from Google, Inc., and the Heinz Endowment. Luis von Ahn was partially supported by a Microsoft Research Graduate Fellowship, a Microsoft Research New Faculty Fellowship, and a MacArthur Fellowship.

Luis von Ahn ( [e-mail]) is an assistant professor in the Computer Science Department at Carnegie Mellon University, Pittsburgh, PA.

Laura Dabbish ( [e-mail]) is an assistant professor of information technology and organizations in the Heinz School, with a joint appointment in the Human-Computer Interaction Institute in the School of Computer Science at Carnegie Mellon University, Pittsburgh, PA.

© 2008 ACM 0001-0782/08/0800 $5.00