the conference room before my research debrief meetings. As people filed in, found their seats, went looking for missing attendees, etc., there would be video playing of a study participant wandering lost through a poorly designed mission or struggling with an overly difficult combat encounter. It provided something for my attendees to watch while they waited and started things off with a concrete example of why we were there. It certainly can be difficult to measure the return on investment of HCI work, but a good example of the cost of not doing the work can be profoundly motivating for your audience.

Second, my research data has the most impact when presented as knowledge about the product, rather than evaluation of the product. As I said earlier, my designers are passionate about their work, and the idea that their favorite feature received a failing grade can add another layer of resistance to their preexisting biases. Telling a designer that a particular game mission is too hard is often counter-produc-tive, but telling them that usability participants took an average of five hours to beat what should have been a one-hour mission tends to work much better. When they perceive their work as being under attack, a skeptical and intelligent audience can generally find a reason to disregard any research finding. The difference between positioning the research findings as “ information to make the product better” rather than “information about what’s wrong with the product” can be subtle, but it can save hours of argument. Furthermore, involving the product team in

the creation of the metrics (“How long should this mission take?”) can also help avoid the perception that their work is being graded on some arbitrary scale. Reframing the debate from “How did the mission score?” to “How did this mission match up with the designer’s intent?” puts the researcher and the designer on the same side—a much better starting place for collaboration.

Finally, the presentation should reflect the goals of the audience, not the presenter. Material should be organized according to whatever schema the audience uses, not according to how they were addressed in the research. It doesn’t matter whether the presentation match-es our logical model of the topic; it matters whether the audience understands the results and goes on to act on them. The presentation is a tool, a machine for making an impression on the minds of the audience, and the depth and accuracy of that impression is the only metric that counts.

One special case of this principle is when there are several audiences for a given piece of research. This can demand multiple separate treatments of the same data, emphasizing different aspects of the work. For example, I’ve begun producing two distinct documents from my usability work: a producer report and a designer report. The producers tend to care more about issues of overall project progress (“Are the usability issues severe enough to delay the project milestone?”), while the designers tend to care about fine details (“How did the players react to that ambush in the first mission?”). I’ve found it more effective to address those audi-

ences separately than to produce a single set of findings that is only partially relevant to any given reader. Both documents accurately represent the same usability study, and both serve the needs of their users.

This is not to say our presentations to our clients should look like marketing fluff, or that we should be telling them what they want to hear. But within the general constraints of sound research and accurate reporting, there is a broad spectrum of ways to accurately convey the results, some of which are more effective than others. As a field, HCI claims to analyze and understand the way users interact with products, and we have no excuse for not analyzing and understanding the impact of our own work. There is no reason we should ever go uncited.

ABOUT THE AUTHOR John Hopson is a user researcher at Microsoft Game Studios and has worked on several bestsell-ing game franchises, including Halo and Age of Empires. John holds a doctorate in experimental psychology and is the author of a number of articles on the interaction of research and game design.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without the fee, provided that copies are not made or distributed for profit or commercial advantage, and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on services or to redistribute to lists, requires prior specific permission and/or a fee. © ACM 1072-5220/08/0500 $5.00

May + June 2008

DOI 10.1145/1353782.1353794