EDITOR Dennis Wixon denniswi@microsoft.com
For this issue it’s my pleasure to introduce John Hopson, who is one of the rising stars in the user research department at Microsoft Game Studios. John’s column is a thoughtful and personal reflection on the purpose and our practice of applied research. I hope you all enjoy it as much as I did. —Dennis Wixon
Microsoft Game Studios | jhopson@microsoft.com
As an HCI researcher working in the games industry, the sweet-est words I have ever heard from my clients have been “Okay, we’ll fix that.” At the end of the day, research is only as good as the amount of impact it has on the experience of our users. The heart of HCI is about understanding and measuring the user experience, about putting metrics around the impact of a design. It would be profoundly hypocritical of us not to apply that same sort of hard-nosed evaluation to our own work products and the impact we have on our users, the designers, and other members of our product teams.
Sadly, most standard methods of communicating research results don’t work very well. Academics have struggled for years to come to terms with the phenomenon of “ uncitedness,” the proportion of published works that are never subsequently cited by other works. Depending on how it is measured and the particular field of research, estimates of uncitedness range from a mere 24 percent for some scientific fields, up to a startling 93 per-
cent for the arts and humanities. Other studies have tried to gauge how many people actually read the average published journal article and have come up with estimates of 10 to 20. That estimate is particularly disheartening when one considers that mere readership doesn’t imply those readers went on to act on the information conveyed in the article.
The academic journal article format is an old if cantankerous friend to anyone who has been trained in the halls of academia, but it is clearly nothing like an optimal format for motivating the reader to action. If anything, these uncitedness numbers underestimate the problem because they represent the best-case scenario, in which the writer and readers are both experts in the same field. Cross-discipline communication, such as that between an HCI researcher and a designer, can be assumed to be even more difficult.
There is a simple underlying truth here: Some research has more impact than other research. And once that dichotomy exists, once we have a split
between the ignored and the effective, I know which side of that divide I want my life’s work to be on. All of the output of an HCI researcher, the publications, presentations, reports, meetings, etc. are merely intermediaries between the research and the impact on the final design. We should evaluate them the same way we evaluate any other piece of design work, examining how they’re used by the actual end user and reshaping them to produce the experience we want.
Working in the games industry, I’ve had the mixed blessing of being the first HCI researcher many of my clients have ever worked with. This means I have the extra burden of convincing these newcomers of the value of HCI work, but it also means that I constantly have the opportunity to reinvent (and hopefully improve) the way I present my data. I’m forced to engage very closely with my clients, walking them through research results step by step, and I’ve had a chance to observe the impact the data has had on their subsequent work. The game design teams I’ve collaborated with put their heart and soul into what
May + June 2008
References:
Archives