The Researcher-Tool Mismatch:
Improving the Fit Between User
Researchers and Technology
Rob Tannen

Bresslergroup | rtannen@bresslergroup.com

[ 1] Wilson, C. and L. Conte. “21st Century Technology for Usability and User Interface Design Activities,” Usability Interface 7, no. 1 (July 2000) <http://www. stcsig.org/usability/ newsletter/0007-tools. html>

[ 2] http://www.techsmith. com/company/ about.asp

September + October 2008

[ 3] Tannen, Rob. “Survey
Results: Technology
for User Research.”
Designing for Humans, 10
December 2006. <http://
www.designingforhu-
mans.com/idsa/2006/12/
survey_results_.html>

interactions

Pity the typical user researcher— hand cramped from writing pages of subsequently illegible observation notes, arm sore from trying to hold a camera steady for hours on end, head aching from trying to assimilate so much information. And if all goes well, that was the easy part. Returning to the office, the researcher now faces piles of notes, recordings, transcriptions, and other unstructured information to make sense of, but little time to do so. The data collection itself is the lesser challenge compared with the organization, analysis, and communication of the findings (i.e., the “secondary” research project).

Early on in my career I recognized an irony in design research: Professionals who are striving to understand user needs for new products were often doing so with relatively low-tech, generic tools—pen and paper, basic audio-video equipment, and Microsoft Office. The user research profession has accomplished a lot with a limited toolbox, and guidance on how technology could benefit user research practitioners is rare (e.g., Wilson and Conte [ 1]). Fortunately, this has been changing, due to the introduction of technologies that directly

support the user research process. For example, Techsmith’s Morae software, launched in 2004 [ 2], specifically streamlined the time and equipment needed for conducting and recording software usability testing.

I began studying how technologies were being utilized in user research and what tools could improve data collection and analysis. A survey of user research professionals I conducted in 2006 [ 3], found more than half of the respondents used videocameras to document their observations, with more than a third using computer-based video. Presumably, the use of various technologies in user research has grown since then. But while technology use was widespread, there was also strong evidence of dissatisfaction. More than half of the survey respondents indicated that user research technology lags behind technology that is being researched, and more than one-third reported that they had created custom technology tools or solutions for user research. In other words, technology is important for conducting research, but off-the-shelf solutions frequently did not meet the needs of user researchers.

Characteristics of User
Research Technology

In defining the needs of user researchers, I identified the following characteristics of current and potential user research technology:

• Documentation—capability to record occurrences for subsequent review. Documentation tools can range from low-tech (pencil and paper) to high-tech (digital video) but share the common function of improving accuracy and reducing dependence on memory.

• Measurement—taking documentation a step further, measurement references an attribute against a known, typically quantitative, comparator. Measurement tools too can be simple (a tape measure, rating scale, or stopwatch) or complex (body-measurement scanning system).

• Efficiency—referring to tools that allow user research tasks to be completed more quickly or with less effort than otherwise. For example, an online survey allows access to a greater number of respondents versus an in-person survey.

• Enhancement—tools that allow researchers to observe phenomena that are invisible or otherwise difficult to access. For