• The “Urban Spoon” application for the iPhone provides real-time restaurant recommendations based on proximity to your current location.

 

rant database irrelevant. In both cases, users would cancel the service because it just “didn’t work.”

The current suite of usability methods is inadequate for this new, enterprise-reliant context. Testing an enterprise application is vastly different from testing a standalone product. How do we accurately test the usability of an overall enterprise, like Urban Spoon, while it’s still in development? How do we ensure it passes both the ease-of-use and utility benchmarks that users require? We could easily test narrow portions of the application using traditional techniques: Lab testing could evaluate the process of entering constraints or purchasing the application. We could even test the latency or resolution alternatives in a simulation (e.g., is 100-foot resolution adequate for dense urban settings? Is the resolution requirement equal in both urban and suburban settings?). Unfortunately, the sum of these

narrow tests would not be able to get at the true value of the service or its real-world usability.

We see sophisticated clients who still focus their design on lab testing and scripted usability, when it’s not really appropriate for their networked environment. Certainly, there is a narrow role for “sanity checking” the usability of each interface within the enterprise, and there are naive clients who see “usability testing” as shorthand for all user-centered design. But scripted usability testing forces the product into unnaturally small pieces, tested in series. Worse, it means you are setting yourself up to miss the larger design issues (e.g., does the latency make the core idea impossible? Does actual context of use make this feature meaningless, or does the size of the hardware make this product unattractive?). Lab testing, once regarded as the gold standard, isn’t meaningful for most current products. Products are no longer

“rooted” in singular settings; they are collaborative, interwoven, and interdependent. To be accurate and relevant, the testing must be mobile, modular, and contextual (none of which can be accomplished in a lab).

With service-based, cooperative enterprise applications, we are limited to hacked-together usability methods (i.e., a series of narrow simulations) or rough design estimation (i.e., a combination of contextual studies and design research) to try to understand the benefits and pitfalls with the system. And these enterprise challenges are becoming ubiquitous: location-based services like Urban Spoon, social-networking applications like Facebook and Twitter, or third-party platform sellers like eBay and Amazon. We need to replace our narrow usability methods with rich design methods that can address these types of enter-prise-design challenges.

May + June 2009