entering an address by sending it from the web to the car [ 11].

All of these connected functions can potentially be exploited by UE for research needs. For example, variants of the software can be sent to target user populations for either their subjective feedback or to generate “A/B” test cases. A population’s configuration settings can be analyzed to determine the most and least popular setting changes.

Most important, the software development organization can rapidly update the product during usability testing and then observe users for their reactions to the device and its new software.

Many Groups Use Back-haul Data

User experience is hardly the only group motivated to scoop up back-haul data. Back-haul data collection is essential for other functions to fulfill their objectives .

The ability for UE to understand, leverage, and champion these other needs will maximize its influence over the form that back-haul infrastructure takes. In particular, UE needs to realize its stake in the design of these infrastructure components when initial implementations are sketched. A common mistake for typical small UE teams is not seeing the potential at early stages.

For example, the operations group will require usage data for authentication and billing. IT needs aggregate usage metrics that trigger alarms when usage levels drop, whether due to instability in its own infrastructure or to service-provider outages. Support needs some form of back-haul data to understand customer issues more efficiently, such as versioning, configuration settings, stack traces [ 12], and prior usage. QA values back-haul data on common performance metrics, such as “time until first GPS fix” and “time to first network connection” so that it can assess release readiness. QA also values back-haul data to help track metrics of system performance and stability. With this data, the QA team can better evaluate release readiness. Finally, everyone wants some method to “file a problem report” from the device to reduce the time to file, reproduce, and fix bugs.

may say the product is performing as specified, but they don’t indicate if the users are actually happy with that level of performance.

For UE design, it is beneficial to triage use cases into buckets such as “frequent for all,” “frequent for some,” or “infrequent for all.” In order to do this, the data source must maintain a marker of individuality along with the data, so the data analyst can slice and dice the data to discover such relationships.

In addition, the UE group desires the ability to run longitudinal queries, particularly the ability to see that new user features lead to perceptible user benefits. This requires the UE group itself to create and maintain over time a database of high-level user events in a normalized format.

[ 11] MyDash, Send2Car, product description http://www.dash.net/ product/mydash-send- 2car.php

[ 12] Wikipedia, “Windows Error Reporting ( WER),” http://en.wikipedia.org/ wiki/Windows_Error_ Reporting

September + October 2009

Issues Unique to UE Needs

The UE group has needs distinct from the other functional groups. Both QA and operations, by and large, are more concerned with aggregate numbers than an individual’s experience. Their numbers

Infrastructure Needs

In order to exploit the back-haul data, central sampling issues such as who, what, and when (how frequently) need addressing.

Behavioral logging benefits from the ability to understand and filter the results through the lens of various user-grouping mechanisms. For example, stakeholders will question unwelcome results because various outlier groups may have skewed the results.

Invariably, both business and operational groups have an interest in creating user segmentation; this functionality is thus almost certainly available to the UE group. However, the grouping mechanisms will be driven by customer purchasing segments, and this in turn can constrain an experiment’s design.

Problem reports. Since users have problems, they need a mechanism to explain what those problems are: the problem report. There are two different design approaches:

Mostly passive, where the problem report is • created on behalf of the users. They just need to submit the report.

Mostly active, where the user initiates the prob- • lem report.

Both are, of course, useful. But for devices, the active report is particularly useful, as it is often very difficult to create bug reports independent of a complex environment that only the user fully understands.

The work required is not just the data transmission and recording. In implementing the problem report feature, I recommend budgeting a fair amount of time for implementing tools that facili-