effectiveness as practitioners, as well as on our corporate career growth options?

March + April 2013

interactions

Your Most Important Invisible Customers

What C-level executives need to know is actually not complicated in the abstract. First, whatever the issue being considered, they need to know if things are getting better or worse, plus where they rank relative to the competition. Their need to quantify trends is simple to understand and easily satisfied with common business measures such as sales, revenue, and profit. It can also be satisfied by some common market-research measures. For example, even when there is questionable confidence in the absolute numeric values produced by marketing surveys, such as those that use the ubiquitous Net Promoter Score methodology, the CEO can still make a before-and-after judgment based solely on the direction of change in the data from quarter to quarter.

In this context, the typical
usability-related questions a CEO
would ask would resemble the fol-
lowing:
• Is product A’s user experience
improving or declining with this
release?

• How does it compare with other similar products we sell?

• Does it cause usability problems if packaged in a suite of applications?

• How does it compare with the competition?

However, giving executives a lon-
gitudinal view of a trend requires
consistency and repeatability in
the measures collected. Some com-
panies with very mature usability
programs have continuous bench-
marking programs that provide lon-
gitudinally valid information. This
is an emerging trend, but it is far
from standard practice outside of
companies with a very quantitative
culture. In those companies that do
track usability longitudinally, it is
just one of many benchmark indi-
cators. It is rare when usability is
the sole focus of a corporate bench-
marking program. And, in most
companies, UX practitioners are
failing to provide this kind of infor-
mation at all, either longitudinally
or across products lines. This leads
to what I call the CEO credibility gap.
I have observed that over time this
gap has limited both the influence
of UX departments and the career
growth of many UX leaders. It is
something I have had to wrestle
with throughout my own career.

Placing the sole focus on the software engineer as the usability-data consumer leads to each evaluation being treated as a one-off exercise. In my opinion, it is also one of several outcomes resulting from the overuse of quick-and-dirty and often experimentally questionable “discount” usability methods for both lab and field work. Unfortunately, these discount methods have one thing in common: They are not repeatable in a fashion that generates longitudinally valid or useful data.

I would suggest that the root cause leading to CEOs remaining underserved by the typical usability data available to them is a continued lack of business leadership focus and practice understanding among the UX community. This is not a new observation; it first surfaced as a larger theme in the May 2007 issue of interactions on UX business leadership, which I had the privilege to organize and guest edit.

If, as noted, the incremental cost of collecting more complete data is marginal, why don’t we do it more often? It is easy to recognize that this is a chicken-and-egg problem, because if you don’t have data that is relevant to business strategy and investment decisions, you are not likely to be invited into the boardroom in the first place. My experience is that if you have meaningful, business-relevant data to share, you can eventually work your way up the corporate management chain and get access to the CEO to share it, because everyone is struggling to gain an advantage through additional insight, regardless of corporate scale or type of industry. However, you will not get invited back on a regular basis to share usability and evaluation data unless it meets that C-level requirement for longitudinal validity.