Open innovation systems face, however, serious challenges that, paradoxically, are largely a result of how
successful they have been at eliciting huge
volumes of participation. In this Viewpoint, we review these challenges and
propose some promising directions for
The challenges faced by open innovation systems occur both with idea generation and idea evaluation. Key challenges with idea generation include:
˲ Harvesting costs: Open innovation
engagements tend to generate idea
corpuses that are large, disorganized,
and highly redundant. Pruning such
a list to find the best ideas can be a
massive undertaking. Google’s 10 to
the 100th project, for example, had to
engage 3,000 employees to prune the
150,000 ideas they received, putting
them nine months behind schedule.
IBM flew 100 senior executives into
New York from around the world to
prune the results of their Idea Jam.
˲ Unsystematic coverage: Open innovation systems have no inherent mechanism for ensuring the ideas submitted comprehensively cover the most
critical facets of the problem at hand,
so the coverage is hit-or-miss and may
not align with the customer’s needs.
˲ Shallowness: Open innovation systems tend to generate large numbers of
relatively shallow ideas. A major reason
for this, we believe, is that collaborative
idea development, and accurate credit
assignment, is typically not well supported in current tools.
Open innovation systems also face
challenges with crowd-sourced idea
evaluation: there is often a disconnect
between what the customer wants and
what the crowd selects. This can occur
for several reasons:
˲ Shallow evaluations: Little support
is provided for the crowd building upon
each other’s evaluative expertise, since
users usually do not examine and correct each other’s facts and reasoning.
˲ Rating lock-in: When there are
thousands of ideas, many potentially
valuable ideas may not end up being
evaluated in sufficient depth, and the
system can quickly “lock” into a fairly
static, and arbitrary, ranking, where
the winning ideas are inferior to others in the list.
Open innovation systems thus face
critical challenges in terms of ensuring the potentially massive contributions of the crowd provide high value
to the customer without incurring
prohibitive harvesting costs.
How can we meet these challenges and
more fully achieve the promise of open
innovation systems? Progress will require, we believe, advances on the following two key fronts.
Better open innovation processes.
New open innovation processes are
needed that provide more guidance
about how the crowd can best contribute, help crowd members build on
each other’s inputs, and make it easier
to harvest their contributions,
˲Collaborative idea definition:
Helping the crowd make more deeply
considered contributions will require
progress on incentive schemes and
collaborative authoring structures.
Participants, for example, can be
asked to structure their contributions
as deliberation maps1—as trees made
up of problems to solve, potential solutions for these problems, and the
arguments for and against each potential solution, all single-authored.
Participants can then compose proposals from the best solution ideas
in each map. Credit assignment becomes straightforward because each
proposal is built from components
with clear authorship.
˲ Novel rating mechanisms can help
ensure the crowd evaluates ideas
quickly and accurately with respect to
the criteria the customer cares about.
One possibility, for example, is to use
a kind of prediction market where par-
faced by open
occur both with idea
generation and idea
Contact: Raimund Dachselt
22nd ACM SIGSOFT Symposium
on the Foundations of Software
Hong Kong, China,
Contact: Shing-Chi Cheung
November 30–December 2
13th ACM International
Conference on Virtual Reality
Continuum and Its Applications
Contact: Jin Xiaogang
Conference on Emerging
Contact: Aruna Seneviratne
Contact: Laurent Reveillere
47th Annual IEEE/ACM
Cambridge, U. K.,
Contact: Krisztian Flautner
9th International Conference
on Ubiquitous Information
Contact: Sukhan Lee