ing an edit policy may potentially affect
million of pages. As another example,
when building a structured KB, flagging an incorrect data piece typically
has less potential impact than supplying an inference rule, which may be
used in many parts of the CS system.
Quantifying the potential impact of
a contribution type in a complex CS
system may be difficult.
12, 13 But it is important to do so, because we typically
have far fewer high-ranking users such
as editors and admins (than regulars,
say). To maximize the total contribution of these few users, we should ask
them to make potentially high-impact
contributions whenever possible.
Third, what about machine contributions If a CS system employs an algorithm for a task, then we want human
users to make contributions that are
easy for humans, but difficult for machines. For example, examining textual
and image descriptions to decide if
two products match is relatively easy
for humans but very difficult for machines. In short, the CS work should
be distributed between human users
and machines according to what each
of them is best at, in a complementary
and synergistic fashion.
Finally, the user interface should
make it easy for users to contribute.
This is highly non-trivial. For example,
how can users easily enter domain
knowledge such as “no current living
person was born before 1850” (which
can be used in a KB to detect, say, incorrect birth dates)? A natural language format (such as in openmind.
org) is easy for users, but difficult for
machines to understand and use, and a
formal language format has the reverse
problem. As another example, when
building a structured KB, contributing
attribute-value pairs is relatively easy
(as Wikipedia infoboxes and Freebase
demonstrate). But contributing more
complex structured data pieces can be
quite difficult for naive users, as this
often requires them to learn the KB
schema, among others.
How to combine user contributions?
Many CS systems do not combine con-
tributions, or do so in a loose fashion.
For example, current evaluation sys-
tems do not combine reviews, and com-
bine numeric ratings using relatively
simple formulas. Networking systems
simply link contributions (homepages
and friendships) to form a social net-
work graph. More complex CS systems,
however, such as those that build soft-
ware, KBs, systems, and games, com-
bine contributions more tightly. Exactly
how this happens is application depen-
dent. Wikipedia, for example, lets users
manually merge edits, while ESP does
so automatically, by waiting until two
users agree on a common word.