Figure 1: Workers collaborated to trans- late a poem using Etherpad, and the results are shown. Each color indicates a different worker’s contribution.

 

Many collaborative tasks in the real world, however, involve people interacting with each other. Examples range from scientists collaborating on a discovery to students collaborating on a report to volunteers writing articles together on Wikipedia.

In real-world collaborations, interaction is the norm rather than the exception. There are many advantages to interacting groups, such as the ability to communicate and coordinate on the fly rather than having to follow prespecified plans or rules, motivational gains from identifying with a group, and the bonds formed from interacting with other group members and helping behavior between them. There may be interesting benefits from breaking the assumption of independence and enabling workers to collaborate interactively in crowdsourcing.

However, such benefits are by no means certain. For example, would workers participating in a financial market really help each other without any financial incentives?

We examined this question in the
context of a problem that is both dif-
ficult overall and especially difficult to
do in a crowdsourcing context: collab-
orative translation. Unlike the short,
simple, objective, and verifiable tasks
that are typical of Mechanical Turk,
translation can be complex, challeng-
ing, time-consuming, highly subjec-
tive, and impossible to verify automati-
cally. It is also highly interdependent,
requiring a consistent voice and ap-
proach throughout. However, if we
were able to harness the power of the
crowd for translation, there could be
many potential benefits ranging from
supporting disaster relief efforts (as
already demonstrated by CrowdFlow-
er and Samasource in crowdsourcing
translation in the Haitian relief efforts,
see page 10) to providing essential
training data to help machine transla-
tion research.

Figure 2: Comparing the original poem (left), a published translation by Havard, 1990 (center), and final crowdsourced version from our experiment (right), the crowdsourced version was preferred by 14 out of 16 bilingual raters.