Communications of the ACM

Society | DOI: 10.1145/2492007.2492015
Paul Hyman
software aims to Ensure fairness
in crowdsourcing Projects
The debate rages on about whether crowdsourcing
is a win-win for workers, as well as for employers.

Are Workers Whoparticipate in the highly distributed microlabor online system known as crowdsourcing treated fairly? And what about the crowdsourcing employers?

It is not a new topic of debate, but a new paper presented at the ACM SIGCHI Conference on Human Factors in Computing Systems in Paris in April is likely to heat up the discussion considerably, especially since one of the paper’s two authors urges computer professionals to take a harder look at crowdsourcing—a market which reaches an estimated tens of millions of people annually—and think not just about the technology that makes it possible, but also about the human workers and how they are impacted.

That author, Lilly Irani, a researcher and Ph.D. candidate in the Department of Informatics at the University of California Irvine, also developed Turkopticon, a software program released in February 2009 that was designed “as an ethically motivated response to crowdsourcing workers’ invisibility,” says Irani.

Photogra Ph from Shutter Stock.com

She recalls being troubled by what she perceived as the plight of workers being paid just a few dollars an hour to perform tasks on Amazon Mechanical Turk (MTurk), an online crowdsourcing marketplace launched by Amazon. com in November 2005 as a meeting place for “requesters” (employers) with large volumes of microtasks or HITs (Human Intelligence Tasks). Indeed, one worker complaint heard over and over again was that requestors are able to walk away with the work submitted without paying for it, because Amazon leaves payment completely up to the discretion of the employers, who can claim they are unhappy with the quality of the work.

“There is no process for the employ-

er to justify its decisions to workers or to Amazon,” she says.

And so Irani created Turkopticon, a software tool cheekily named after a prison surveillance design with a guard tower in which there may or may not be a guard. The possibility of surveillance

“our hope is that
turkopticon will not
only hold employers
accountable,
but also induce
better behavior.”

induces prisoners to discipline themselves, says Irani.

Functionally, Turkopticon is a browser extension that, when workers search MTurk for HITs, “scrapes the requester ID that is within the HITs market page list and inserts any reviews that other workers have written about that particular requester. So that when a worker is deciding whether they want to take the assignment or not, they can also review a quick summary of what other workers have said. Our hope is that Turkopticon will not only hold employers accountable, but also induce better behavior,” she explains.