“suddenly, when
you need a whole
bunch of people to
do something useful,
platforms like ours
let you organize
them very well.”
Still, not everyone in the crowdsourcing space believes abuses are a
major concern.
At CrowdFlower, founder and CEO
Lukas Biewald observes that crowdsourcing gives people the opportunity
to choose those tasks they want to perform when they want to perform them,
and earn money or coupons or in-game
currency for completing those tasks.
“That seems like a good deal to
me,” he says. “We have over four mil-
lion people working on CrowdFlower
syndicated tasks, and I log into our
partner sites all the time to make sure
that it’s a good experience for every-
one. There are bound to be some com-
plaints and some issues, but we work
as hard as we can to resolve the issues
as quickly as possible.”
CrowdFlower’s business model is
different from that of MobileWorks;
instead of posting jobs on its own site,
it uses an API to syndicate tasks from
customers and then farm them out to
thousands of partner sites, one of them
being MTurk.
Like Turkopticon and MobileWorks, CrowdFlower uses technology—but mainly to guarantee the quality of the work for the employer.
Biewald explains that having a
background in AI enabled him write
the statistical algorithms that predict
the likelihood that a task will be done
correctly—and how many checks and
redundancies need to be built into the
tasks to produce optimal results.
“We have the work history of the
people who sign on,” he says, “so many
of our decisions are based on their his-
torical performance. For instance, if we
think someone is going to be 90% ac-
curate and the customer requires 99%
accuracy, we know we need to have a
second person check the work.”
Also, he says, there are other indica-
tors—if a person works too quickly or
has never done a similar task before,
the software sends up a flag.
Further Reading
von Ahn, L.,
“human Computation,” (video), August
22, 2012, http://www.youtube.com/
watch?v=tx082gDwGcM
Cushing, E.,
“Dawn Of The Digital Sweatshop,” August
1, 2012, “East Bay Express,” http://www.
eastbayexpress.com/oakland/dawn-of-the-
digital-sweatshop/Content?oid=3301022
Kittur, A., Nickerson, J., Bernstein, M.,
Gerber, E., Shaw, A., Zimmerman, J.,
Lease, M., and Horton, J.,
“The Future of Crowd Work,” published
February 2013 at the 16th ACM Conference
on Computer Supported Cooperative Work,
http://hci.stanford.edu/publications/2013/
CrowdWork/futureofcrowdwork-cscw2013.pdf
Neumann, E.,
“Tech Company CrowdFlower Denies
Labor Violation,” December 4, 2012,
“MissionLocal,” http://missionlocal.
org/2012/12/tech-company-crowdflower-
denies-underpaying-workers/
Ipeirotis, P.,
“Mechanical Turk: now with 40.92% spam,”
December 16, 2010, http://www.behind-the-
enemy-lines.com/2010/12/mechanical-turk-
now-with-4092-spam.html
Irani, L., and Silverman, M.,
“Turkopticon: Interrupting Worker
Invisibility in Amazon Mechanical Turk,”
April 2013, ACM SIGChI Conference on
human Factors in Computing Systems,
http://www.ics.uci.edu/~lirani/Irani-Silberman-Turkopticon-camready.pdf
Van Pelt, C., and Sorkin, A.,
“Designing a Scalable Crowdsourcing
Platform,” 2012 ACM SIGMOD International
Conference on Management of Data, http://
dl.acm.org/citation.cfm?id=2213951
Biewald, L.,
“Massive Multiplayer human Computation
for Fun, Money, and Survival,” ICWE
2011, http://link.springer.com/
chapter/10.1007%2F978-3-642-27997-
3_ 18?LI=true#
Hester, V., Shaw, A,. and Biewald, L.,
“Scalable crisis relief: Crowdsourced SMS
translation and categorization with Mission
4636,” ACM Symposium on Computing
for Development 2010, http://dl.acm.org/
citation.cfm?id=1926199
Paul hyman is a science and technology writer based in
great neck, ny.