2. ACM Code of Ethics and Professional Conduct. 1992;
3. Citron, D. and Pasquale, F. The scored society: due
process for automated predictions. Washington Law
Review 89 (2014).
4. Clerwall, C. Enter the robot journalist. Journalism
Practice 8, 5 (2014): 519–531.
5. Diakopoulos, N. Algorithmic accountability:
Journalistic investigation of computational power
structures. Digital Journalism 3, 3 (2015): 398–415.
6. Diakopoulos, N. Algorithmic defamation: the case of
the shameless autocomplete. Tow Center for Digital
7. Diakopoulos, N., et al. Data-driven rankings: the design
and development of the IEEE Top Programming
Languages news app. Proceedings of the Symposium
on Computation + Journalism, 2014.
8. Diakopoulos, N. How Uber surge pricing really works.
Washington Post Wonkblog (Apr. 17, 2015).
9. Don Ray Drive-A-Way Co. v. Skinner, 785 F. Supp. 198
(D.D. C. 1992); http://law.justia.com/cases/federal/
10. Epstein, R. and Robertson, R.E. The search engine
manipulation effect (SEME) and its possible impact
on the outcomes of elections. In Proceedings of the
National Academy of Sciences 112, 33 (2015).
11. Eslami, M. et al. ‘I always assumed that I wasn’t
really that close to [her]:’ Reasoning about invisible
algorithms in the news feed. In Proceedings of the 33rd
Annual ACM SIGCHI Conference on Human Factors in
Computing Systems, 2015.
12. Feldman, M., et al. Certifying and removing disparate
impact. In Proceedings of the 21st ACM International
Conference on Knowledge Discovery and Data Mining,
13. Herlocker, J.L. et al. Explaining collaborative filtering
recommendations. In Proceedings of the ACM
Conference on Computer Supported Cooperative Work,
14. Kalhan, A. Immigration policing and federalism
through the lens of technology, surveillance, and
privacy. Ohio State Law Journal 74 (2013).
15. Kashin, K. et al. Systematic bias and nontransparency
in US Social Security Administration forecasts.
Journal of Economic Perspectives 29, 2 (2015).
16. Kraemer, F. et al. Is there an ethics of algorithms? Ethics
and Information Technology 13, 3 (2010), 251–260.
17. Letham, B. et al. Building interpretable classifiers
with rules using Bayesian analysis. Annals of Applied
18. Mitchell, A. et al. Millennials and Political News.
Pew Research Center, Journalism and Media (June
1, 2015); http://www.journalism.org/2015/06/01/
19. Muckrock. Source code of HEAT SAFETY TOOL, 2011;
20. Mühlbacher, T. et al. Opening the black box: strategies
for increased user involvement in existing algorithm
implementations. IEEE Transactions on Visualization
and Computer Graphics 20, 12 (2014) 1643–1652.
21. Nissenbaum, H. Accountability in a computerized society.
Science and Engineering Ethics 2, 1 (1996): 25–42.
22. Schaffer, J. et al. Getting the message?: a study of
explanation interfaces for microblog data analysis. In
Proceedings of the 20th International Conference on
Intelligent User Interfaces, 2015, 345–356.
23. Sen, S. et al. Turkers, Scholars, ‘Arafat’ and ‘Peace:’
Cultural communities and algorithmic gold standards.
In Proceedings of the 18th ACM Conference on
Computer Supported Cooperative Work and Social
Computing, 2015. 826–838.
24. Sifry, M. Facebook wants you to vote on Tuesday.
Here’s how it messed with your feed in 2012. Mother
Jones (Oct. 31, 2014); http://www.motherjones.
25. Tintarev, N. and Masthoff, J. A survey of explanations
in recommender systems. Proceedings of the
International Conference on Data Engineering, 2007,
Nicholas Diakopoulos is an assistant professor at the
University of Maryland, College Park, Philip Merrill College
of Journalism, with courtesy appointments in the College
of Information Studies and Department of Computer
Science. He is also a fellow at the Tow Center for Digital
Journalism at Columbia University
Copyright held by author.
Publication rights licensed to ACM. $15.00.
sis and see how a change in a factor would impact the resulting output ranking. Based on 1,285 tweets
that people shared about the app,
we found about one in six indicated
people were reweighting the ranking
in various ways. While it is too early
to claim victory in designing dynamic
and transparent ranking interfaces,
this is at least a step in the direction I
envision for interactive modeling.
There are technical challenges
here, too. In particular, concerns often arise over manipulation and gaming that may be enabled by disclosing
information about how systems work.
A certain amount of threat modeling
may be necessary if transparency is
required. If a particular piece of information were made available about an
algorithm, how might that be gamed,
manipulated, or circumvented? Who
would stand to gain or lose? Manipu-lation-resistant algorithms also need
to be designed and implemented. Feature sets that are robust and difficult to
game need to be developed.
The software engineering of algorithms also needs to consider architectures that support transparency and
feedback about algorithmic state so they
can be effectively steered by people. 20
Algorithm implementations should
support callbacks or other logging
mechanisms that can be used to report
information to a client module. This is
essential systems work that would form
the basis for outputting audit trails.
Finally, we must work on machine-
learning and data-mining solutions that
directly take into account provisions for
fairness and anti-discrimination. For
example, recent research has explored
algorithmic approaches that can iden-
tify and correct for disparate impact
in classifiers by statistically transform-
ing the input data set so that prediction
of protected attributes is not possible. 12
Additional research is needed in this
space as different types of models and
data types may demand different techni-
cal approaches and adaptations.
Society must grapple with the ways in
which algorithms are being used in government and industry so that adequate
mechanisms for accountability are built
into these systems. The ideas presented
here about acting ethically and responsibly when empowering algorithms to
make decisions are important to absorb into your practice. There is much
research still to be done to understand
the appropriate dimensions and modalities for algorithmic transparency,
how to enable interactive modeling,
how journalism should evolve, and how
to make machine learning and software
engineering sensitive to, and effective
in, addressing these issues.
Online Algorithms in
Jacob Loveless, Sasha Stoikov, and Rolf Waeber
AI Gets a Brain
Jeff Barr and Luis Felipe Cabrera
Other People’s Data
1. ACM. Software Engineering Code of Ethics and
Professional Practice, 2015; https://www.acm.org/
IEEE top programming languages ranking and reweighting interfaces.