changed between the raw, unprocessed
file and visualized marks, and how the
patterns they see reflect the underlying
data. What statistics are used? What
was filtered for? What happened to
outliers? By being transparent with
visualizations, we can help people
better understand the available data and
intuitively generate informed insights
and decisions, even with large data
TOWARD BETTER PRACTICES
This article focuses on common
mistakes in visualizations that bias data
analysis. These guidelines are deeply
grounded in empirical studies and
decades of observation and practice.
Vision science and visualization
offer some explanation for why these
phenomena occur and allow us to design
alternative representations that more
faithfully depict data.
However, we are far from
understanding all of the mechanisms
at play when people interpret data. For
example, how might visualizations
account for illusions that occur
naturally in data? Can we rescale or
renormalize visualizations to account
for biases introduced by the ways we
see the world? How do we intuitively
navigate high-dimensional data? How
do we effectively pair visualization
and computation to help people better
leverage petabyte datasets?
A principled and quantified
understanding of the way we see data
can empower people to better leverage
the many benefits offered by data.
Crafting optimal visualizations is
still an unsolved and wicked problem.
Deeper collaboration between data
science, cognitive science, and vision
science is necessary to move us toward
algorithmic and visual solutions that
can scaffold an informed and inclusive
1. Larson, A. M., Freeman, T.E., Ringer, R.V.,
and Loschky, L.C. The spatiotemporal
dynamics of scene gist recognition. Journal of
Experimental Psychology: Human Perception
and Performance 40, 2 (2014), 471.
2. Borkin, M. A., Gajos, K.Z., Peters, A.,
Mitsouras, D., Melchionna, S., Rybicki,
F.J., Feldman, C.L., and Pfister, H.
Evaluation of artery visualizations for
heart disease diagnosis. IEEE Trans. on
Visualization and Computer Graphics 17, 12
3. Wong, B. Points of view: Color blindness.
Nature Methods 8, 441 (2011).
4. Ball, K., and Sekuler, R. A specific and
enduring improvement in visual motion
discrimination. Science 218, 4573 (1982),
5. Franconeri, S.L., Jonathan, S.V., and
Scimeca, J.M. Tracking multiple objects
is limited only by object spacing, not by
speed, time, or capacity. Psychological
Science 21, 7 (2010), 920–925.
7. Neisser U. The control of information
pickup in selective looking. In Perception
and Its Development: A Tribute to Eleanor J.
Gibson. A.D. Pick, ed. Erlbaum, New York,
8. Simons, D.J. and Levin, D. T. Failure to
detect changes to people during a real-
world interaction. Psychonomic Bulletin &
Review 5, 4 (1998), 644–649.
9. Pandey, A.V., Rall, K., Satterthwaite,
M.L., Nov, O., and Bertini, E. How
deceptive are deceptive visualizations?: An
empirical analysis of common distortion
techniques. Proc. of the ACM Conference
on Human Factors in Computing Systems.
ACM, Ne w York, 2015, 1469–1478.
12. Sarikaya, A., Gleicher, M., and Szafir, D.A.
Design factors for summary visualization
in visual analytics. Computer Graphics
Forum 37, 3 (2018).
13. Ariely, D. Seeing sets: Representation by
statistical properties. Psychological Science
12, 2 (2001), 157–162.
14. Szafir, D. A., Haroz, S., Gleicher, M., and
Franconeri, S. Four types of ensemble
coding in data visualizations. Journal of
Vision 16, 5 (2016), 1–19.
15. Correll, M. and Gleicher, M. Error bars
considered harmful: Exploring alternate
encodings for mean and error. IEEE Trans.
on Visualization and Computer Graphics 20,
12 (2014), 2142–2151.
16. Newman, G.E. and Scholl, B.J. Bar graphs
depicting averages are perceptually
misinterpreted: The within-the-bar bias.
Psychonomic Bulletin & Review 19, 4
Danielle Albers Szafir is an assistant
professor in the Department of Information
Science at the University of Colorado Boulder.
Her research bridges data science and vision
science to develop interactive visualization
systems, guidelines, and techniques for
exploratory data analysis.
DOI: 10.1145/3231772 COPYRIGH T HELD BY AUTHOR. PUBLICATION RIGHTS LICENSED TO ACM. $15.00
Approach B Approach A
Approach B Approach A
Approach B Approach A
Approach C Approach B Approach A
Figure 7. Traditional aggregation methods, such as bar charts encoding means, replace data with statistics, obscuring important patterns in the
underlying data distribution.