kappa statistic. The lowest pairwise kappa value was 0.74,
indicating a satisfactory inter-rater reliability.
Most commentary on sense.us involved data analysis.
A typical comment made note of an observed trend or outlier,
often coupled with questions, explanatory hypotheses, or both.
A typical reply involved discussing hypotheses or answering
questions. The results of coding the comments are shown in
Figure 5. In total, 80.6% of comments involved an observation
of visualized data, 35.5% provided an explanatory hypothesis,
and 38.1% included a question about the data or a hypothesis.
Most questions and hypotheses accompanied an observation ( 91.6% and 92.2%, respectively) and half the hypotheses
were either phrased as or accompanied by a question ( 49.0%).
For example, participants in both lab studies discovered
a large drop in bartenders around the 1930s and posted
comments attributing the drop to alcohol prohibition. In
the live deployment, one user commented on a scatterplot
view, asking why New Hampshire has such a high level of
retail sales per capita (Figure 4). Another user noted that
New Hampshire does not have sales tax, and neither does
Delaware, the second highest in retail sales. In this fashion,
discussion regularly involved the introduction of contextual
information not present in the visualization. For instance,
Figure 1 includes a timeline of events that was iteratively
constructed by multiple users, while the graph of teachers in
Figure 6 notes the introduction of compulsory education.
One instance of social data analysis occurred around a
rise, fall, and slight resurgence in the percentage of dentists
in the labor force. The first comment (one of the five seed
comments) noted the trends and asked what was happening. One subject responded in a separate thread, “Maybe
this has to do with fluoridation? But there’s a bump . . . but kids
got spoiled and had a lot of candy??” To this another subject
responded “As preventative dentistry has become more effective, dentists have continued to look for ways to continue working
(e.g., most people see the dentist twice a year now v. once a year
just a few decades ago).” Perhaps the most telling comment,
however, included a link to a different view, showing both
dentists and dental technicians. As dentists had declined in
percentage, technicians had grown substantially, indicating
specialization within the field. To this, another user asked
“I wonder if school has become too expensive for people to think
about dentistry, or at least their own practice when they can go
to technical school for less?” Visual data analysis, historical
knowledge, and personal anecdote all played a role in the
sensemaking process, explicating various factors shaping
Another role of comments was to aid data interpretation, especially in cases of unclear meaning or anomalies in
data collection. Overall, 15.7% of comments referenced data
naming, categorization, or collection issues. One prominent occupation was labeled “Operative,” a general category consisting largely of skilled labor. This term had little
meaning to subjects, one of whom asked “what the hell is an
operative?” Others responded to reinforce the question or to
suggest an explanation, e.g., “I bet they mean factory worker.”
Another subject agreed, noting that the years of the rise and
fall of operatives seemed consistent with factory workers.
Other examples include views missing data for a single year
figure 5: content analysis categorization of sense.us comments.
categories are not mutually exclusive.
figure 6: Visualization of the number of teachers. annotations
indicate the start of compulsory education and the rise of teachers
in the post-World War ii era.
(1940 was a common culprit), leading users to comment on
the probable case of missing data.
Some users were less interested in specific views than
in recurring patterns. One user was interested in exploring
careers that were historically male-dominated, but have
seen increasing numbers of females in the last half-century.
The user systematically explored the data, saving views in a
bookmark trail later shared in a comment named “Women’s
Rise.” Similarly, a more mathematically minded participant
was interested in patterns of job fluctuations, creating a
trail showcasing recurring distributions. Another searched
for jobs that had been usurped by technology, such as bank
tellers and telephone operators. In each of these cases, the
result was a tour or story winding through multiple views.
Overall, 14.2% of comments referenced an additional
view, either implicitly in the text or explicitly through drag-and-drop bookmark links. Although 22 of the 24 lab study
subjects ( 87.5%) saved at least one view to the bookmark
trail, only 14 ( 58.3%) created one or more drag-and-drop
bookmark links. The amount of view linking varied by user,
ranging from 0 to 19 links with an average of 2. 17.
Comments served other purposes as well. A number were
simple tests of system functionality ( 5.6%), often deleted by