kappa statistic. The lowest pairwise kappa value was 0.74, indicating a satisfactory inter-rater reliability.

Most commentary on sense.us involved data analysis. A typical comment made note of an observed trend or outlier, often coupled with questions, explanatory hypotheses, or both. A typical reply involved discussing hypotheses or answering questions. The results of coding the comments are shown in Figure 5. In total, 80.6% of comments involved an observation of visualized data, 35.5% provided an explanatory hypothesis, and 38.1% included a question about the data or a hypothesis. Most questions and hypotheses accompanied an observation ( 91.6% and 92.2%, respectively) and half the hypotheses were either phrased as or accompanied by a question ( 49.0%).

For example, participants in both lab studies discovered a large drop in bartenders around the 1930s and posted comments attributing the drop to alcohol prohibition. In the live deployment, one user commented on a scatterplot view, asking why New Hampshire has such a high level of retail sales per capita (Figure 4). Another user noted that New Hampshire does not have sales tax, and neither does Delaware, the second highest in retail sales. In this fashion, discussion regularly involved the introduction of contextual information not present in the visualization. For instance, Figure 1 includes a timeline of events that was iteratively constructed by multiple users, while the graph of teachers in Figure 6 notes the introduction of compulsory education.

One instance of social data analysis occurred around a rise, fall, and slight resurgence in the percentage of dentists in the labor force. The first comment (one of the five seed comments) noted the trends and asked what was happening. One subject responded in a separate thread, “Maybe this has to do with fluoridation? But there’s a bump . . . but kids got spoiled and had a lot of candy??” To this another subject responded “As preventative dentistry has become more effective, dentists have continued to look for ways to continue working (e.g., most people see the dentist twice a year now v. once a year just a few decades ago).” Perhaps the most telling comment, however, included a link to a different view, showing both dentists and dental technicians. As dentists had declined in percentage, technicians had grown substantially, indicating specialization within the field. To this, another user asked “I wonder if school has become too expensive for people to think about dentistry, or at least their own practice when they can go to technical school for less?” Visual data analysis, historical knowledge, and personal anecdote all played a role in the sensemaking process, explicating various factors shaping the data.

Another role of comments was to aid data interpretation, especially in cases of unclear meaning or anomalies in data collection. Overall, 15.7% of comments referenced data naming, categorization, or collection issues. One prominent occupation was labeled “Operative,” a general category consisting largely of skilled labor. This term had little meaning to subjects, one of whom asked “what the hell is an operative?” Others responded to reinforce the question or to suggest an explanation, e.g., “I bet they mean factory worker.” Another subject agreed, noting that the years of the rise and fall of operatives seemed consistent with factory workers. Other examples include views missing data for a single year

figure 5: content analysis categorization of sense.us comments. categories are not mutually exclusive.

Observation
Question
Hypothesis
Data Integrity
Linking
Socializing
System Design
Testing
Tips
To-do
Affirmation

80.6%

38.1% 35.5%

15.7%
14.2%
9.0%
9.0%
5.6%
4.1%

2.6%

1.5%

0%

20%

40%

60%

80%

100%

figure 6: Visualization of the number of teachers. annotations indicate the start of compulsory education and the rise of teachers in the post-World War ii era.

(1940 was a common culprit), leading users to comment on the probable case of missing data.

Some users were less interested in specific views than in recurring patterns. One user was interested in exploring careers that were historically male-dominated, but have seen increasing numbers of females in the last half-century. The user systematically explored the data, saving views in a bookmark trail later shared in a comment named “Women’s Rise.” Similarly, a more mathematically minded participant was interested in patterns of job fluctuations, creating a trail showcasing recurring distributions. Another searched for jobs that had been usurped by technology, such as bank tellers and telephone operators. In each of these cases, the result was a tour or story winding through multiple views.

Overall, 14.2% of comments referenced an additional view, either implicitly in the text or explicitly through drag-and-drop bookmark links. Although 22 of the 24 lab study subjects ( 87.5%) saved at least one view to the bookmark trail, only 14 ( 58.3%) created one or more drag-and-drop bookmark links. The amount of view linking varied by user, ranging from 0 to 19 links with an average of 2. 17.

Comments served other purposes as well. A number were simple tests of system functionality ( 5.6%), often deleted by