Survey Participation
We approached two groups with similar characteristics: 20 experts were approached based on personal relationships and 11 responded to a message
that we sent to a mailing list.a N = 31
is a reasonable number for such a preliminary analysis. The demographic
data presented in Table 1 indicates
that the majority of the experts work at
research universities, have many years
of teaching experience of both introductory and advanced CS and SE courses to both CS major and SE students.
This background clearly indicates they
understand issues related to the challenge of teaching the topic of abstraction to good CS and SE students as well
as assessing their abstraction skills.
a From the questionnaire sent to the first group
( http://bit.ly/2f9GWgr), two questions were excluded in the subsequent questionnaire sent
to the second group ( http://bit.ly/2eQJH7N):
a) Question 1 was a specific case of question 2;
and b) Question 9 that the first group assessed
lowest concerning its ability to assess abstraction skills. In addition, to the second questionnaire, we added the open question “Can you
suggest an example which best fits Pattern X?”
Data Analysis: When Do CS
and SE Instructors Agree and
Disagree about the Appropriate
Means to Assess Abstraction?
For each pattern, we calculated the
average evaluation score and the standard deviation (see Table 2). Though
an agreement was not reached with
respect to any pattern, we can see that:
˲ The highest agreement (Average = 7. 53;
SD = 2; seven different scores) with respect to a pattern’s suitability for measuring abstraction skills was reached
for Pattern 5: Given a system representation, students are asked to give one representation that is more abstract than
the given one and one representation
that is less abstract than the given one.
˲ The lowest agreement (Average = 6,
SD = 3.05; 10 different scores) with respect to a pattern’s suitability for measuring abstraction skills was reached
for Pattern 2: Given several representations of a specific system, students are
asked to rank them according to their
level of abstraction and to consider the
purpose of each abstraction.
Disagreement: What was surprising
in the data analysis was the vast range
C
O
M
M
U
N
I
C
AT
I
O
NS
A
P
P
S
Available for iPad,
iPhone, and Android
Available for iOS,
Android, and Windows
http://cacm.acm.org/
about-communications/
mobile-apps
Access the
latest issue,
past issues,
BLOG@CACM,
News, and
more.
Table 1. Survey participants’ demographic data.
Where do the experts teach?
Research university
23 74%
Teaching college
5 16%
High school 0 0%
Other*
3 10%
31
* One indicated that he or she is a post doc who taught
in the past and one indicated that she or he works in
a research institute.
Teaching Experience
0– 5 years
2 6%
6–10 years
2 6%
11–15 years
3 10%
16–20 years
3 10%
More than 20 years
21 68%
Students’ major/minor CS/SE taught by the experts (more than one option can be checked)
Computer science major students
25
Computer science minor students
10
Prospective computer science high school teachers
1
Software engineering students
18
Electrical engineering students
5
Other engineering students
9
Other
7
Courses the experts teach (more than one option can be checked)
Introductory computer science courses
18
Advanced computer science courses
20
Software engineering courses
21
Engineering courses
5
Other
6
