We approached two groups with similar characteristics: 20 experts were approached based on personal relationships and 11 responded to a message
that we sent to a mailing list.a N = 31
is a reasonable number for such a preliminary analysis. The demographic
data presented in Table 1 indicates
that the majority of the experts work at
research universities, have many years
of teaching experience of both introductory and advanced CS and SE courses to both CS major and SE students.
This background clearly indicates they
understand issues related to the challenge of teaching the topic of abstraction to good CS and SE students as well
as assessing their abstraction skills.
a From the questionnaire sent to the first group
( http://bit.ly/2f9GWgr), two questions were excluded in the subsequent questionnaire sent
to the second group ( http://bit.ly/2eQJH7N):
a) Question 1 was a specific case of question 2;
and b) Question 9 that the first group assessed
lowest concerning its ability to assess abstraction skills. In addition, to the second questionnaire, we added the open question “Can you
suggest an example which best fits Pattern X?”
Data Analysis: When Do CS
and SE Instructors Agree and
Disagree about the Appropriate
Means to Assess Abstraction?
For each pattern, we calculated the
average evaluation score and the standard deviation (see Table 2). Though
an agreement was not reached with
respect to any pattern, we can see that:
˲ The highest agreement (Average = 7. 53;
SD = 2; seven different scores) with respect to a pattern’s suitability for measuring abstraction skills was reached
for Pattern 5: Given a system representation, students are asked to give one representation that is more abstract than
the given one and one representation
that is less abstract than the given one.
˲ The lowest agreement (Average = 6,
SD = 3.05; 10 different scores) with respect to a pattern’s suitability for measuring abstraction skills was reached
for Pattern 2: Given several representations of a specific system, students are
asked to rank them according to their
level of abstraction and to consider the
purpose of each abstraction.
Disagreement: What was surprising
in the data analysis was the vast range
Available for iPad,
iPhone, and Android
Available for iOS,
Android, and Windows
Table 1. Survey participants’ demographic data.
Where do the experts teach?
High school 0 0%
* One indicated that he or she is a post doc who taught
in the past and one indicated that she or he works in
a research institute.
0– 5 years
More than 20 years
Students’ major/minor CS/SE taught by the experts (more than one option can be checked)
Computer science major students
Computer science minor students
Prospective computer science high school teachers
Software engineering students
Electrical engineering students
Other engineering students
Courses the experts teach (more than one option can be checked)
Introductory computer science courses
Advanced computer science courses
Software engineering courses