Follow us on Twitter at http://twitter.com/blogCACM
The Communications Web site, http://cacm.acm.org,
features more than a dozen bloggers in the BLOG@CACM
community. In each issue of Communications, we’ll publish
selected posts or excerpts.
be true, but it is common that we think
what we experience is above average.
The Inverse Lake Wobegon Effect is
a term I am coining for a fallacy that I
see sometimes in computer science (CS)
education: we sample from a clearly biased source and assume the sample describes the overall population. We know
we are observing a superior sample, but
act like we are getting a randomly distributed sample. This is a form of sampling bias ( http://bit.ly/1R358iK).
I introduce the term in a book I just
published with Morgan & Claypool,
Learner-Centered Design of Computing
Education: Research on Computing for
Everyone. One example of the Inverse
Lake Wobegon Effect in CS education
is assuming a successful undergraduate
introductory curriculum will be simi-
larly successful in high school. Students
in undergraduate education are elite. In
the U.S., undergraduates are screened in
an application process and are in the top
half of most scales (such as intellectual
achievement and wealth). Elite students
can learn under conditions in which
average students might not succeed,
which educators call aptitude-treatment
interactions ( http://bit.ly/1PiaGB6).
Consider Bennedsen and Caspers-en’s work on predictors for success in
introductory computing (http://bit.
ly/1TEkY3W). Students in undergraduate education have better math grades
and more course work than average
students in high school, and both factors predict success in introductory CS
courses. Think about the role of algebra
in programming. There are high schools
in Atlanta, GA, where less than half the
students pass algebra. The same CS
curriculum that assumes success with
algebra is unlikely to work well for undergraduate and high school audiences.
Imagine a highly successful undergraduate introductory computing curriculum in which 80% of the students
succeed; that is, 80% of students from
the top half of whatever scale we are talking about. The same curriculum might
fail for 60% of the general population.
We see a similar sampling error when
we talk about using MOOC data to inform our understanding of learning.
The edX website says it offers a platform
for “exploring how students learn.” Students who take MOOCs are overwhelmingly well-educated, employed, and
from developed countries—
characteristics that describe only a small percentage of the overall population. We cannot
assume what we learn from the biased
sample of MOOC participants describes
the general population.
Psychologists are concerned many of
their findings are biased because they
oversample from “WEIRD” students:
“The Inverse Lake
January 11, 2016
Every episode of the radio variety
show “A Prairie Home Companion”
includes a segment in which host
Garrison Keillor tells stories from his
mythical hometown, Lake Wobegon.
Each segment ends with, “Well, that’s
the news from Lake Wobegon, where
all the women are strong, all the men
are good looking, and all the children
are above average.” That notion, that
“all the children are above average,”
is an example of what is known as
the Lake Wobegon Effect (http://bit.
ly/1JYKFKr), also known as “illusory
superiority” ( http://bit.ly/23JkX33).
The Lake Wobegon Effect is where
we consider the small sample in our
experience superior to the population overall. A concrete example: 80%
of drivers consider themselves above-average drivers. Obviously that cannot
Sampling Bias in CS
Education, and Where’s
the Cyber Strategy?
Mark Guzdial examines a logical fallacy in consumer
science education; John Arquilla sees an absence of discussion
about the use of information technologies in future conflicts.