Follow us on Twitter at http://twitter.com/blogCACM
The Communications Web site, http://cacm.acm.org,
features more than a dozen bloggers in the BLOG@CACM
community. In each issue of Communications, we’ll publish
selected posts or excerpts.
18 communIcaTIonS of The acm | jANuARY2014 | vol. 57 | No. 1
“Results from the
moocs: not There Yet”
october 18, 2013
MOOCs in the Coursera, Udacity, and
edX form are tightly tied to CS. The
leaders of the xMOOC movement came
out of computer science, and most of
the first generation of xMOOCs focused
on teaching computer science. Many of
the MOOC evaluations so far have been
expert reviews. Our Learning Sciences
and Technologies seminar at Georgia
Tech’s College of Computing just read
Moti Ben-Ari’s travelogue on his experiences in Coursera’s and Udacity’s
introductory CS MOOC. The empirical
results of the first rounds of MOOCs on
intro courses are now in, so it is worth
considering how they are doing.
Karen Head has finished her series
of posts in The Chronicle of Higher Education on the freshman-composition
MOOC she taught with Gates Foundation funding. The stats were disappointing—only 238 of the approximately 15K students who did the first
homework finished the course. That is
presentation with visual elements. There
were other students who joined the course
after the second week; we cautioned them
that they would not be able to pass it because there was no mechanism for doing
peer review after an assignment’s due
date had passed.
Georgia Tech also received funding
from the Gates Foundation to develop
a MOOC approach for a first-year college physics course. I met with Mike
Schatz, the lead teacher on that effort.
It is a remarkable course, including a
“laboratory” where students take videos of moving objects, then construct
computational simulations in Python
to match the real-world observations.
The completion results were pretty
similar to Karen’s: 20K students signed
up, 3K students completed the first assignment, and only 170 finished.
In terms of empirical studies, Mike
had an advantage that Karen did not —
there are standardized tests for measuring the physics knowledge he was
testing, and he used those tests before
and after the course. Mike said the
completers fell into three categories:
those who came in with a lot of physics knowledge and who ended with
relatively little gain, those who came
in with very little knowledge and made
almost no progress, and a group of students who really did learn a lot. They
do not yet know the relative percentages of the three categories. However,
it is clear that being a completer does
not mean that anything was learned.
I also met with Jason Freeman who
finished his Survey of Music Technol-
ogy MOOC for Coursera. His results
even less than the ~10% we saw com-
pleting other MOOCs.
Karen Head writes:
No, the course was not a success. Of
course, the data are problematic: Many
people have observed that MOOCs often
have terrible retention rates, but is retention an accurate measure of success? We
had 21,934 students enrolled, 14,771 of
whom were active in the course. Our 26
lecture videos were viewed 95,631 times.
Students submitted work for evaluation
2,942 times and completed 19,571 peer
assessments (the means by which their
writing was evaluated). However, only
238 students received a completion certificate—meaning that they completed all assignments and received satisfactory scores.
Our team is now investigating why so
few students completed the course, but we
have some hypotheses. For one thing, students who did not complete all three major assignments could not pass the course.
Many struggled with technology, especially in the final assignment, in which
they were asked to create a video presentation based on a personal philosophy
or belief. Some students, for privacy and
cultural reasons, chose not to complete
that assignment, even when we changed
the guidelines to require only an audio
moocs need more Work;
So Do cS Graduates
Mark Guzdial assesses the first full year of massive open online courses,
while Joel C. Adams considers the employment outlook for CS grads.
DOI: 10.1145/2555813 http://cacm.acm.org/blogs/blog-cacm