to the top of a list of challenges for MOOC educators:

˲ Business Models. No one has yet determined how to make the MOOC model self-sustaining. Educators say the current emphasis is just on delivering content; the rest, they say, will work itself out “eventually.”

˲ Student Evaluations. How does an instructor effectively grade students who are doing homework and labs, writing essays, creating projects, and taking exams online? It’s easy when the tests are multiple choice or when the work is peer-graded, but technology still needs to catch up to evaluating the rest.

In the case of Norvig and Thrun’s AI class, the tests and quizzes were all multiple choice or fill-in-the-blank and could be graded automatically.

“It takes some effort to convert an
open-ended question into a fill-in-the-
blank question,” recalls Norvig, “but
with some practice it can be done, at
least for classes in technical subjects.
For other, more open-ended subjects,
like literature, one would have to use
peer-assessment or self-assessment.
Scott Klemmer of Stanford has a paper
out [see the “Further Reading” list] in
which he describes the successful use
of self-assessment in the design course
he taught.”
And, just as in physical classrooms,
there is a trust issue: How does one au-
thenticate that the person doing the
work is the person who signed up for
the class?

˲ Certifications. Because most MOOCs are not part of degree programs, what do students receive after completing a course? In most cases, it is a certificate. But is completing a free online course the same as having paid for and completed a course on campus? How potential employers will deal with this branding issue and how they will evaluate the meaning of, say, a certificate that testifies to a student’s taking an online Stanford course versus getting a degree from Stanford still needs to be played out.

“Students who put classes on their resumes who have not received official credit or degrees is, indeed, an issue,” says Norvig. “I think there will be a variety of accreditation schemes and we’ll just have to wait and see how valuable each of them turns out to be.”

“i think there’s
a tremendous value
to what we’re doing,”
says Daphne Koller.
“We just haven’t
decided yet how to
make it pay for itself.”

Meanwhile, Carnegie Mellon University, through its Open Learning Initiative, has been providing online learning for over a decade, with a focus on systematic student assessment.

“We’ve concentrated more on the science of learning than on delivering courses to the masses,” says Jeannette Wing, professor of computer science and department head of Carnegie Mellon’s computer science department. “But now, because the media has been focusing so heavily on course delivery, we have been having internal discussions about where we should be positioning Carnegie Mellon in this space. I can’t predict what decisions will be made, but I can say this is the number one topic on campus—just as it is the number one topic on many campuses right now, especially in the computer sciences.”

full speed ahead

Other universities, however, are rapidly moving forward with delivering classes online, using various delivery platforms.

In mid-April, for example, two Stanford University computer science professors, Daphne Koller and Andrew Ng, launched Coursera, a for-profit “social entrepreneurship company” that partners with a growing number of universities—including Stanford, University of Michigan, Princeton, and University of Pennsylvania—and presents online many of the universities’ courses in the fields of computer science, biology, medicine, the humanities, and more.

“In October [2011], for the very first
time, Stanford made available on-
line—and at no charge—three of its
most popular computer science cours-
es,” says Koller. “More than 300,000
students registered for ‘Introduction
to Artificial Intelligence,’ ‘Introduc-
tion to Databases,’ and ‘Machine
Learning.’ Seeing the success of that
experiment, my colleague, Andrew Ng,
and I decided we needed to build on it
but couldn’t just make it a Stanford-
specific effort. Great as Stanford is,
there are other universities that offer
amazing content currently to only a
tiny sliver of the world’s population.
So we decided we’d spin this out of
Stanford and open the opportunities
to other top institutions to engage
with this amazing experiment on free
access to top education for everyone.”
While Coursera was initially funded
by a $16 million venture-capital grant,
Koller and Ng are still debating how to
make their efforts sustainable.