some of the most innovative ideas will
come from people much younger than us.
The progress in the field has been amaz-
ing. What would you have been surprised
to learn was possible 20 or 30 years ago?
LECUN: There’s so much I’ve been
surprised by. I was surprised by how
late the deep learning revolution was,
but also by how fast it developed once
it started. I would have expected things
to happen more progressively, but people abandoned the whole idea of neural nets between the mid-1990s and
mid-2000s. We had evidence that they
were working before, but then, once
the demonstrations became incontrovertible, the revolution happened
really fast, first in speech recognition,
then in image recognition, and now in
natural language understanding.
HIN TON: I would have been amazed, 20
years ago, if someone had said that you
could take a sentence in one language,
carve it up into little word fragments,
For pioneering researchers, you seem un-
feed it into a neural net that starts with
people are basically massive analogy-
making machines. They develop these
representations quite slowly, and then
the representations they develop deter-
mine the kinds of analogies they can
make. Of course, we can do reasoning,
and we wouldn’t have mathematics
without it, but it’s not the fundamental
way we think.
usually unwilling to rest on your laurels.
HINTON: I think there’s something
special about people who invented techniques that are now standard. There
was nothing God-given about them, and
there could well be other techniques that
are better. Whereas people who come to
a field when there’s already a standard
way of doing things don’t understand
quite how arbitrary that standard way is.
BENGIO: Students sometimes talk
about neural nets as if they were describing the Bible.
LECUN: It creates a generation of dog-
matism. Nevertheless, it’s very likely that
connection alive with people who are try-
ing to understand how the brain works.
HINTON: That said, neuroscientists
are now taking it seriously. For many
years, neuroscientists said, “artificial
neural networks are so unlike the real
brain, and they’re not going to tell us
anything about how the brain works.”
Now, neuroscientists are taking seri-
ously the possibility that something
like backpropagation is going on in the
brain, and that’s a very exciting area.
LECUN: Almost all the studies now of
human and animal vision use convolutional nets as the standard conceptual
model. That wasn’t the case until relatively recently.
HINTON: I think it’s also going to have
a huge impact, slowly, on the social sciences, because it’s going to change our
view of what people are. We used to think
of people as rational beings, and what
was special about people was that they
used reasoning to derive conclusions.
Now we understand much better that
[CONTINUED FROM P. 96]