some of the most innovative ideas will come from people much younger than us.

The progress in the field has been amaz-
ing. What would you have been surprised
to learn was possible 20 or 30 years ago?

LECUN: There’s so much I’ve been surprised by. I was surprised by how late the deep learning revolution was, but also by how fast it developed once it started. I would have expected things to happen more progressively, but people abandoned the whole idea of neural nets between the mid-1990s and mid-2000s. We had evidence that they were working before, but then, once the demonstrations became incontrovertible, the revolution happened really fast, first in speech recognition, then in image recognition, and now in natural language understanding.

HIN TON: I would have been amazed, 20 years ago, if someone had said that you could take a sentence in one language,

carve it up into little word fragments,
feed it into a neural net that starts with
people are basically massive analogy-
making machines. They develop these
representations quite slowly, and then
the representations they develop deter-
mine the kinds of analogies they can
make. Of course, we can do reasoning,
and we wouldn’t have mathematics
without it, but it’s not the fundamental
way we think.

For pioneering researchers, you seem un-
usually unwilling to rest on your laurels.

HINTON: I think there’s something special about people who invented techniques that are now standard. There was nothing God-given about them, and there could well be other techniques that are better. Whereas people who come to a field when there’s already a standard way of doing things don’t understand quite how arbitrary that standard way is.

BENGIO: Students sometimes talk about neural nets as if they were describing the Bible.

LECUN: It creates a generation of dog-
matism. Nevertheless, it’s very likely that
keep the
connection alive with people who are try-
ing to understand how the brain works.
HINTON: That said, neuroscientists
are now taking it seriously. For many
years, neuroscientists said, “artificial
neural networks are so unlike the real
brain, and they’re not going to tell us
anything about how the brain works.”
Now, neuroscientists are taking seri-
ously the possibility that something
like backpropagation is going on in the
brain, and that’s a very exciting area.

LECUN: Almost all the studies now of human and animal vision use convolutional nets as the standard conceptual model. That wasn’t the case until relatively recently.

HINTON: I think it’s also going to have a huge impact, slowly, on the social sciences, because it’s going to change our view of what people are. We used to think of people as rational beings, and what was special about people was that they used reasoning to derive conclusions. Now we understand much better that [CONTINUED FROM P. 96]

P H O T O B Y A L E X A N D E R B E R G