I M A G E C O U R T E S Y O F X E N O N S T A C K . C O M / B L O G

Deep Learning Hunts for
Signals Among the Noise

Neural networks can deliver surprising, and sometimes unwanted, results.

Science | DOI: 10.1145/3204445 Chris Edwards

OVER THE PAST decade, ad- vances in deep learning have transformed the fortunes of the artificial intelligence (AI) community. The neural network approach that researchers had largely written off by the end of the 1990s now seems likely to become the most widespread technology in machine learning. However, protagonists find it difficult to explain why deep learning often works well, but is prone to seemingly bizarre failures.

The success of deep learning came with rapid improvements in computational power that came through the development of highly parallelized microprocessors and the discovery of ways to train networks with enormous numbers of virtual neurons assembled into tens of linked layers. Before these advances, neural networks were limited to simple structures that were easily outclassed in image and audio classification tasks by other machine-learning architectures such as support vector machines.

Theorists have long assumed networks with hundreds of thousands of neurons and orders of magnitude more individually weighted connections between them should suffer from a fundamental problem: over-parameterization. There are so many weights that determine how much each neuron influences its neighbors that the network could simply find a way to encode the data used to train it. It would then correctly classify anything in the training set, but fail miserably when presented with new data.

In practice, deep neural networks
do not fall easily into overparameter-
ization; instead, they are surprisingly
good at dealing with new data. When
trained, they seem able to ignore parts
of images used for training that had
little bearing on classification perfor-
mance, rather than trying to build syn-
aptic connections to deal with them.
ly misinterpret the contents,” says
Chiyuan Zhang, a researcher working
at the Center for Brains, Mind and
Machines based at the Massachu-
setts Institute of Technology (MIT).
“This leads to security concerns. It
could potentially be used to implant
backdoors in neural network models
in ways that are hard to identify.”
Mathematical interpretations of
how deep neural networks learn offer
one path to understanding why they
generalize so effectively, and may pro-
vide mechanisms for them to avoid
training on the wrong types of feature.
Researchers regard the layering used
by deep learning as one vital attribute.
The layers make it possible to pull iden-
tifying marks out of images no matter
where they are within the sample.

However, that is only part of the problem.

Tomaso Poggio, principal investiga-
tor at the McGovern Institute for Brain
Research based at MIT, says, “It is im-
portant to understand there is much
more work to be done [in deep learn-
ing]. Our hope is that if we understand
better how they work we will under-
stand better how they fail and, by doing
that, improve them.”
One strand of math-oriented re-
search focuses on information theory.
Naftali Tishby of the Hebrew University
“Most of the variability in images is
irrelevant to the task. For instance,
if you want to recognize a friend in a
picture, you want to do so regardless
of where she will be, how she will be
dressed, whether she is partially oc-
cluded, what sensor will be used for
the picture, etc. If you think of all the
possible images of your friend, they
are, for all practical purposes, infi-
nite. So if you wanted a minimal rep-
resentation—something that distills
the essence of ‘your friend’ in every
possible future image of her—that
should be a much, much smaller ob-
ject than an image.”
Unfortunately, networks can home in
on details that are very different to those
used by humans. This leads to some-
times intriguing failures. Researchers at
Kyushu University in Japan discovered
late last year that modification of just
one individual pixel in an image could
upset neural networks trained to clas-
sify objects and animals; a taxi might
suddenly be misidentified as a dog with
such a tiny change.

˲ “Trained neural networks can be tricked to focus on patterns in images that are barely noticeable by humans into a situation where they complete-

 

Simple Neural Network
Input Layer Hidden Layer Output Layer
Deep Learning Neural Network

A simple neural network has up to two layers hidden between the input and output layers; more than that, and it becomes a Deep Learning Neural Network, which can model complex non-linear relationships.