reasoning is required to compute the
function outputs from its inputs. The
main tool of this approach is the neural
network. Many college students have
exercised a version of it in a physics
or chemistry lab, where they fit simple
functions to data collected from vari-
ous experiments, as in Figure 2. The
main difference here is we are now em-
ploying functions with multiple inputs
and outputs; the structure of these
functions can be quite complex; and
the problems being tackled are ones
we tend to associate with perception or
cognition, as opposed to, say, estimat-
ing the relationship between volume
and pressure in a sealed container.d
The main observation in AI recently
is that the function-based approach
can be quite effective at certain AI
tasks, more so than the model-based
approach or at least earlier attempts at
using this approach. This has surprised
not only mainstream AI researchers,
who mainly practice the model-based
approach, but also machine learning
researchers who practice various ap-
proaches, of which the function-based
approach is but one.e This has had
many implications, some positive and
some giving grounds for concern.

On the positive side is the increasing number of tasks and applications now within reach, using a tool that can be very familiar to someone with only a broad engineering background, particularly one accustomed to estimating functions and using them to make predictions. What is of concern, however, is the current imbalance between exploiting, enjoying, and cheering this tool on the one hand and thinking about it on the other. This thinking is not only important for realizing the full potential of the tool but also for scientifically characterizing its potential d This is also called the “curve-fitting” approach. While the term “curve” highlights the efficient evaluation of a function and captures the spirit of the function-based approach, it underplays the complex and rich structure of functions encoded by today’s (deep) neural networks, which can have millions if not billions of parameters.

e Machine learning includes the function-based

approach but has a wide enough span that it overlaps with the model-based approach; for example, one can learn the parameters and structure of a model but may still need non-trivial reasoning to obtain answers from the learned model.

that does not require explicit model-
ing or sophisticated reasoning is suf-
ficient for reproducing human-level
intelligence. This dilemma is further
amplified by the observation that re-
cent developments did not culminate
in a clearly characterized and profound
scientific discovery (such as a new
theory of the mind) that would nor-
mally mandate massive updates to the
AI curricula. Scholars from outside AI
and computer science often sense this
dilemma, as they complain they are
not receiving an intellectually satisfy-
ing answer to the question: “What just
happened in AI?”
The answer lies in a careful assess-
ment of what we managed to achieve
with deep learning and in identifying
and appreciating the key scientific out-
comes of recent developments in this
area of research. This has unfortunate-
ly been lacking to a great extent. My
aim here is to trigger such a discussion,
encouraged by the positive and curious
feedback I have been receiving on the
thoughts expressed in this article.

Background

To lay the ground for the discussion, I first mark two distinct approaches for tackling problems that have been of interest to AI. I call the first one “ model-based” and the second “ function-based.” Consider the object-recogni-tion and -localization task in Figure 1. To solve it, the model-based approach requires one to represent knowledge about dogs and hats, among other things, and involves reasoning with such knowledge. The main tools of the approach today are logic and probability (mathematical modeling more generally) and can be thought of as the “represent-and-reason”c approach originally envisioned by the founders of AI. It is also the approach normally expected, at some level, by informed members of the scientific community. The function-based approach, on the other hand, formulates this task as a function-fitting problem, with function inputs coming directly from the image pixels and outputs corresponding to the high-level recognitions we seek. The function must have a form that can be evaluated efficiently so no c This term might be likened to what has been called “good old-fashioned AI.”

In my own quest
to fully appreciate
the progress
enabled by deep
learning, I came
to the conclusion
that recent
developments tell
us more about the
problems tackled
and the structure
of our world than
about neural
networks per se.