of DARPA’s L2M program and a computer science professor at the University of Massachusetts, Amherst. “We
will never be safe in a self-driving car
without it,” she says. But it is just one
of many necessary steps toward that
goal. “It’s definitely not the end of the
story,” she says.
There are five “pillars” of lifelong
learning as DARPA broadly defines
it, and synaptic plasticity falls into
the first of these. The pillars are: continuous updating of memory, without
catastrophic forgetting; recombinant
memory, rearranging and recombining previously learned information toward future behavior; context
awareness and context based modulation of system behavior; adoption of
new behaviors through internal play,
self-awareness, and self-simulations;
and safety and security, recognizing
whether something is dangerous and
changing behavior accordingly, and
ensuring security through a combination of strong constraints.
Siegelmann cites smart prosthe-ses as an example of an application of
these techniques. She says the control
software in an artificial leg could be
trained via conventional backpropagation by its maker, then trained to
the unique habits and characteristics
of its user, and finally enabled to very
quickly adapt to a situation it has not
seen before, such as an icy sidewalk.
A computational neuroscientist,
Siegelmann says lifelong learning has
been a goal of AI researchers for many
years, but major advancements have
only recently become feasible, en-
abled by advancements in computer
power, new theoretical foundations
and algorithms, and a better under-
standing of biology. “In a few years,
much of what we call AI today won’t be
considered AI without lifelong learn-
ing,” she predicts.
Miconi’s team is now working on
making learning more dynamic and
sophisticated than it is in his test systems so far. One way to do that is to
make the plasticity coefficients, now
fixed as a design choice, themselves
variable over the life of a system. “The
plasticity of each connection can be
determined at every point by the network itself,” he says. Such “
neuro-modulation” likely occurs in animal
brains, he says, and that may be a key
step toward the most flexible decision-making by AI systems.
Further Reading
Chang, O. and Lipson, H.
Neural Network Quine,
Data Science Institute, Columbia
University, New York, NY 10027, May 2018
https://arxiv.org/abs/1803.05859v3
Chen, Z. and Liu, B.
Lifelong Machine Learning, Second Edition,
Synthesis Lectures on Artificial Intelligence
and Machine Learning, August 2018
https://www.morganclaypool.
com/doi/10.2200/
S00832ED1V01Y201802AIM037
Hebb, D.
The Organization of Behavior: A
Neuropsychological Theory, New York:
Wiley & Sons, 1949
http://s-f-walker.org.uk/pubsebooks/pdfs/The_
Organization_of_Behavior-Donald_O._Hebb.pdf
Miconi, T., Clune, J., and Stanley, K.
Differentiable Plasticity: Training Plastic
Neural Networks with Backpropagation,
Proceedings of the 35th International
Conference on Machine Learning (ICML
2018), Stockholm, Sweden, PMLR 80, 2018
https://arxiv.org/abs/1804.02464
Miconi, T.
Backpropagation of Hebbian Plasticity
for Continual Learning,
NIPS Workshop on Continual Learning, 2016
https://github.com/ThomasMiconi/
LearningToLearnBOHP/blob/master/paper/
abstract.pdf
Gary Anthes is a technology writer and editor based in
Arlington, VA, USA
© 2019 ACM 0001-0782/19/6 $15.00
highly accurate results when classify-
ing different kinds of automobiles,
but when a new kind of car (a Tesla,
say) is seen, the system stumbles.
“You want it to recognize this new
car very quickly, without retraining,
which can take days or weeks. Also,
how do you know that something new
has happened?”
Artificial intelligence systems that
learn on the fly are not new. In “neu-
roevolution,” networks update them-
selves by algorithms that employ a
trial-and-error method to achieve a
precisely defined objective, such as
winning a game of chess. They require
no labeled training examples, only
definitions of success. “They go only
by trial and error,” says Uber’s Miconi.
“It’s a powerful, but a very slow, es-
sentially random, process. It would
be much better if, when you see a new
thing, you get an error signal that tells
you in which direction to alter your
weights. That’s what backpropagation
gets you.”
Military Apps
Miconi’s ideas represent just one of
a number of new approaches to self-
learning in AI. The U.S. Department of
Defense is pursuing the idea of synap-
tic plasticity as part of a broad family
of experimental approaches aimed at
making defense systems more accu-
rate, responsive, and safe. The U.S.
Defense Advanced Research Projects
Agency (DARPA) has established a
Lifelong Learning Machines (L2M)
program with two major thrusts, one
focused on the development of com-
plete systems and their components,
and the second on exploring learning
mechanisms in biological organisms
and translating them into computa-
tional processes. The goals are to en-
able AI systems to “learn and improve
during tasks, apply previous skills
and knowledge to new situations, in-
corporate innate system limits, and
enhance safety in automated assign-
ments,” DARPA says at its website.
“We are not looking for incremental
improvements, but rather paradigm-
changing approaches to machine
learning.”
Uber’s work with Hebbian plastic-
ity is a promising step toward lifelong
learning in neural networks, says Hava
Siegelmann, founder and manager
DARPA’s
Lifelong Learning
Machines program
does not seek
incremental
improvements,
“but rather
paradigm-changing
approaches to
machine learning.”