Communications of the ACM

contributed articles

OF ALL THE experiences we have in life, face-to-face interaction fills many of our most meaningful moments.

The complex interplay of facial expressions, eye gaze, head movements, and vocalizations in quickly evolving “social interaction loops” has enormous influence on how a situation will unfold. From birth, these interactions are a fundamental element of learning and lay the foundation for successful social and emotional functioning through life.

What are the underlying processes from which this
most human form of interaction emerges? Will we be
able to interact with computers in a face-to-face way
that feels natural? This article discusses the unique
challenges of realistically simulating the appearance and
behavior of the face to create interactive autonomous
virtual human models that support naturalistic
learning and have the “illusion of life.” We describe our
recent progress toward this goal with “BabyX,” an
autonomously animated psycho-
biological model of a virtual infant.
While we explore drivers of facial
behavior, we also expect this founda-
tional approach has the potential for
more “human” computer interfaces.
We also describe our work on our
“Auckland Face Simulator” we are de-
veloping to broaden this work beyond
infants and give a more realistic face
and a greater biological basis to adult
conversational agents.
Simulating the face has great poten-
tial for human-computer interaction
(HCI), as it increases the available com-
munication channels between humans
and machines in an intuitive, accessible
way. But it is also a vehicle with which
to explore our own nature. Akin to de-
velopmental robotics, 6 which explores
ways of learning and mental develop-
ment through child-like robots, simu-
lating the underlying processes driving
the face during social interaction will
enable HCI researchers to explore be-
havioral and learning models involving
naturalistic face-to-face interaction.
There is a trend in the game and
visual-effects industries to create ever
more realistic animated characters,
especially humans, but it turns out to
not be a straightforward transition
from the stylized faces of traditional
animation. For these industries, real-
ism of appearance and movement is
very important, evidenced by the large
financial investment going toward cre-
ating the most realistic illusion they
can achieve. This is done presumably
because the experience becomes more
immersive and powerful the closer it is
Creating
Connection
with Autonomous
Facial Animation
DOI: 10.1145/2950041

Biologically based computational modeling
promises virtual characters capable of
face-to-face human interaction.
BY MARK SAGAR, MIKE SEYMOUR, AND ANNETTE HENDERSON
key insights

˽ The expressive and communicative power of the face has been untapped in HCI but can indeed create deeper human-machine connection and engagement.

˽ The holistic interplay of biologically based computational behavioral models driving a virtual character can give rise to an emotionally affecting experience.