to, and learn from, the interaction as
they jointly determine its content and
quality through real-time contingent
and reciprocal coaction.”
Another important factor in the ac-
ceptance of a virtual human face is its
visual quality. However, realistically
simulating a face—even when still—
has proved to be formidable.
Challenges of Modeling
the Human Face
The way a digital face moves and appears can cause unwanted effects.
Rather than aiding the appearance of
life, a partially realistic solution can
elicit a negative response—the uncanny valley effect. This response is
thought to be triggered by any number
of non-expected responses, alarming
the viewer’s perceptual system.
19 To
avoid this response, many factors must
be taken into account, including the
physical appearance of the face and the
eye-gaze movement of the skin, any of
which can trigger a form of dissonance
that interferes with the affinity of the
perceived face.
Appearance. The ability to “read”
faces is so important that several different parts of the brain play a role in
face perception. We are sensitive to
many factors that act as signs of health
and vitality. People often refer to someone being “as white as a sheet,” “red
faced,” or “sickly looking”; it is thus
important to render physically plausible healthy skin with correct surface
properties, detail, and subsurface scattering of light that provides diffuse
properties of skin.
This challenge has been approached
in two broadly different ways. First, by
using “image-based methods” that
sample the face under different light-
ing and viewing conditions9 and then
render the face through a combina-
tion of weighted image-blended sets,
photogrammetry, and/or image pro-
jection. Second, by using “parametric
methods” that fit the captured data to
a face and material model used during
rendering, allowing for more flexibility
but at the cost of potentially increased
rendering complexity.
16 Given the con-
straints producing imagery fast enough
for user interaction, adding further to
the complexity of achieving an effec-
tive interactive face, a simplified imple-
mentation of the second approach is
dog “Silas” developed by Blumberg3 us-
ing sophisticated ethological models
to simulate how animals are able to or-
ganize and adapt.
These and similarly inspired works
are important on many levels, as they
explain how behavior can emerge,
made observable through animation
with constraints. Blumberg3 suggested
for a creature to appear alive it must react, have goals, make choices, convey
its intentionality, emotionally respond
to events, adapt, and vary its movement
and response.
For autonomous animation of the
face in real time, Terzopolous and Lee37
developed a physics-based face model
driven by a basic behavioral animation
model. Despite this pioneering work,
few other virtual human studies have
focused on this lower level of detail in
real-time facial animation.
Most research in building autonomous human agents has been as
“embodied conversational agents”
(such as in Allbeck1 and in Cassell7)
at a generally more phenomenologi-cal and higher level, not specifically
focused on the subtler details of facial
expression and nonverbal behavior;
Vinciarelli et al.
39 included a survey of
social-signal processing in computer
interaction. Simulating these signals
is getting greater attention from the
interdisciplinary “intelligent virtual
agent” community,
5 exploring agents
that are capable of real-time perception, cognition, and actions in the
social environment; Marsella and
Gratch21 discussed simulations of psychological theories of emotion. Emo-tion-oriented APIs (such as SEMAINE)
have been developed.
33 And Scherer32
showed how cumulative effects of sequential checks of an eliciting event,
mediated by autonomic and somatic
components, might combine to create
compound facial expressions.
Much of the work on virtual humans
has an unfortunately robotic “feeling,”
particularly with facial interaction.
This is possibly due to most virtual human models not focusing on the microdynamics of expression or on facial
realism. These microdynamics are considered particularly critical in learning
contexts. Rohlfling and Deak27 stated:
“When infants learn in a social environment, they do not simply pick up
information passively. They respond
The dynamic
behavior of the face
emerges from many
systems interacting
on multiple levels,
from high-level
social interaction
to low-level biology.