simulation, scanning, and geometric
modeling. Fine details of visually important elements (such as the mouth,
eyes, eyelashes, and eyelid geometry)
are painstakingly constructed for life-like reality (see Figure 4).
A highly detailed biomechanical
face model, as in Figure 3, has been
constructed from MRI scans and anatomic reference, akin to Wu.
40 Skin deformation is generated by individual or
grouped-muscle activations. We have
modeled the deep and superficial fat, as
well as muscle, fascia, connective tissue,
and their various properties. We have
used large-deformation finite-element
elasticity40 to deform the face from rest
position through simulated muscle activation. Individual and combined muscle activations were simulated to form
an expression space,
30 interpolated on
the fly in BL as the face animates. The
response to muscle activation is consistent skin deformation and motion.
Nervous system. BabyX’s biologically
inspired nervous system consists of an
interconnected set of neural system and
subsystem models. The models implemented so far are sparse yet span the
neuroaxis and generate muscle-activa-tion-based animation as motor output
from continuously integrated neural
network models. Due to the Lego-like
nature of BL, we can have a closed-loop functioning system allowing experimental interchange of components
while exploring different theoretical
models. In total, the models aim to
form a “large functioning sketch” of interconnected mechanistic systems contributing to behavior, containing both
top-down and bottom up mechanisms
interacting as an integrated system.
BabyX’s neural networks and circuits implemented so far cover basic
elements of motor control, behavior
selection, reflex actions, visual attention, learning, salience, emotion, and
motivation. An architectural diagram
relating some of the key functional
components, neuroanatomical structures, and functional loops is included
in Figure 5; note cortical and subcortical input to the facial nucleus. A characteristic of this modeling approach
is the representation of subcortical
structures (such as the basal ganglia)
and brainstem nuclei (such as the oc-
ulomotor nuclei). The structures are
functionally implemented as neural
can be shared and drive any aspect of
a sophisticated 3D animation system;
for more detail on BL, see Sagar et al.
28
BabyX Project
To illustrate how these concepts come
together, we describe an experimental
psychobiological simulation of an infant we call “BabyX,”
29, 31 that aims to
embody models of interactive behavior
and social learning to create an autonomous virtual infant one can interact
with naturally.
Facial expression. At a conceptual
level, BabyX’s computer-graphic face
model is driven by muscle activations
generated from motor-neuron activ-
ity. The facial expressions are created
by modeling the effect of individual
muscle activations and their non-lin-
ear combination forming her range of
expressions, as in Figure 3. The model-
ing procedures involve biomechanical
networks can be visually investigated in
multiple ways, either through 3D com-
puter graphics (see Figure 1) or through
a 2D schematic interface showing activ-
ity on the virtual connectome (see Fig-
ure 2). Individual variable activity (such
as neuron voltage or firing rate) can be
inspected with scopes during a simula-
tion, as in Figure 2 left. The simulation
can be interactively modified (such as
by changing neural-model parameters
while viewing the effect on the anima-
tion) (see Figure 3 right).
Sensory input is typically through
camera, microphone, and keyboard
to enable computer vision audition
and “touch” processing, but data can
be input from any arbitrary sensor or
output to an effector through the API.
Computer graphics output is through
OpenGL and the OpenGL Shading Language. A key feature of BL is that any
variable in the neural network system
Figure 2. Screenshot of interactive BL viewer: (left) BL scopes viewing activity of a single
neuron (top) or an array of retinotopic neurons (bottom) during a live interaction; (right)
partial view of BabyX’s virtual connectome, which can be explored interactively;
connections light up (green or red) when activated.
Figure 1. BabyX’s interactive brain: (left) superior colliculus activity driving visual attention
is visible (green) in the brainstem; (middle) BL raster plot of neural activity and scrolling
display of modulatory activity; (right) basal ganglia circuit and interactive dopamine level
modification (green) affecting cortico-thalamic feedback and eye movement.