simulation, scanning, and geometric modeling. Fine details of visually important elements (such as the mouth, eyes, eyelashes, and eyelid geometry) are painstakingly constructed for life-like reality (see Figure 4).

A highly detailed biomechanical face model, as in Figure 3, has been constructed from MRI scans and anatomic reference, akin to Wu. 40 Skin deformation is generated by individual or grouped-muscle activations. We have modeled the deep and superficial fat, as well as muscle, fascia, connective tissue, and their various properties. We have used large-deformation finite-element elasticity40 to deform the face from rest position through simulated muscle activation. Individual and combined muscle activations were simulated to form an expression space, 30 interpolated on the fly in BL as the face animates. The response to muscle activation is consistent skin deformation and motion.

Nervous system. BabyX’s biologically inspired nervous system consists of an interconnected set of neural system and subsystem models. The models implemented so far are sparse yet span the neuroaxis and generate muscle-activa-tion-based animation as motor output from continuously integrated neural network models. Due to the Lego-like nature of BL, we can have a closed-loop functioning system allowing experimental interchange of components while exploring different theoretical models. In total, the models aim to form a “large functioning sketch” of interconnected mechanistic systems contributing to behavior, containing both top-down and bottom up mechanisms interacting as an integrated system.

BabyX’s neural networks and circuits implemented so far cover basic elements of motor control, behavior selection, reflex actions, visual attention, learning, salience, emotion, and motivation. An architectural diagram relating some of the key functional components, neuroanatomical structures, and functional loops is included in Figure 5; note cortical and subcortical input to the facial nucleus. A characteristic of this modeling approach is the representation of subcortical structures (such as the basal ganglia)

and brainstem nuclei (such as the oc-
ulomotor nuclei). The structures are
functionally implemented as neural
can be shared and drive any aspect of
a sophisticated 3D animation system;
for more detail on BL, see Sagar et al. 28

BabyX Project

To illustrate how these concepts come together, we describe an experimental psychobiological simulation of an infant we call “BabyX,” 29, 31 that aims to embody models of interactive behavior and social learning to create an autonomous virtual infant one can interact with naturally.

Facial expression. At a conceptual
level, BabyX’s computer-graphic face
model is driven by muscle activations
generated from motor-neuron activ-
ity. The facial expressions are created
by modeling the effect of individual
muscle activations and their non-lin-
ear combination forming her range of
expressions, as in Figure 3. The model-
ing procedures involve biomechanical
networks can be visually investigated in
multiple ways, either through 3D com-
puter graphics (see Figure 1) or through
a 2D schematic interface showing activ-
ity on the virtual connectome (see Fig-
ure 2). Individual variable activity (such
as neuron voltage or firing rate) can be
inspected with scopes during a simula-
tion, as in Figure 2 left. The simulation
can be interactively modified (such as
by changing neural-model parameters
while viewing the effect on the anima-
tion) (see Figure 3 right).

Sensory input is typically through camera, microphone, and keyboard to enable computer vision audition and “touch” processing, but data can be input from any arbitrary sensor or output to an effector through the API. Computer graphics output is through OpenGL and the OpenGL Shading Language. A key feature of BL is that any variable in the neural network system

Figure 2. Screenshot of interactive BL viewer: (left) BL scopes viewing activity of a single neuron (top) or an array of retinotopic neurons (bottom) during a live interaction; (right) partial view of BabyX’s virtual connectome, which can be explored interactively; connections light up (green or red) when activated.

Figure 1. BabyX’s interactive brain: (left) superior colliculus activity driving visual attention is visible (green) in the brainstem; (middle) BL raster plot of neural activity and scrolling display of modulatory activity; (right) basal ganglia circuit and interactive dopamine level modification (green) affecting cortico-thalamic feedback and eye movement.