The main challenge
ahead will be to

Boulder have led to the development of a cognitive architecture that combines the functionality of a visual neuroscience model with traditional rule-based elements [ 5]. Such hybrid architectures may represent the future of cognitive modeling approaches to usability analyses.

demonstrate that a
neuroscience approach
to HCI adds value
beyond what can be

Inside the User’s Head

Only a small percentage of current neuroscience research is explicitly aimed at understanding aspects of HCI. Nonetheless, some recent neuroimaging experiments point to ways in which experimental neuroscience methodology might be leveraged to measure facets of the user-interaction experience at a deeper level than can be achieved with other contemporary methods. For instance, modern neuroscience has begun to characterize the brain circuitry that governs reward-related behaviors, with fMRI experiments revealing that unexpected rewards elicit activation in areas of the human brain that utilize the chemical transmitter dopamine [ 6]. (A reward in these experiments is typically anything from a squirt of juice to a $10 bill.) These studies raise the intriguing possibility that neuroimaging techniques might someday be used to identify which aspects of the interaction experience a user finds pleasing.

Another example of applicable neuroscience research comes from a series of experiments examining how humans perceive computer-animated characters that vary in their degree of physical realism. One study showed that the tendency of a subject to perceive

a virtual character as realistic is correlated with activation in areas of the brain known to play a role “mentalizing” [ 7]. Mentalizing refers to our ability to place ourselves in the mind of another person and predict their intentions. It is fundamental to human social interaction. Research points to the possibility that neuroimaging methods might be used to assess the degree in which a user perceives a virtual entity (for instance, an avatar) as a fellow autonomous being, or merely as a non-sentient computer artifact.

The main challenge ahead will be to demonstrate that a neuroscientific approach to HCI adds value beyond what can be gleaned from behavioral studies alone. If other disciplines offer any indication, the outlook is promising. Consider the medical field, where tests that reveal what is going on “under the hood” (angiograms, throat cultures, or simple blood tests) are ordered precisely because they provide diagnostic value beyond what is available through observation of the patient’s symptoms alone. And although the user (like the patient) possesses a unique awareness of what’s happening inside his or her own brain (or body), and therefore can provide useful information simply by describing his or her own thought processes, an individual’s ability to introspect is limited. In fact, a major thrust of modern psychological research is focused on mapping the extent of so-called implicit cognition—that vast chunk of the cognitive iceberg that floats beneath the surface of conscious thought but drives

gleaned from behavioral
studies alone.

behavior in powerful ways [ 8]. Neuroscience will likely make valuable contributions to the discipline of HCI by providing a richer account of user cognition than that which is obtained from any other source, including the user himself.

 

Current Research As described here, vision is one of the brain’s most extensively studied subsystems. In addition, the brain’s memory circuits have also been the subject of intense research. Since visual perception and memory are key areas of study in HCI, neuroscience-based models of these functions may be particularly well poised to have an impact on HCI research. In our work here at the MITRE Corporation, we are exploiting models of visual attention and memory to predict how visual display properties influence perception and recall by users. As part of this study, we are implementing a neurocompu-tational model of visual attention developed by researchers

[ 2] Wilson, G. F. and
C.A. Russell. “Operator
functional state clas-
sification using multiple
psychophysiological
features in an air traffic
control task.” Human
Factors 45, no. 3 (2003):
381-389.
[ 3] Wilson, G.F. and C. A.
Russell. “Performance
enhancement in an
uninhabited air vehicle
task using psychophysi-
ologically determined
adaptive aiding.” Human
Factors 49, no. 6 (2007):
1005-1018.
[ 4] Muller, K.R.,
M. Tangermann,
G. Dornheg, M.
Krauledat, G. Curio,
and B. Blankertz.
“Machine learning
for real-time single-
trial EEG-analysis: from
brain-computer inter-
facing to mental state
monitoring.” Journal of
Neuroscience Methods
167, no. 1 (2008): 82-90.
[ 5] Jilk, D. J., C. Lebiere,
R.C. O’Reilly, and J.R.
Anderson. “SAL: An
explicitly pluralistic
cognitive architecture.”
Journal of Experimental
& Theoretical Artificial
Intelligence 20, no. 3
(2008): 197-218.
[ 6] D’Ardenne, K., S.M.
McClure, L.E. Nystrom,
and J.D. Cohen. “BOLD
Responses Reflecting
Dopaminergic Signals
In the Human Ventral
Tegmental Area.”
Science 319, no. 5867
(2008): 1264-1267.