to reflect the activity of both “bottom-up” perceptual processes and “top down” interpretive processes that play a role in recognizing and coordinating a response to critical task-relevant information. The ERP signal originates in the visual areas of the brain, and within a few hundred milliseconds, propagates to frontal areas responsible for interpretation. The ERP signal has several advantages over an overt response. The ERP signal precedes an overt physical response by several hundred milliseconds. It also exhibits far less variability than an overt physical response since it does not require the deliberation needed to initiate a physical response.

sional image analysts and users drawn from the general population. Users searched for a variety of targets in satellite imagery— including ships, surface-to-air missile sites, oil-storage depots, and golf courses. Compared with conventional broad area image analysis, the neurally driven search technique has produced a sixfold speed-up in search efficiency with no loss of accuracy.

While the approach just described has shown promise as a way to boost the efficiency of searching for information within imagery, there are several important issues that have to be addressed. As a manual search technique, performance hinges on human perceptual abilities. Targets embedded in cluttered scenes can be difficult to detect. Targets are also difficult to detect as they move away from the center of a user’s field of view—there is little time for a user to search within a chip during the brief presentation duration of each image. Performance also suffers if chips are at an inappropriate scale with respect to targets of interest. We are currently developing computer vision techniques to raise the salience of features associated with targets. Computer vision techniques can also be used to detect segments of imagery where target detection may be challenging—presentation rates can be manipulated based on estimates of detection difficulty. Other factors that can have an impact on target performance include distractions and lapses of attention. We are currently working on identifying EEG signatures associated with these states. Images processed during

periods of low attention could be marked for later review.

detecting the ErP Signal

Reliable detection of the ERP signal is difficult; background EEG is often an order of magnitude higher in amplitude than the ERP signal. Overcoming this signal-to-noise problem requires signal-processing steps to minimize the impact of noise artifacts and pattern-recognition techniques to reliably identify the signal. Pattern-recognition algorithms estimate the likelihood of a given image being a target given the pattern of neural activity associated with each image. These probability estimates can be used to produce a probability map, which is overlaid in the original broad area image to produce target hotspots. The analyst can confirm or rule out the presence of targets by zooming into these hotspots.

The search technique described has been evaluated in experiments with both profes-

From Clinical Tool to hCI Modality Until recently, the use of EEG sensor technology was largely limited to clinical contexts. However, researchers are now exploring a broad range of neurally based HCI application areas, including input mechanisms, interruptibility estimation, and usability evaluation. A variety of factors—lower sensor costs, practical form factors, robust signal processing, and classification algorithms—have all combined to bring EEG technology out from the obscurity of the clinical laboratory to numerous real-world task contexts. It may not be too long before you find yourself interacting with your computer via neural interface.

 

ABOUT ThE AUThOr Santosh Mathan is a researcher in the Human Centered Systems Group at Honeywell Laboratories in Redmond, Washington. His research focuses on the development of techniques for estimating cognitive state in challenging real-world application contexts. He is currently principal investigator on the DARPA funded Neurotechnology for Intelligence Analysts program. He is also exploring the feasibility of using sensor-based workload estimates for usability evaluation. Santosh has a Ph.D. in human computer interaction from Carnegie Mellon University.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without the fee, provided that copies are not made or distributed for profit or commercial advantage, and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on services or to redistribute to lists, requires prior specific permission and/or a fee. © ACM 1072-5220/08/0700 $5.00

July + August 2008