constraints. STP presents the planner with a running visualization of
matching tasks in the evolving plan
under creation. The planner can readily inspect the potential tasks, accepting or correcting them as needed.
Here, STP has inferred that the Combat Service Support unit and Main
Supply Route A can be combined into
the task Resupply along Main Supply
Route A. If that is correct, the planner
can select the checkbox that then updates the task matrix and schedule.
As the planner adds more symbols to
the map, the system’s interpretations
of matching tasks are likewise updated. Task start and end times can
be spoken or adjusted graphically in
a standard Gantt chart task-synchro-nization matrix. Note STP is not trying to do automatic planning or plan
recognition but rather assist during
the planning process; for instance,
STP can generate a templated “
operations order” from the tasks and
graphics, a required output of the
planning process. Much more planning assistance can, in principle, be
provided, though not clear is what a
planner would prefer.
Because the system is database
driven, the multimodal interface and
system technology have many potential commercial uses, including other
types of operations planning (such as
“wildland” firefighting, as firefighters
say), as well as geographical information management, computer-aided design, and construction management.
Four types of evaluations of STP have
been conducted by the U.S. military:
component evaluations, user juries,
controlled study, and exercise planning tests.
Component evaluations. In rec-
ognition tests of 172 symbols by a
DARPA-selected third-party evalua-
tor during the Deep Green Program
in 2008, the STP sketch-recognition
algorithm had an accuracy of 73% for
recognizing the correct value at the
top of the list of potential symbol-rec-
ognition hypotheses. The next-best
Deep Green sketch recognizer built
for the same symbols and tested at
the same time with the same data had
a 57% recognition accuracy for the
12 Rather than
use sketch alone, most users prefer
to interact multimodally, speaking
labels while drawing a point, line, or
area. STP’s multimodal recognition
in 2008, as reported by an externally
contracted evaluator, was a consider-
ably higher 96%. If STP’s interpreta-
tion is incorrect, users are generally
able to re-enter the multimodal in-
put, select among symbols on a list of
alternative symbol hypotheses, or in-
voke the multimodal help system that
presents the system’s coverage and
can be used for symbol creation.
STP has also been tested using
head-mounted noise-canceling microphones in high-noise Army vehicles. Two users—one male, one
female—issued a combined total of
221 multimodal commands while
riding in each of two types of moving
vehicles in the field, with mean noise
76.2dbA and spikes to 93.3dbA. They
issued the same 220 multimodal commands to STP with the recorded vehicle noise played at maximum volume
in the laboratory, with mean 91.4dbA
and spikes to 104.2dbA. These tests
Figure 4. Implicit task creation.
The map depicts a combat-service-support unit and main supply route, medical unit, and casualty-collection point;
the task inference process finds tasks (such as resupply) that can include various entities along main supply route A.
Figure 5. COA sketch used in controlled study, January 2013.