INTERACTIONS.ACM.ORG 54 INTERACTIONS JULY–AUGUST2014
The boundaries between ‘the digital’ and our everyday physical world are dissolving as we develop more physical ways of
interacting with computing. This forum presents some of the topics discussed in the colorful multidisciplinary field of
tangible and embodied interaction. — Eva Hornecker, Editor
FORUM LET'S GET PHYSICAL
around the world still use paper flight
strips. Printed on each strip is the
planned route of an aircraft through
a controller’s sector of responsibility.
Controllers annotate, grasp, move,
and organize their paper strips on
a stripboard, using these tangible
interactions to organize their mental
picture [ 2].
Unfortunately, paper strips are a
poor connection between the physical
and digital worlds: Once printed
they cannot be updated, nor can the
instructions given to pilots be used as
input to the system. This impedes the
development of more automated ATC
systems in which controllers would be
provided with useful alerts and hints
computed from their instructions.
However, universal solutions for
replacing paper with fully digital
systems have yet to be found. The
proposal to use graphical interfaces
on the sole radar display met with
mixed results. More sophisticated
variants of “electronic stripping”
were then proposed, culminating
Air traffic control (ATC) guides aircraft o maintain safe distances and to ptimize traffic fluidity. ATC has been a life-size
HCI laboratory for many decades,
as witnessed by the introduction of
touchscreens in the 1960s, digital
emulations of paper in the 1990s [ 1],
and research on augmented reality
[ 2]. This history reveals a succession
of compromises between physical and
digital interactions, of which tangible
user interfaces might be the synthesis.
What makes ATC special is the
combination of efficiency and safety that
shapes its evolution. While efficiency
is the driving force for technical
innovation, safety is a selection force:
Only those innovations for which air
traffic controllers can safely adapt their
working methods are put into operation.
This co-evolution of technologies and
working methods is a strong asset of
ATC. However, it also makes life hard
for engineers because of the difficulty in
establishing exactly how a technology
will contribute to safety before
proposing it as an alternative to current
technologies.
An example of this challenging
evolution process is the switch from
paper to digital media, fostered by the
civil aviation agencies since the mid-
1980s, but still not achieved. Almost
everywhere, controllers monitor traffic
through real-time radar displays and
communicate with pilots by radio.
In addition, to anticipate the traffic,
manage their decisions, and program
their actions, many controllers
with combinations of rich graphics,
animation, touch, and gestures [ 1].
These have encountered some success,
but in many cases, particularly in
airports, there is still a need for the
better capturing properties of paper
strips. Combining the benefits of paper
with those of digital through tangible
interaction is a promising direction for
meeting these demands.
THE STRIP’TIC PROTOTYPE
We have designed Strip’ TIC (Stripping
Tangible Interface for Controllers), a
prototype that combines augmented
paper and digital pens on a multitouch
glass stripboard, using vision-based
tracking and augmented rear and front
projection [ 3]. The paper strips, the
stripboard, and the radar screen are
all covered with Anoto Digital Pen
patterns (DP-patterns), small patterns
used by the pen’s infrared digital camera
to compute the location of the pen on
the dotted surface. The controllers’
actions with the digital pen are sent
in real time to the software. They can
annotate paper strips, and the resulting
strokes are both physical (inked) and
digital (projected). They can use the pen
to point at symbols on the radar display
or names on the paper strips. The
stripboard itself is semi-opaque. This
enables both a bottom projection on the
stripboard and strip tracking, thanks to
AR patterns printed on the back of the
strips. A second projector located above
the device displays graphics on top of
the stripboard and the strips.
Designed in close collaboration with
controllers, Strip’TIC allows them
to manage traffic and workload with
their current rules and practices while
Jean-Luc Vinot, Université de Toulouse - ENAC, Catherine Letondal, Université de Toulouse - ENAC,
Rémi Lesbordes, DSNA/DTI/R&D, Stéphane Chatty, Université de Toulouse - ENAC,
Stéphane Conversy, Université de Toulouse - ENAC, Christophe Hurter, Université de Toulouse - ENAC
Tangible Augmented Reality
for Air Traffic Control
Insights
→ Air traffic control procedures, skills,
and systems have co-evolved over
decades in a design process involving
controllers, programmers, and
paper/electronic technologies.
→ Tangible interaction combined with
augmented reality is well suited for
supporting ATC. Developing these
systems is challenging yet feasible.
→ Tangible augmented interactors
should be conceived of as continuous
physical/virtual artifacts.