of sensors and new, more flexible displays opens up all kinds of opportunities for innovation. “You shouldn’t just
look at [displays],” he says, “but interact with them physically: throw them,
kick them, spin them.”
In a sensor-equipped world, almost
anything can become an interface.
At Microsoft, researcher Paul Dietz is
working on a new sensor technology
called SurfaceWare that allows liquid
containers to detect their contents and
send automatic signals in response to
changing conditions. In the technology’s simplest application, a near-empty
glass could automatically signal for a
refill. Beyond realizing efficiency gains
for harried bartenders, the technology
holds out all kinds of possibilities for
“smart” liquid containers.
While working on a predecessor
project called iGlassware, Dietz used
passive RFID tags married to capacitance sensors to determine the contents
of a container, but that solution didn’t
work well with thick liquids that tend
to coat a container’s interior. So Dietz
developed an optical prism embedded
in the glass container that reflects light
when in the air but not when covered
with fluid, enabling the container to
detect a wider range of fluids. Building
on this work, Dietz is developing a new
class of surface interaction widgets
called dynamic tags. “Basically, these
are transducers that can sense physical quantities in the environment and
present the measurement in an optical
form that can be read,” says Dietz.
Before coming to Microsoft, Dietz
worked on a series of sensor-based
technologies for Mitsubishi and Disney, where he created a set of location-aware water installations, including
a fountain that withdraws its stream
when an observer tries to touch it, a
musical harp with strings made of water, and a liquid touch screen known as
the TouchPond.
foldable Interfaces
While the market for sensor-based pillows, drinking glasses and water fountains remains to be proven, other researchers are working on sensor-based
interfaces with more practical applications. Before his brush with YouTube fame, Lee developed a foldable
interface designed to mimic one of the
world’s most familiar analog interfac-
Johnny Chung Lee’s
inspiration for
the foldable
display came from
Renaissance, an
animated cyberpunk/
science-fiction
detective film.
es: the newspaper. Using an innovative
approach to tracking objects with projected light, the display dynamically
updates its contents based on its location and orientation. Like the Wiimote
demo, this prototype relies on infrared
sensors to determine the object’s position. After experimenting with a number of alternative sensor technologies,
Lee settled on infrared sensors like the
ones found in the Wiimote.
An alternative approach might have
involved using a camera to track the
display, by giving the display unit unusual optical characteristics like an orange light flashing at regular intervals.
But Lee believes that camera-based
displays are inherently limiting. He
prefers infrared sensors because they
are cheaper and more reliable.
“One of the problems of camera
tracking is that if there is more than one
point, the camera can’t tell them apart.
But a light sensor can tell them apart.
With cameras, the lights are broadcasting outward, but with a projector, the
light ‘knows’ where it came from,” Lee
explains. “There’s relatively little infrared interference in the world. Very little
else is blinking at 50 kHz.”
The foldable display looks like
something right out of Minority Report,
the 2002 Steven Spielberg film that
has served as a touchpoint for many
interface designers. However, Lee says
his inspiration actually came from
Renaissance, an animated cyberpunk/
science-fiction detective film by Christian Volckman. Lee admits to finding
inspiration throughout the pop culture world, often basing his projects
on ideas found in Japanese anime and
other films. “Usually I already have a list
of things I’d like to do,” Lee says, “but
films sometimes give me the inspiration to actually do it.” In a world where
most people have become accustomed
to the traditional display-keyboard-mouse interface, looking to films and
other reference points outside the
world of computer science may help
developers find fertile new ground for
thinking about alternative interfaces.
Some developers find inspiration
in science-fiction and fantasy films.
For others, the inspiration comes from
hard-won experience. Dietz traces his
original inspiration back thirty years to
his childhood, when he and his brothers hacked a Heathkit H- 8 computer kit
to control a toy train set by using LEDs
as light receivers and small magnetic
reed switches wired into the computer.
As Dietz recalls, “The train would automatically start up and go from station
to station, switching the switches, displaying a live map, and even playing appropriate background music. Not bad
for 1978!”
Whatever the inspiration, researchers are gradually recognizing the potential of position sensors to help
them overcome the limitations of traditional user interfaces. “We are leading increasingly sedentary lives, locked
with our eyes onto the very small world
behind the screen of a traditional display,” says Mertens, “and that is still
how many computer games force you
to behave.”
Mertens believes the future of interaction design involves bringing the
computer out from the other side of
the glass, and building bridges into the
physical realm. “Instead of playing behind the screen in a virtual computer
world, you have to take the display into
the real world,” he says.
When sensors start to do more
than just transmit sensory data to a
traditional two-dimensional computer screen, the way we interact with
computers will fundamentally shift,
as physical objects become “smarter”
about themselves and the world around
them. When that starts to happen—
when computers start taking shape in
three dimensions—sensors may just
start making sense.
Alex Wright is a writer and information architect who
lives and works in new york city.
feBRuaRY 2009 | vol. 52 | No. 2 | CommunICatIons of the aCm
15