Technology | doi:10.1145/1409360.1409366
Ted Selker
touching the future
In combination with finger and hand gestures, multitouch input is enabling users
to manipulate and display information in innovative and unprecedented ways.

, feels, and

The iPhone looks acts like no other phone, thanks to its device-sized screen and multitouch responsiveness to one and

two-fingered gestures. Sweep a finger to the right to unlock its touch screen. Another finger sweep scrolls through all of its features, from applications to photos. Spread two fingers apart or draw them together on a list, photo, or newspaper article to zoom in or out. Press on the screen for a few seconds and the application icons start shaking as though they have become unglued, allowing a user to rearrange them. The judicious combination of simple finger gestures and screen animations, representing physical metaphors, make the iPhone’s interface feel like a physical object that one can intuitively manipulate.

Jeff han, right, employs a simple, two-fingered gesture to manipulate an image on one of Perceptive Pixel’s wall-size multitouch monitors.

With the iPhone, Apple successfully brought together decades of research and dreams. However, one could imagine the iPhone, upon its introduction, faring poorly relative to the Nokia N95 smartphone. The iPhone features an inferior camera, a slower processor, a worse keyboard, no Flash player, and not even a way to attach a lanyard. Yet, the iPhone has been a huge hit. It boasts simple integrated function that introduced millions of consumers to a gestural, multitouch interface. Unlike previous technology deployments, the iPhone did not ask consumers what it can be useful for, but presents a suite of scenarios that a person typically uses several times a day, and its designers carefully matched use scenarios to user actions. So, while one must wrestle with different ways to use the myriad of functions on the Nokia N95, the iPhone user enlarges a roadmap by simply spreading two fingers apart.

Buoyed by the iPhone’s success, companies are competing to show compelling multitouch scenarios that enable users to input, process, and dis-

play information in innovative ways, town Manhattan, which the user pans
often involving finger and hand ges- and tilts; the user can then transform
tures. Jeff Han’s company, Perceptive the Google Earth image into a comput-
Pixel, has created a seven-and-a-half erized scale model of buildings and
foot diagonal multitouch monitor, otherstreet-levelfeatures.
most prominently seen on CNN and Microsoft Surface uses five cam-
known as its “Magic Wall.” (Percep- eras and projector mounted beneath a
tive Pixel has also sold its multitouch table display surface, aided by a Win-
monitors to a number of unspecified dows Vista PC, to produce up to 52
U.S. government agencies.) Han’s wall- points of touch on its 30-inch diagonal
size monitors use a variation of the tabletopscreen. As wellasrecognizing
typical camera and projector mounted gestural input, Surface recognizes ob-
behind the display surface, with the jects with RFID tags placed on them,
camera mapping “frustrated” light in allowing a user to transfer music files
the projection surface to detect touch between an MP3 player and a smart-
inputs. phone by placing each device on the
In Perceptive Pixel’s videos of its computer’s tabletop surface. Surface
wall-size multitouch monitors, Han uses the presence of the object and the
and others use finger and hand ges- RFID to identify the type of device and
tures to type on a virtual keyboard, call its capabilities. The music files appear
up a document, and scroll through its on the tabletop surface near each de-
content; change the composition of a vice, enabling a user to drag material
human face; and manipulate 3D ob- from one device to the other.
jects by turning them any way a user The UnMouse Pad is a multitouch
desires. In one sequence, a user pulls pad, similar to a mouse pad, created
up a distant image from Google Earth by Ken Perlin, a computer scientist at
and zooms in on the image until it’s re- New York University, and several col-
vealed to be a street-level view of mid- leagues. One can use any object, such

PHO TOGRAPH B Y SACHI GAHAN