laser scanner. The scan served as input to a 3D landscape visualization, which was then projected back
onto the clay surface. Illuminating Clay illustrates
the blurring between input and output device: Users
could, for example, alter the flow of a river by molding valleys into the clay. Here, function is also triggered by form, which literally follows the flow of
Gummi: Flexing Plastic Computers. Another early
envisionment of an OUI was Gummi [ 8]—a pressure-sensitive PDA that simulated a flexible
credit card displaying an interactive
subway map. Bending the display
would cause this subway map to zoom
in or out, while a touch panel on the
back would allow users to scroll. Again,
function equals form: the shape of the
display affords zooming, and the interplay between haptics and visuals reinforces this functionality. (For more
information, see Carsten Schwesig’s
article in this section).
Paper Computers. Books and
paper also form powerful sources of
inspiration for flexible computer
design. Paper is particularly versatile as
a medium of information. According
to Sellen and Harper [ 9], users still
prefer paper over current computer
displays because it makes navigation
more flexible. Paper input is direct,
two-handed, and provides a rich synergetic set of haptic and visual cues.
Paper supports easier transitions
between activities by allowing users to
pick up and organize multiple documents two-handedly. Paper is also
extremely malleable: it can be
folded—a primary source of input
when constructing models—or bent,
most often applied in navigation.
Paper can be randomly arranged, or in stacks, and can
even contain other objects. With the development of
flexible E-Ink displays we can imagine that in the
future our computers will be indistinguishable from a
sheet of paper. One of the questions we have been trying to answer is how will we interact with such flexible computers?
We experimented with the use of Foldable Input
Devices (FIDs) for GUI manipulations by tracking
the shape of several cardboard sheets that featured
retroreflective markings (see Figure 4). Behaving like
real paper documents, 3D graphics windows follow
the shape of associated FIDs. When FIDs are stacked,
(b) iPod form
factor on an
so are the window sheets in the GUI (Figure 4a).
Stacks of window sheets are sorted with a shake and
browsed by leafing action (Figure 4b). Using a special
FID, window sheets can even be folded into threre-dimensional models, further blurring the distinction
between a window sheet and its content (Figure 4c).
Inspired by Wellner’s DigitalDesk [ 11], Paperwindows [ 2] was the first computer made entirely
out of three-dimensional sheets of paper (see Figure
5). It simulated a flexible, high-resolution, full-color,
and wireless E-Ink display of the future.
Paperwindows are regular sheets of paper, augmented with eight retroreflective markers. These
markers allow a Vicon to capture the motion and
interactive shape of the paper, which is modeled as a
non-uniform rational B-spline (NURBS) surface textured with the real-time content of an application
window. When projected back onto the paper, the
3D models correct any projection skew caused by
paper folds, giving the illusion the paper is, in fact, an
interactive print. We experimented with a Web
browser of which most functions were accessible
through changing the shape of the paper—the primary display of the computer. Bending the sheet
around its horizontal axis would cause the Web
browser to page down or up. Bending the document
back around its vertical axis would cause the Web
browser to go back or forward in its browsing history.
Fingers were also tracked: a link was clicked by touching it. A paper window was activated by picking it up.
Information could be copied from one document to
the next by rubbing two windows onto each other.
Documents could be enlarged through collation, and
sorted by stacking. Such physical interaction techniques remove any distinction between input and
output: in paper windows the shape, location, and