Several earlier research efforts
sought to achieve end-user design of
interactive behavior. One involved
making traditional programming
(scripting) accessible to casual users
through a highly visual editing environment. 3 In the system, users write a
program using simple drag-and-drop
operations without making syntax errors. Another involved using programming by demonstration5 for character
animation17; in it, users demonstrate
the desired interactive behavior of a
character, with the system learning
the pattern from the demonstration.
Programming with visual replacement rules is a promising approach
for defining a character’s interactive
behavior. 4 The user specifies before-and-after pairs; at runtime, the system
compares the scene configuration
with the before patterns, replacing it
with after patterns when the match is
Though these experiments produced interesting initial results,
designing the arbitrary interactive
behavior of a virtual agent is often
prohibitively difficult. We are particularly interested in teaching interactive
behavior to physical agents (robots).
End-user programming for robot behavior has been tested in some systems1 but is still limited to basic motions. Programming by demonstration
for robots has also been reported but
is used mainly for acquiring physical-manipulation skills. 2 Techniques developed in the user-interface-research
community that should be applicable
to human–robot interaction represent
an interesting research direction.
Designing real-world objects. The
systems outlined here were all designed for virtual representations; one
can produce interesting graphics on
the computer screen but cannot touch
or use them in the real world. Then
there’s development of end-user tools
for designing physical objects (such as
furniture and clothing). The idea is to
help people custom-design the things
they will use instead of having to buy
manufactured products in stores. Objects designed by users themselves
should satisfy their needs more directly and produce greater satisfaction.
Unlike professional designers, the
typical consumer generally lacks the
professional knowledge needed to de-
sign physical objects. Inexperienced
consumers could easily create a bag
that is not sturdy enough or a chair that
cannot stand up. One promising ap-
proach is to introduce physics into the
modeling process. Traditional model-
ing systems ignore physics, possibly
producing physically inappropriate re-
sults, as in, say, objects that penetrate
one another. It might be possible to
help users avoid these issues by con-
sidering physical principles within a
This article introduced our efforts to
make computer-graphics authoring
accessible to the general public, making it as much a daily communication
tool as word processing and presentation applications. What most defines
our research is its focus on end users.
This opens up new application possibilities for existing technologies while
posing unique technological challenges for interface researchers and
developers. We look forward to more
computer-science researchers participating in this fertile field.
I would like to thank Satoshi Matsuoka, Hidehiko Tanaka, John F. Hughes,
Tomer Moscovich, Yuki Igarashi, Ma-neesh Agrawala, and Masahiko Inami for
their contributions and comments.
1. baum, D. and Zurche, R. Definitive Guide to Lego
Mindstorms. Apress, new york, 2000.
2. billard, A., Calinon, s., Dillmann, R., and schaal, s.
Robot programming by demonstration. In Handbook
of Robotics, b. siciliano and o. Khatib, eds.. springer,
new york, 2008, 1371–1394.
3. Cooper, s., Dann, W., and Pausch, R. Teaching objects
first in introductory computer science. In Proceedings
of the ACM Technical Symposium on Computer
Science Education (Reno, nV, Feb. 19–22). ACM Press,
new york, 2003, 191–195.
4. Cypher, A. and smith, D. C. Kidsim: end-user
programming of simulations. In Proceedings of the
ACM Conference on Computer Human Interaction
(Denver, May 7–11). ACM Press, new york, 1995,
5. Cypher, A. Watch What I Do: Programming by
Demonstration. MIT Press, Cambridge, MA, 1993.
6. Dontcheva, M., yngve, G., and Popović, Z. layered
acting for character animation. In Proceedings of ACM
SIGGRAPH (san Diego, CA, july 27–31). ACM Press,
new york, 2003, 409–416.
7. Fang, A.C. and Pollard, n.s. efficient synthesis of
physically valid human motion. ACM Transactions on
Graphics 22, 3 (july 2003), 417–426.
8. Igarashi, T., Moscovich, T., and hughes, j.F. As-rigid-as-possible shape manipulation. ACM Transactions on
Graphics 24, 3 (july 2005), 1134–1141.
9. Igarashi, T., Moscovich, T., and hughes, j.F. spatial
keyframing for performance-driven animation, In
Proceedings of ACM SIGGRAPH/Eurographics
Symposium on Computer Animation (los Angeles,
july 29–31). ACM Press, new york, 2005, 107–115.
10. Igarashi, T. and hughes, j. F. Clothing manipulation. In
Proceedings of the ACM Symposium on User Interface
Software and Technology (Paris, oct. 27–30). ACM
Press, new york, 2002, 91–100.
11. Igarashi, T., Matsuoka, s., and Tanaka, h. Teddy:
A sketching interface for 3D freeform design. In
Proceedings of ACM SIGGRAPH (los Angeles, Aug.
8–13). ACM Press, new york, 1999, 409–416.
12. james, D.l. and Pai, D.K. ArtDefo: Accurate real-time deformable objects. In Proceedings of ACM
SIGGRAPH (los Angeles, Aug. 8–13). ACM Press,
new york, 1999, 65–72.
13. lewis, j. P., Cordner, M., and Fong, n. Pose space
deformations: A unified approach to shape
interpolation and skeleton-driven deformation. In
Proceedings of ACM SIGGRAPH (new orleans, july
23–28). ACM Press, new york, 2000, 165–172.
14. MacCracken, R. and joy, K.I. Freeform deformations
with lattices of arbitrary topology. In Proceedings
of ACM SIGGRAPH (new orleans, Aug. 4–9). ACM
Press, new york, 1996, 181–188.
15. Mori, y. and Igarashi, T. Plushie: An interactive design
system for plush toys. ACM Transactions on Graphics
26, 3 (july 2007).
16. Rekimoto, j. smartskin: An infrastructure for freehand
manipulations on interactive surfaces. In Proceedings
of ACM Conference on Human Computer Interaction
(Minneapolis, Apr. 20–25). ACM Press, new york,
17. young, j.e., Igarashi, T., and sharlin, e. Puppet
Master: Designing reactive character behavior by
demonstration. In Proceedings of ACM SIGGRAPH
Symposium on Computer Animation (Dublin, july
7–9). ACM Press, new york, 2008, 183–191.
18. Zeleznik, R. C., herndon, K.P., and hughes, j. F.
sKe TCh: An interface for sketching 3D scenes. In
Proceedings of ACM SIGGRAPH (new orleans, Aug.
4–9). ACM Press, new york, 1996, 163–170.
Takeo Igarashi ( firstname.lastname@example.org) is an associate
professor in the Department of Computer science in the
Graduate school of Information science and Technology
at The university of Tokyo.