figure 14: object removal: find the missing shoe (original image is top
left). in this example, in addition to removing the object (one shoe), the
image was enlarged back to its original size. note that this example
would be difficult to accomplish using in-painting or texture synthesis.
figure 15: an example where seam carving fails because the image is
too condensed. in such cases the best strategy might be to use scaling.
6. conclusions anD futuRE WoRK
We presented an operator for content-aware resizing of
images and video using seam carving. Seams are computed
as the optimal paths on an image or video and are either
removed or inserted. This operator can be used for a variety
of image and video manipulations including aspect ratio
change, image and video retargeting and object removal.
The operator can be easily integrated with various saliency
measures, as well as user input, to guide the resizing process. In addition, we define a novel media representation
called multisize images and video that support continuous
resizing ability in real time.
There are numerous possible extensions to this work. One
would be to investigate better importance function that produces visually more pleasing results. Another direction is to
rely on the seam map created by our method as a saliency map
that can guide other processes such as content-aware compression. Finally, one could think of ways to unify the seam carving
operator with other types such as scaling and cropping.
We thank Micheal Rubinstein for his work on video retargeting, Eric Chan for the use of the waterfall image and numerous flickr (
http://www.flickr.com/) members for making their
images available through creative common rights (http://
creativecommons.org/): crazyegg95 (Buddha), Gusthy (Couple
and Surfers), mykaul (Car), 02ma (Vase).
1. agarwala, a., Dontcheva, m., agrawala,
m., Drucker, s., colburn, a., curless, b.,
salesin, D., and cohen, m. interactive
digital photomontage. ACM Trans.
Graph. 23, 3 (2004), 294–302.
2. avidan, s. and shamir, a. seam carving
for content-aware image resizing. ACM
Trans. Graph. 26, 3 (2007), 10.
3. boykov, y. and Jolly, m.-P. interactive
graph cuts for optimal boundary &
region segmentation of objects in n-d
images. in International Conference
on Computer Vision (ICCV), volume i,
4. chen, l., xie, x., fan, x., ma, w.,
zhang, h., and zhou, h. a visual
attention model for adapting images
on small displays. Multimedia
Systems 9, 4 (2003), 353–364.
5. Davis, J. mosaics of scenes with moving
objects. in Proceedings of CVPR, 1998.
6. efros, a.a. and freeman, w.t. image
quilting for texture synthesis and
transfer. in SIGGRAPH 2001,
Computer Graphics Proceedings.
e. fiume, ed. acm/acm siggraPh,
7. fan, x., xie, x., zhou, h.-Q., and ma,
w.-y. looking into video frames n
small displays. in MULTIMEDIA
‘03: Proceedings of the 11th ACM
International Conference on
Multimedia. acm, 2003, 247–250.
8. gal, r., sorkine, o., and cohen-or, D. feature-aware texturing.
in Eurographics Symposium on
9. itti, l., Koch, c., and neibur, e.
a model of saliency-based visual
attention for rapid scene analysis.
PAMI 20, 11 (1999), 1254–1259.
10. Jia, J., sun, J., tang, c.-K., and shum,
h.-y. Drag-and-drop pasting. in
Proceedings of SIGGRAPH, 2006.
11. Kwatra, V., schödl, a., essa, i.,
turk, g., and bobick, a. graphcut
textures: image and video synthesis
using graph cuts. ACM Trans. Graph.
22, 3 (2003), 277–286.
12. liu, f. and gleicher, m. automatic image
retargeting with fisheye-view warping.
in ACM UIST, 2005, 153–162.
13. liu, f. and gleicher, m. Video
retargeting: automating pan and scan.
in MULTIMEDIA’06: Proceedings of
the 14th annual ACM International
Conference on Multimedia. acm,
14. liu, h., xie, x., ma, w., and zhang, h.
automatic browsing of large pictures
on mobile devices. Proceedings of the
11th ACM International Conference
on Multimedia, 2003, 148–155.
15. roditty, l. and zwick, u. on dynamic
shortest paths problems. in
Proceedings of the 12th Annual
European Symposium on Algorithms
(ESA), 2004, 580–591.
16. rother, c., bordeaux, l., hamadi, y., and
blake, a. autocollage. in Proceedings
of SIGGRAPH 2006, 2006.
17. rubinsteing, m., shamir, a., and
avidan, s. improved seam carving for
video retargeting. ACM Trans. Graph.
27, 3 (2008), 10.
18. santella, a., agrawala, m., Decarlo, D.,
salesin, D., and cohen, m. gaze-based
interaction for semi-automatic photo
cropping. in ACM Human Factors
in Computing Systems (CHI), 2006,
19. schödl, a., szeliski, r., salesin, D. h.,
and essa, i. Video textures. in
SIGGRAPH’00: Proceedings of the
27th Annual Conference on Computer
Graphics and Interactive Techniques.
acm/addison-wesley, 2000, 489–498.
20. setlur, V., takagi, s., raskar, r.,
gleicher, m., and gooch, b. automatic
image retargeting. in The Mobile and
Ubiquitous Multimedia (MUM). acm,
21. simakov, D., caspi, y., shechtman,
e., and irani, m. summarizing visual
data using bidirectional similarity.
in Proceedings of CVPR, 2008.
22. suh, b., ling, h., bederson, b.b., and
Jacobs, D.w. automatic thumbnail
cropping and its effectiveness. in
UIST’03: Proceedings of the 16th
Annual ACM Symposium on User
Interface Software and Technology.
acm, new york, ny, 95–104.
23. Viola, P. and Jones, m. rapid object
detection using a boosted cascade
of simple features. in Conference
on Computer Vision and Pattern
Recognition (CVPR), 2001.
24. wang, J., bhat, P., colburn, r. a.,
agrawala, m., and cohen, m. f.
interactive video cutout. ACM Trans.
Graph. 24, 3 (2005), 585–594.
25. wang, J., reinders, m., lagendijk, r.,
lindenberg, J., and Kankanhalli, m.
Video content presentation on tiny
devices. in IEEE International
Conference on Multimedia and Expo
(ICME), volume 3, 2004, 1711–1714.
26. wang, J., xu, y., shum, h.-y., and
cohen, m. f. Video tooning. ACM
Trans. Graph. 23, 3 (2004), 574–583.
27. wang, y.-s., tai, c.-l., sorkine, o.,
and lee, t.-y.. optimized scale-and-stretch for image resizing. ACM
Trans. Graph. Proceedings of ACM
SIGGRAPH ASIA 27, 5 (2008).
28. wei, l.-y., han, J., zhou, K., bao, h.,
guo, b., shum, h.-y.. inverse texture
synthesis. ACM Trans. Graph. 27, 3
29. wolf, l., guttmann, m., and
cohen-or, D. non-homogeneous
in Proceedings of the 11th IEEE
International Conference on Computer
Vision (ICCV’07), 2007, 1–6.
Ariel Shamir ( firstname.lastname@example.org), efi
arazi school of computer science, the
interdisciplinary center, herzliya, israel.
© 2009 acm 0001-0782/09/0001 $5.00
Shai Avidan ( email@example.com), adobe
systems, inc., newton, ma.