powering the VI user to participate
more actively in the conversation.
The same technology may also help
a VI person become aware of how he
or she is perceived by others. A survey
conducted with 25 visually impaired
persons and two sighted specialists15
has highlighted some of the function-alities that would be most desirable
in such a system. These include: understanding whether one’s personal
mannerisms may interfere with social
interactions with others; recognizing
the facial expressions of other interlocutors; and knowing the names of
the people nearby.
Assisted Videoscripting. Due to their
overwhelmingly visual content, movies
are usually considered inaccessible to
blind people. In fact, a VI person may
still enjoy a movie from its soundtrack,
especially in the company of friends
or family. In many cases, though, it is
difficult to correctly interpret ongoing activities in the movie (for example, where the action is taking place,
which characters are currently in the
scene and what they are doing) from
the dialogue alone. In addition, many
relevant nonverbal cues (such as the
facial expression of the actors) are lost.
Videodescription (VD) is a technique
meant to increase accessibility of existing movies to VI persons by adding
a narration of key visual elements,
which is presented to the listener during pauses in the dialogue. Although
the VD industry is fast growing, due to
increasing demand, the VD generation
process is still tedious and time consuming. This process, however, could
be facilitated by the use of semiauto-mated visual recognition techniques,
which have been developed in different contexts (such as surveillance and
video database indexing). An early
example is VDManager, 7 a VD editing
software tool, which uses speech recognition as well as key-places and key-faces visual recognition.
Roberto Manduchi was supported
by the National Science Foundation
under Grants IIS-0835645 and CNS-
0709472. James Coughlan was supported by the National Institutes of
Health under Grants 1 R01 EY018345-
01, 1 R01 EY018890-01 and 1 R01
1. ackel, w. a segway to independence. Braille Monitor,
2. bigham, j.P., jayant, C., ji, h., little, g., Miller, a.,
Miller, r.C., Miller, r, tatarowicz, a., white, b., white,
s. and yeh, t. vizwiz: nearly real-time answers to
visual questions. In Proc. ACM Symposium on User
Interface Software and Technology (2010).
3. blasch, b., wiener, w. and welsh, r. Foundations of
Orientation and Mobility. afb Press, second edition,
4. Chen, X. and yuille, a. detecting and reading text
in natural scenes. In Proc. IEEE Conference on
Computer Vision and Pattern Recognition (2004).
5. Coughlan, j. and Manduchi, r. functional
assessment of a camera phone-based wayfinding
system operated by blind and visually impaired
users. International Journal on Artificial
Intelligence Tool 18, 3 (2009), 379−397.
6. dingler, t., lindsay, j. and walker, b.n. learnability
of sound cues for environmental features: auditory
icons, earcons, spearcons, and speec. In Proc.
International Conference on Auditory Display,
7. gagnon, l., Chapdelaine, C., byrns, d., foucher, s.,
heritier, M. and gupta, v. a computer-vision-assisted
system for videodescription scripting. In Proc.
Workshop on Computer Vision Applications for the
Visually Impaired (2010).
8. gallo, o. and Manduchi, r. reading 1-d barcodes
with mobile phones using deformable templates
barcodes with mobile phones using deformable
templates. IEEE Transactions on Pattern Analysis
and Machine Intelligence, in press.
9. hesch, j.a. and roumeliotis, s.I. design and
analysis of a portable indoor localization aid for the
visually impaired. International Journal on Robotics
Research 29, (sept. 2010), 1400−1415.
10. hile, h., liu, a., borriello, g., grzeszczuk, r.,
vedantham, r. and kosecka, j. visual navigation
for mobile devices. IEEE Multimedia 17, 2 (2010),
11. hile, h., vedantham, r., Cuellar, g., liu, a., gelfand,
n., grzeszczuk, r. and borriello, g. landmark-based
pedestrian navigation from collections of geotagged
photos. In Proc. International Conference on Mobile
and Ubiquitous Multimedia (2008).
12. Ivanchenko, v., Coughlan, j. and shen, h.
Crosswatch: a camera phone system for orienting
visually impaired pedestrians at traffic intersections.
In Proc. International Conference on Computers
Helping People with Special Needs (2008).
13. Ivanchenko, v., Coughlan, j. and shen, h. real-time
walk light detection with a mobile phone. In Proc.
International Conference on Computers helping
people with Special Needs (2010).
14. kalia, a.a., legge, g.e., ogale, a. and roy, r.
assessment of indoor route-finding technology
for people who are visually impaired. Journal of
Visual Impairment & Blindness 104, 3 (Mar. 2010),
15. krishna, s., Colbry, d., black, j., balasubramanian,
v. and Panchanathan, s. a systematic requirements
analysis and development of an assistive device
to enhance the social interaction of people who
are blind or visually impaired. In Proc. Workshop
on Computer Vision Applications for the Visually
16. kulyukin, v. and kutiyanawala, a. accessible
shopping systems for blind and visually impaired
individuals: design requirements and the state of the
art. The Open Rehabilitation Journal 2 (2010).
17. kutiyanawala, a. and kulyukin, v. an eyes-free
vision-based uPC and MsI barcode localization
and decoding algorithm for mobile phones. In Proc.
Envision Conference, (2010).
18. ladetto, Q. and Merminod, b. Combining gyroscopes,
magnetic compass and gPs for pedestrian
navigation. In Proc. Int. Symposium on Kinematic
Systems in Geodesy, Geomatics and Navigation
19. liu, j. Phillips, C. and daniilidis, k. video-based
localization without 3d mapping for the visually
impaired. In Proc. Workshop on Computer Vision
Applications for the Visually Impaired (2010).
20. loomis, j.M., golledge, r.g., klatzky, r.l. and
Marston, j.r. assisting wayfinding in visually
impaired travelers. Applied Spatial Cognition:
From Research to Cognitive Technology. lawrence
erlbaum assoc., Mahwah, nj, 2007, pages 179−202.
21. Manduchi, r and kurniawan, s. Mobility-related
accidents experienced by people with visual
impairment. AER Journal: Research and Practice in
Visual Impairment and Blindness, in press.
Roberto Manduchi ( firstname.lastname@example.org) is an
associate professor in the department of Computer
engineering at university of California, santa Cruz.
James Coughlan ( email@example.com) is senior scientist
at the smith-kettlewell eye research Institute, san