ally unrealistic in look and behavior.
Moreover, user gear can be expensive,
aesthetically unpleasing, and uncomfortable to use, especially for a prolonged period of time.
There are a myriad of technical approaches to each of these problems,
but they tend to solve one problem by
creating others. A solution that is adequate for a living room game may be a
complete failure in an operating room
or construction site.
Martin Banks, a professor of optometry and vision science at the University of California, Berkeley, and direc-
THIS IS OUR most desperate hour,” said the flickering blue image. “Help me, Obi- Wan Kenobi. You’re my only hope.” In 1977, in a classic
moment in cinematic history, the Star
Wars movie epic gave the public a preview of augmented reality (AR).
But it was only a preview. The three-dimensional (3D) hologram of Princess
Leia standing on a real table had been
simulated by special effects artists at
Lucasfilm, but it was easy to imagine
a day when consumers could bring actual 3D virtual images seamlessly into
their physical environments and interact with them in real time. That is
now possible in a variety of consumer,
medical, and industrial applications,
generally through the use of head-mounted displays (HMDs).
AR, sometimes loosely called
“mixed reality,” combines virtual reality (VR) with the physical world. A recent
application, offered by Schell Games,
uses technology from Disney and Leno-vo to bring Darth Vader into your living
room—your actual living room. In the
game Jedi Challenges, users with a
smartphone-enabled headset, a lightsaber controller, and a tracking beacon
can engage the life-sized movie villain
in a lightsaber battle.
The AR game has received enthusiastic reviews for its realism, yet no
one would for an instant think the
fallen Jedi Knight is actually standing
between the sofa and the coffee table.
Darth Vader has the same unreal blue
hue as the Princess Leia avatar; he is
semi-transparent, pixelated, subject to
ghosting, and lacking in detail.
Yet the market for AR headsets is
potentially huge, and the optical chal-
lenges in AR are the subject of intense
research by companies and universi-
ties around the world. MarketWatch,
published by Dow Jones, forecasts the
AR market will grow at a compound
annual rate of 75% to reach $50 billion
by 2024, just five years from now. Most
analysts look for the greatest growth in
retail, automotive, and medical appli-
cations, although some predict eventu-
ally consumer AR devices will replace
all conventional displays on laptops
The optical challenges when combining VR and AR are complex, and
they have yielded ground only grudgingly. Depending on the application
and the technology for it, images seen
by users are often primitive: blurred,
low in resolution, slow to refresh, subject to a narrow field of view, or gener-
Formidable optical challenges are yielding
to intensive research, development.
Technology | DOI: 10.1145/3344293 Gary Anthes
Battling a virtual Kylo Ren in the augmented reality game Star Wars: Jedi Challenges.