systems, for instance multi-user
interaction or even gaming, where the
user in front of the camera is the active
user. Additionally, it could add a “smart”
aspect to existing devices, such as
making any smart TV gesture-enabled
(Figure 7).
Thermal biometrics. A body of
recent work recognizes the need for a
more transparent usable authentication
mechanism. To be truly usable,
authentication should be seamless and
require no explicit actions from users.
Biometric authentication is a promising
approach because biometric
characteristics can be used to
simultaneously identify and
authenticate a user. The ability of
thermal cameras to detect subtle
temperature changes enables the
unique vein patterns on the back of our
hands to be captured. We investigated
using veins on the back of the hand for
contactless and seamless authentication
(Figure 8). It is invariant to changes in
how the hand is posed as well as
changes to the environment. Being
accurate and invariant, vein-based
authentication has the potential to
seamlessly authenticate users of
desktop computers and tabletops.
The dark side of thermal imaging.
As the writer David Wong once said,
“New technology is not good or evil in
and of itself. It’s all about how people
choose to use it.” Although thermal
imaging enables seamless
authentication and novel interactive
techniques based on the heat traces, it
raises a threat in the form of thermal
attacks. As shown in Figure 9, the heat
traces could be extracted from
smartphone screens and used to infer
the PIN or pattern used for
authentication. In our exploration [ 3],
we found that the most-used
smartphone authentication techniques,
namely PINs and patterns, could be
uncovered with up to a 100 percent
success rate, even 30 seconds after
authentication.
A window into our mind. A
technology with the capability of sensing
and inferring physiological signals has
the ability to provide a window into our
mind that can be used to adapt system
behavior accordingly. Internal-state
information is largely invisible from the
outside of a user’s brain, and
introspection often fails to reason about
it in an unbiased and objective way.
However, our emotional state influences
our body temperature, in particular our
facial temperature (Figure 10), as the
face contains critical blood vessels
connected to our autonomic nervous
system. Using thermal imaging, these
cues give glimpses into our internal
states, such as our cognitive load.
Thermal cameras overcome the
limitations of the contact sensors used in
related research. It is also more robust
than other contactless approaches, since
the temperature signature is more
resistant to conscious manipulation [ 4, 5].
Thermal imaging technologies.
Thermal imaging has entered the
commercial market and is available in
different forms, such as integrated into
smartphones and as attachments to
phone or USB cameras (Figure 11).
Figure 2. (left) RGB view; (right) equivalent
thermal camera view.
Figure 4. Traces of object interaction. Figure 3. Step traces after the user is gone.
Figure 5. (left) RGB reflection behavior; (right) thermal mirror-like reflection for
the same surface.
Figure 7. Potential use cases for thermal reflection interaction.
Figure 8. Veins from the back of the hand
captured by a thermal camera.
Figure 9. Heat traces for sample PIN and
pattern.
Figure 6. (left) The field of view (FOV) of the camera in gray. (right) The extended interaction
space due to thermal reflection, highlighted in colors.