Impact of ambient light. We have also evaluated
LiShield’s performance under different types of ambient
lights. We found the stripes are almost completely removed
under direct sunlight due to its extremely high intensity.
Flash light can increase the quality slightly thanks to its
close distance to the scene, but the improvement is marginal and far from unprotected. In addition, we only found
a marginal decrease of barcode detection rate in every case.
Thus, we conclude that LiShield is robust against most ambient lights.
7. RELATED WORK
Camera recording of copyright screen-displayed videos (e.g.,
in a movie theater) accounts for 90% of pirated online content. 21 Since screen refresh rate is much higher than video
frame rate, Kaleido21 scrambles multiple frames within the
frame periods to deter recording, while preserving viewing
experience by taking advantage of human eyes’ flicker fusion
effects. Many patented technologies addressed the same
issue. In contrast, the problem of automatic protection of
private and passive physical space received little attention.
Certain countries dictate that smartphone cameras must
make shutter sound to disclose the photo capturing actions,
yet this does not enforce the compliance, cannot block
the photo distribution, and cannot automatically protect
against video recording.
Certain optical signaling systems can remotely ban
photography in concerts, theaters, and other capturing-sensitive sites. For example, BlindSpot17 adopts a computer
vision approach to locate retro-reflective camera lenses
and pulses a strong light beam toward the camera to cause
overexposure. Such approaches fail when multiple cameras
coexist with arbitrary orientations.
Conventional visual privacy-protection systems have
been replying on postcapture processing. Early efforts
employed techniques like region-of-interest masking, blurring, mosaicking, etc., 11 or re-encoding using encrypted
scrambling seeds. 3 There also exists a vast body of work
for hiding copyright marks and other information in digital images/videos (e.g., 10). LiShield’s barcode protection is
inspired by these schemes, but it aims to protect physical
scenes before capturing.
8. CONCLUSION
Privacy protection in passive indoor environment has been
an important but unsolved problem. In this paper, we propose LiShield, which uses smart-LEDs and specialized
intensity waveforms to disrupt unauthorized cameras,
while allowing authorized users to record high quality
image and video. We implemented and evaluated LiShield
under various representative indoor scenarios, which
demonstrates its effectiveness and robustness. We consider LiShield as a first exploration of automatic visual
privacy enforcement and expect that it can inspire more
research along the same direction.
Acknowledgment
This project was partially supported by the NSF under Grant
CNS-1506657, CNS-1518728, and CNS-1617321.
90% for monochrome barcodes if attacked. We conclude
that LiShield’s barcode detector provides reliable detection,
whereas RGB barcodes are more detectable and robust than
monochrome ones, thanks to extra redundancy provided by
color channels.
6. 4. Robustness against attacks
Manual exposure attack. One possible attack against
LiShield is to manually set the exposure time te to smooth
out the flickering patterns (Section 2. 2). Our experiment
shows that although the image quality first increases with te,
it drops sharply as overexposure occurs. Therefore, LiShield
traps the attacker in either extremes by optimizing the waveform (Section 2. 1) and thwarts any attempts through exposure
time configuration.
We also tested the effectiveness of randomization with
auto-exposure (except for attacker). We set f1 = fB = 200,
300, . . ., 600 Hz, ∆f = 50Hz, and M = 2, 3, …, 6 to scramble
multiple frequencies. We found that the image degradation with scrambling is comparable with single frequency setup, thus frequency randomization does not harm
LiShield’s protection.
Multiframe attack. Figure 11 plots the recovered scene’s
quality under the multiframe attack. Here, we set te to be
1/500 s to avoid overexposure and then record a video in
30 fps. CW-SSIM remains low at 0.5 using 1000 frames,
which means the impact of stripes on structure of scene is still
strong, making quality still unacceptable for professionals who
spend such a great cost. We also ask five volunteers to hold
the smartphone as stable as they can on a table, and Figure 11
shows the quality is even lower, because it is impossible
to completely avoid dithering with hands. Extending the
recording duration increases disturbance and probability of
being identified by the protected user, making it impractical
for the attack to occur.
Image recovery processing attack. We evaluate the
image quality after postprocessing with denoising or
debanding (Section 5). The denoising methods fail to
improve the quality significantly (CW-SSIM ≈ 0.3–0.4)
as the disruption pattern of LiShield does not fit most
known Gaussian noise model. The deformation removal
methods (i.e., debanding and unstriping) do not help too
much (CW-SSIM ≈ 0.4–0.5), since interpolation process
cannot bring back the exact pixel values. The CIEDE2000
color metric also shows a low quality (around 35). Thus,
it is hard to fully remove LiShield’s impact by simple image
restoration. More advanced computer vision techniques
may provide better recovery, but even they will not recover
the exact original scene since information is already lost at
capture time.
0
0.2
0.4
0.6
0.8
1
10 102 103
C
WSS
I
M
Number of Frames
Tripod
Hand Holding
Figure 11. Image quality by multiframe ensemble.