Communications of the ACM

Impact of ambient light. We have also evaluated LiShield’s performance under different types of ambient lights. We found the stripes are almost completely removed under direct sunlight due to its extremely high intensity. Flash light can increase the quality slightly thanks to its close distance to the scene, but the improvement is marginal and far from unprotected. In addition, we only found a marginal decrease of barcode detection rate in every case. Thus, we conclude that LiShield is robust against most ambient lights.

7. RELATED WORK

Camera recording of copyright screen-displayed videos (e.g., in a movie theater) accounts for 90% of pirated online content. 21 Since screen refresh rate is much higher than video frame rate, Kaleido21 scrambles multiple frames within the frame periods to deter recording, while preserving viewing experience by taking advantage of human eyes’ flicker fusion effects. Many patented technologies addressed the same issue. In contrast, the problem of automatic protection of private and passive physical space received little attention. Certain countries dictate that smartphone cameras must make shutter sound to disclose the photo capturing actions, yet this does not enforce the compliance, cannot block the photo distribution, and cannot automatically protect against video recording.

Certain optical signaling systems can remotely ban photography in concerts, theaters, and other capturing-sensitive sites. For example, BlindSpot17 adopts a computer vision approach to locate retro-reflective camera lenses and pulses a strong light beam toward the camera to cause overexposure. Such approaches fail when multiple cameras coexist with arbitrary orientations.

Conventional visual privacy-protection systems have been replying on postcapture processing. Early efforts employed techniques like region-of-interest masking, blurring, mosaicking, etc., 11 or re-encoding using encrypted scrambling seeds. 3 There also exists a vast body of work for hiding copyright marks and other information in digital images/videos (e.g., 10). LiShield’s barcode protection is inspired by these schemes, but it aims to protect physical scenes before capturing.

8. CONCLUSION

Privacy protection in passive indoor environment has been an important but unsolved problem. In this paper, we propose LiShield, which uses smart-LEDs and specialized intensity waveforms to disrupt unauthorized cameras, while allowing authorized users to record high quality image and video. We implemented and evaluated LiShield under various representative indoor scenarios, which demonstrates its effectiveness and robustness. We consider LiShield as a first exploration of automatic visual privacy enforcement and expect that it can inspire more research along the same direction.

Acknowledgment

This project was partially supported by the NSF under Grant CNS-1506657, CNS-1518728, and CNS-1617321.

90% for monochrome barcodes if attacked. We conclude that LiShield’s barcode detector provides reliable detection, whereas RGB barcodes are more detectable and robust than monochrome ones, thanks to extra redundancy provided by color channels.

6. 4. Robustness against attacks

Manual exposure attack. One possible attack against LiShield is to manually set the exposure time te to smooth out the flickering patterns (Section 2. 2). Our experiment shows that although the image quality first increases with te, it drops sharply as overexposure occurs. Therefore, LiShield traps the attacker in either extremes by optimizing the waveform (Section 2. 1) and thwarts any attempts through exposure time configuration.

We also tested the effectiveness of randomization with auto-exposure (except for attacker). We set f1 = fB = 200, 300, . . ., 600 Hz, ∆f = 50Hz, and M = 2, 3, …, 6 to scramble multiple frequencies. We found that the image degradation with scrambling is comparable with single frequency setup, thus frequency randomization does not harm LiShield’s protection.

Multiframe attack. Figure 11 plots the recovered scene’s quality under the multiframe attack. Here, we set te to be 1/500 s to avoid overexposure and then record a video in 30 fps. CW-SSIM remains low at 0.5 using 1000 frames, which means the impact of stripes on structure of scene is still strong, making quality still unacceptable for professionals who spend such a great cost. We also ask five volunteers to hold the smartphone as stable as they can on a table, and Figure 11 shows the quality is even lower, because it is impossible to completely avoid dithering with hands. Extending the recording duration increases disturbance and probability of being identified by the protected user, making it impractical for the attack to occur.

Image recovery processing attack. We evaluate the image quality after postprocessing with denoising or debanding (Section 5). The denoising methods fail to improve the quality significantly (CW-SSIM ≈ 0.3–0.4) as the disruption pattern of LiShield does not fit most known Gaussian noise model. The deformation removal methods (i.e., debanding and unstriping) do not help too much (CW-SSIM ≈ 0.4–0.5), since interpolation process cannot bring back the exact pixel values. The CIEDE2000 color metric also shows a low quality (around 35). Thus, it is hard to fully remove LiShield’s impact by simple image restoration. More advanced computer vision techniques may provide better recovery, but even they will not recover the exact original scene since information is already lost at capture time.