Automating Visual Privacy Protection Using a Smart LED

By Shilin Zhu, Chi Zhang, and Xinyu Zhang

DOI: 10.1145/3375571

Abstract

The ubiquity of mobile camera devices has been triggering an outcry of privacy concerns, whereas existing privacy protection solutions still rely on the cooperation of the photographer or camera hardware, which can hardly be enforced in practice. In this paper, we introduce LiShield, which automatically protects a physical scene against photographing, by illuminating it with smart LEDs flickering in specialized waveforms. We use a model-driven approach to optimize the waveform design, so as to ensure protection against the (uncontrollable) cameras and potential image-processing– based attacks. We have also designed mechanisms to unblock authorized cameras and enable graceful degradation under strong ambient light interference. Our prototype implementation and experiments show that LiShield can effectively destroy unauthorized capturing while maintaining robustness against potential attacks.

1. INTRODUCTION

Cameras are now pervasive on consumer mobile devices, such as smartphones, tablets, drones, smart glasses, first-person recorders, etc. The ubiquity of these cameras, paired with pervasive wireless access, is creating a new wave of visual sensing applications, for example, autonomous photograph, quantified-self (life-logging), photo-sharing social networks, physical analytics in retail stores, 12 and augmented reality applications that navigate users across unknown environment. 19 Zooming into the photo-sharing application alone, statistics report that 350 million photos/videos are uploaded to Facebook every day, majority of which are from mobile users. 15 Many of these applications automatically upload batches of images/videos online, with a simple one-time permission from the user. Although these technologies bring significant convenience to individuals, they also trigger an outcry of privacy concerns.

Privacy is ultimately a subjective matter and often varies
with context. Yet, many of the privacy-sensitive scenes occur
in indoor environment and are bound to specific locations.
For example, recent user studies2 showed that people’s
acceptability of being recorded by augmented reality glasses
has a strong correlation with location. User studies of life-
logging cameras6 also indicate that 70.2% of the cases when
the user disables capturing are associated with specific loca-
tions. In numerous real-world scenarios, cameras are forbid-
den, for example, concerts, theaters, museums, trade shows,
hospitals, dressing rooms and exam rooms, manufacturing
plants, etc. However, visual privacy protection in such passive
physical spaces still heavily relies on rudimentary approaches
like warning signs and human monitors, and there is no
way to automatically enforce the requirements. In personal
visual sensing applications like life-logging, even if a user
The original version of this paper appeared in ACM
Annual International Conference on Mobile Computing
and Networking (MobiCom), 2017.

were to disable the camera in private space, malware could perform remote reconnaissance and target visual theft by hijacking the victim’s camera. 16

In this paper, we propose LiShield, a system that deters photographing of sensitive indoor physical space and automatically enforces location-bound visual privacy protection. LiShield protects the physical scenes against undesired recording without requiring user intervention and without disrupting the human visual perception. Our key idea is to illuminate the environment using smart LEDs, which are intensity-modulated following specialized waveforms. We design the waveform in such a way that its modulation pattern is imperceptible by human eyes but can interfere with the image sensors on mobile camera devices.

Adversary model and protection goals. LiShield aims to prevent ad-hoc capturing from benign camera-phone holders. The physical space under protection can be static or dynamic. In either case, we assume that one or multiple LiShield-enabled smart LEDs can cover the whole area, while providing illumination similar to normal office lighting without human-perceptible flickering. Although conventional lighting and sunlight may co-exist with LiShield’s smart LEDs, covering the entire target scene with LiShield will ensure the strongest protection.

Now consider an unauthorized user (attacker) who wants to take pictures or videos within the protected space, with cameras and ashes embedded in smartphones, but with no professional equipment such as global shutter cameras, filters, or tripods. The attacker has full control over the camera parameters (e.g., exposure time, capturing time, and white-balancing) and can run any postprocessing on the captured images. Nonetheless, with LiShield’s protection, the image frames are corrupted, so that major fraction of each frame is either blank or overexposed, whereas colors are distorted (Section 2), which deters image viewing/sharing.

In addition, LiShield should possess the following capabilities for practical usage scenarios: (i) allowing an authorized camera, which shares secret configuration information with the LED, to recover the image or video frames it captures. (ii) when strong ambient light interferes with the smart LED, LiShield cannot ensure full protection, but it can still emit structured light which embeds invisible “barcode” into the physical environment. The embedded information can convey a “no distribution” message, allowing online servers (e.g., from Facebook and Instagram) to block and prevent the image from being distributed.