Communications of the ACM

3. 2. Static scene recovery

When the target scene is static, the authorized user may capture a few complementary frames at a specific time to recover the scene as depicted in Figure 3, where frequency and intensity randomization (Section 2. 2) are employed in each frame to ensure robustness. Although it does require recording a very short video, the process is extremely short (200 ms at most) and barely noticeable to the authorized user. Meanwhile, an out-of-sync attacker will still receive corrupted images that cannot reconstruct the original scene by direct frame combination.

Suppose a static scene is to be recovered using Lf frames, referred to as critical frames. To prevent attackers from launching the multiframe attack, the timing of the critical frames is negotiated only between the smart LED and the authorized user through the secure side channel. These Lf frames together must contain the information of the entire scene, that is, they must be complementary, as shown in Figure 3. Meanwhile, all other frames will follow the normal flickering pattern. Since the attackers can neither identify nor predict the timing of the critical frames, the best they can do is to launch the brute-force multiframe attack, which has proven to be ineffective (“Illumination intensity randomization” section).

4. AUTOMATIC PHYSICAL WATERMARKING FOR
PRIVACY ENFORCEMENT

High-intensity ambient light sources (e.g., sunlight, legacy lighting, and ash lights) can create strong interference to LiShield’s illumination waveform, degrading the contrast by adding a constant intensity to both the bright and dark stripes, which may weaken LiShield’s protection. In such scenarios, LiShield degrades itself to a barcode mode, where it embeds barcode in the physical scene to convey privacy policies. The barcode forms low-contrast stripes, which may not fully corrupt the images of the scene, but can still be detected by online photo-distributing hubs (e.g., social website servers), which automatically enforce the policies, without co-operation of the uploader or evidence visible by naked eye. LiShield forms the watermark with just a single light fixture, instead of active displays (e.g., projectors) that are required by conventional systems. The key challenge here is how should LiShield encode the information, so that it can be robustly conveyed to the policy enforcers, despite the (uncontrollable) attacker camera settings?

Embedding. LiShield’s barcode packs multiple frequen-
cies in every image following “Frequency scrambling” sec-
tion but aims to map the ratios between frequencies into
digital information. Suppose LiShield embeds two wave-
forms with frequencies F0 and F1, it chooses the two frequency
components such that F1/F0 equals to a value Rp well known
to the policy enforcers. In other words, the presence of Rp
conveys “no distribution/sharing allowed.” Although width
of stripes is affected by sampling interval ts and exposure
time te (Figure 1(a) and (b)), ratio of stripe widths resulted
from two frequencies (which equals to Rp) remains constant.
Therefore, this encoding mechanism is robust against cam-
era settings.

Since physical scenes usually comprise a mix of spatial frequencies, and spectral power rolls off in higher spatial frequencies, thanks to camera lenses’ limited bandwidth while temporal frequencies are unaffected, LiShield’s barcode uses frequencies that are much higher than the natural frequencies (>400 Hz) in the scene to reduce interference. It is worth noting that since the rolling-shutter sampling rate of all cameras falls in a range ( 30 kHz to slightly over 100 kHz20), LiShield limits its highest flickering frequency to 15 kHz, which respects the Nyquist sampling theorem, so that the barcode can eventually be recovered without any aliasing effect.

To further improve robustness, LiShield leverages redundancy. It embeds multiple pairs of frequency components to make multiple values of Rp either at different rows of the image or in different color channels, further mitigating interference caused by intrinsic spatial patterns within the scene.

Detection. Since the barcode contains M frequencies, that is, fn = fB + (n − 1)∆f, n = 2, 3, …, M (“Frequency scrambling” section), there are MR = C2 M possible frequency ratio values across the image for monochrome barcode (MR = C2 M× 3 for RGB barcode). ∆f must be set large enough to avoid confusion (∆f = 200 Hz in experiments). The barcode decoder, running on the policy enforcer, recognizes the image as protected if there are at least Mb values that roughly match the known ratio Rp, that is, when the value falls within Tb of Rp. Suppose Matt is the number of Rp removed by manual exposure attack (Section 2. 2), these parameters are determined by bounding the false positive rate following an empirical procedure (to be discussed in Section 6. 3).

To detect the frequency ratios, LiShield averages the intensity of each row to get a one-dimension time series sr. LiShield then runs FFT over each series to extract the Mp strongest frequencies. Finally, LiShield combines all unique frequencies extracted and computes all frequency ratios. The redundancy in barcode ensures that it can be robustly detected.

5. IMPLEMENTATION

Testbed setup. Figure 4 shows our smart LED prototype, and the target scenes containing five capture-sensitive objects (document and painting are 2-D objects and others are all 3-D objects). We mount the LED inside a diffusive plastic cover similar to conventional ceiling light covers. We use a programmable motor to hold the camera and control its dis-tance/orientation, in order to create static or dynamic scene setup in a repeatable manner.