Figure 3. top: the front side of the sensor PCB showing the 7 × 5 array of iR optosensors. the transistors that enable each detector are visible to the right of each optosensor. Bottom: the back of the sensor PCB has little more than a PiC microcontroller, a uSB interface and FEts to drive the rows and columns of iR emitting LEDs. three such PCBs are used in our thinSight laptop while there are thirty in the tabletop prototype.

IR LED column drivers

USB interface

IR LED row drivers

PIC micro

3. 2. LCD technology overview

To understand how the ThinSight hardware is integrated into a display panel, it is useful to understand the construction and operation of a typical LCD. An LCD panel is made up of a stack of optical components as shown in Figure 4. At the front of the panel is a thin layer of liquid crystal material which is sandwiched between two polarizers. The polarizers are orthogonal to each other, which means that any light which passes through the first will naturally be blocked by the second, resulting in dark pixels. However, if a voltage is applied across the liquid crystal material at a certain pixel location, the polarization of light incident on that pixel is twisted through 90° as it passes through the crystal structure. As a result it emerges from the crystal with the correct polarization to pass through the second polarizer. Typically, white light is shone through the panel from behind by a backlight and red, green, and blue filters are used to create a color display. In order to achieve a low profile construction while maintaining uniform lighting across the entire display and keeping cost down, the backlight is often a large “light guide” in the form of a clear acrylic sheet which sits behind the entire LCD and which is edge-lit from one or more sides. The light source is often a cold cathode fluorescent tube or an array of white LEDs. To maximize the efficiency and uniformity of the lighting, additional layers of material may

92 CommuniCationS oF thE aCm | DeCeMBeR 2009 | vOL. 52 | NO. 12

Standard edge-lit LCD

Figure 4. typical LCD edge-lit architecture shown left. the LCD comprises a stack of optical elements. a white light source is typically located along one or two edges at the back of the panel. a white reflector and transparent light guide direct the light toward the front of the panel. the films help scatter this light uniformly and enhance brightness. however, they also cause excessive attenuation of iR light. in thinSight, shown right, the films are substituted and placed behind the light guide to minimize attenuation and also reduce noise caused by LCD flexing upon touch. the sensors and emitters are placed at the bottom of the resulting stack, aligned with holes cut in the reflector.

Standard edge-lit LCD with ThinSight

LCD and polarizers Diffuser and brightness Enhancing Film Light guide White light source Neutral density filter and Radiant Light Film Reflector

Sensor Emitter

be placed between the light guide and the LCD. Brightness enhancing film (BEF) “recycles” visible light at suboptimal angles and polarizations and a diffuser smoothes out any local nonuniformities in light intensity.

3. 3. integration with an LCD panel

We constructed our ThinSight prototypes using a variety of desktop and laptop LCD panels, ranging from 17" to 21". Two of these are shown in Figures 5 and 6. Up to 30 PCBs were tiled to support sensing across the entire surface. In instances where large numbers of PCBs were tiled, a custom hub circuit based on an FPGA was designed to collect and aggregate the raw data captured from a number of tiled sensors and transfer this to the PC using a single USB channel. These tiled PCBs are mounted directly behind the light guide. To ensure that the cold cathode does not cause any stray IR light to emanate from the acrylic light guide, we placed a narrow piece of IR-blocking film between it and the backlight. We cut small holes in the white reflector behind the light guide to coincide with the location of every IR emitting and detecting element.

During our experiments we found that the combination of the diffuser and BEF in an LCD panel typically caused excessive attenuation of the IR signal. However, removing these materials degrades the displayed image significantly: without BEF the brightness and contrast of the displayed image is reduced unacceptably; without a diffuser the image appears to “float” in front of the backlight and at the same time the position of the IR emitters and detectors can be seen in the form of an array of faint dots across the entire display.

To completely hide the IR emitters and detectors we required a material that lets IR pass through it but not visible light, so that the optosensors could not be seen but would operate normally. The traditional solution would be