Figure 4. Command strokes: gesture Cmd-c-o-p could send Copy
command to the oS.
command effect as soon as the command stroke is unambiguous15 (Figure 4).
4. 2. Case key
Most of the time the case of a word (lower, upper or title) can
be determined automatically in modern text input systems,
particularly word-based systems. For example, in English
the first word in a sentence and proper nouns are typically
capitalized. However, there are exceptions to these normal
rules. Automatic casing makes the use of the legacy Shift
key unnecessary most of the time, but not all the time. This
situation makes it difficult for the user to decide if to press
the Shift key before entering a word. To correct the case of a
word afterward with the Shift key is even more cumbersome
because the user has to first select the text to be modified,
delete it, and then use the Shift or CapsLock keys to trigger
a mode change, and finally retype the text.
We introduced a new key on the keyboard, the Case
key (see the lower left corner of Figure 1). This key cycles
through the different word case alternatives for the word
just entered or preceding the text caret. The Case key uses
dictionary information to intelligently support nonstandard
casing convention for some words, such as “iPhone”. Since
the Case key modifies the word preceding the current text
caret position (“reverse Polish”) it enables users to perform
case corrections after the word is entered and only when
they are actually needed.
4. 3. Systems
We have designed and implemented many versions of
experimental gestures keyboard systems, variably named
HSK, 16 SHARK, 46 and SHARK221 which was publicly released
from the IBM AlphaWorks site in 2004 (Figure 5). Until
recently both CPU and memory were limited on mobile
devices. But with indexing and aggressive pruning it was
still possible to achieve real-time performance. For exam-
ple, one of the first mobile versions of gesture keyboards
we implemented could store both the gesture and the lan-
guage model for 50,000 words in 450K memory and return
recognition results with less than 20ms average latency on
a phone equipped with a 32-bit 168 MHz Texas Instruments
OMAP1510 CPU (Figure 6). We also led the design and
development of a commercial version of word-gesture key-
board, Shape Writer, released on the iPhone, Android and
Window Mobile platforms in many languages. 37 These sys-
tems reflected increased maturity and practicality, as well
as the mobile platform hardware and software constraints
at the time. Working with platform and technical con-
straints was a part of a journey of research and innovation.
5. emPiRiCaL ReSeaRCh
One would imagine it is simple to determine a new text
input method’s efficacy by measuring the average user’s
average speed. An example to the contrary is the decades’
old debate of QWERTY versus the Dvorak simplified keyboard that spilled over even into economic theories. 10, 26 It
is difficult to design and execute decisive tests for text entry.
There are many reasons for this challenge, including learning, speed-accuracy trade-off, and the multifaceted nature
of use quality.
Figure 5. Shorthand aided rapid keyboarding (ShaRK). the first publicly
released fully functioning word-gesture keyboard (october 2004).
Figure 6. one of the first implementations of mobile word-gesture keyboard
running on a mobile device with a 168mhz processor in real time (2006).