Fighting Physics: A Tough Battle
Consider a company that wants to build a distributed
application that does IPC (interprocess communication) over the long haul. The obvious advice is “just say
no”—don’t do it. If you’re going far outside your local
networking environment, the physics of distance and
the speed of light, combined with the delays that come
from the Internet’s routing infrastructure, tell us that it
will be much too slow. These concepts are not generally
understood, however, and even when they are, they’re
So, exactly what are the basic principles related to
speed of light and network hops that all software developers need to acquaint themselves with? This article
answers that question by first working out some quantitative preliminaries with an example, moving on to the
networking implications, and then covering applications.
Finally, it provides some rules of thumb to keep in mind
as applications and architectures evolve in reaction to
new network capabilities and unchanging physics.
PRELIMINARIES: THE PH YSICS
The speed of light in a vacuum is exactly 299,792,458
meters/second.1 This is as fast as you can move a bit of
data, and according to our current understanding of physics, it is a fundamental constraint of the universe we live
in. In fiber, the speed of light is 2.14×108 meters/second
or about 70 percent of the speed of light in a vacuum. If
a fiber were stretched in a straight line from New York to
San Francisco, it would be about 4,125 kilometers long,
and it would take about 19 ( 4,125 ÷ 214) milliseconds for
light to make the one-way trip. Assuming an 8,250-km
length of fiber was used, you can just double this time to
get an estimate for minimum round-trip time.
At first glance, 19 ms might seem like a short time, certainly on a human scale. As computer scientists, however,
we are usually concerned with a different time scale: that
of the computer. Here we can calculate the 19 ms
in terms of instructions, the fundamental units of work
for computers. As an example, we can use a 2003-vintage
single-core machine: the Intel Pentium 4 Extreme Edition, which at a 3.2-GHz clock rate was rated at 9,726 million instructions per second: 9,726 × 0.019 is 184 million
instructions—sufficient, for example, to search through or
sort millions of names.
It is always important to keep in mind that the
purpose of computer networking is to interconnect
computers, and that computers operate on very short
timescales. Also, a single human operation sometimes
translates to many computer operations (i.e., round-trips).
For example, opening a single Web page usually requires
many round-trips, even if you are getting only a single
large object (e.g., a large picture), as is discussed further
later in this article.
PROPAGATION, BANDWIDTH, LATENCIES, AND HOPS
The traversal of the fiber loop between New York and
San Francisco presumes a data-transfer unit of a single
encoded binary digit of information. The lower bound
for that traversal would be 2×
19, or 38 ms (or 368 million instructions). The time for this bit to travel from
its source to its destination and back again is called its
Propagation delay is important, but compared with
the much more common metric of bandwidth—measured
in bits per second—it is rarely quoted as a figure of merit.
At least partially, this is because the observed propagation delay depends on context, whereas bandwidth (say
of a fiber-optic transmission system) can be measured
in isolation. Bandwidth can also be increased through
engineering (for example, through encoding schemes
for transmission systems that encode multiple bits per
symbol) and thus is more attractive as a figure of merit to
those who build transmission systems. Finally, bandwidth
is a measure of work, which is attractive to purchasers.
Bandwidth can also affect latency, which is distinct, in
my view, from propagation delay; the propagation delay