sizing was to have a bandwidth-delay
product (BDP’s) worth of buffer, where
bandwidth is the bottleneck link and
delay is the round-trip time (RTT) between sender and destination. The rationale is that such a buffer can hold an
entire “flight” of packets should they all
arrive at the bottleneck link in a burst.
To apply this rule, the bandwidth used
in link buffer sizing was that of the
immediately outgoing link, since the
location of the actual bottleneck is unknown. Similarly, a canonical value was
suggested for the RTT: 100ms, a continental delay for the U.S. and Europe.
Once adequate buffers became routine, another problem could occur: the
buffers were now part of the pipe that
TCP is so good at filling. Filling these
buffers would cause delay to increase,
and persistently full buffers lack the
space to absorb the routine burstiness
of a packet network. John Nagle’s seminal work in 198510 first drew attention
to the consequences of large buffering.
While working on TCP congestion-avoidance algorithms, Van Jacobson
recognized the “persistently full buffer” issue in 1989, culminating in the
development of Random Early Detection (RED) with Sally Floyd in 1993.5
A number of implementations,
variations, imitations, and reports on
RED’s use are available in the literature. 14 These are generically termed active queue management (AQM), which
attempts to keep the queues at the
figure 2. smokeping from Jim’s home to Mit.
bottleneck from growing too large by
monitoring the growth of the packet
queue and signaling the sender’s TCP
to slow down by dropping (or marking)
packets in a timely fashion. Different
approaches have been taken to monitoring the packet queue and making
the drop (or mark) decision. The Internet Research Task Force (IRTF) urged
the deployment of active queue management in the Internet, publishing an
RFC in 1998, popularly known as “the
RED manifesto.” 2
Note that packet loss for TCP is not
in itself a problem but is essential for its
functioning in the face of congestion.
The excessive and consecutive packet
losses that come from persistently full
buffers do present a problem, which is
what the “warning” drops of AQM prevent (in addition to long delays).
The truth is, AQM is not widely or
consistently configured and enabled
in routers and it is completely unavailable in many devices. Furthermore,
the existence of cheap memory and
the misguided desire to avoid packet
loss have led to larger and larger buffers being deployed in the hosts, routers, and switches that make up the Internet. It turns out this is a recipe for
bufferbloat. Evidence of bufferbloat
has been accumulating over the past
decade, but its existence has not yet
become a widespread cause for concern. The next section outlines Jim’s
personal journey of discovery.
“The Internet is slow today, Daddy.”
This had become a frequent refrain in
the Gettys household. When I would attempt to debug the problem, like a will-o’-the-wisp, it would usually vanish. On
several occasions symptoms occurred
long enough for me to waste significant
amounts of time on my ISP’s support
line before they vanished. I attributed
the recurring problem to the doubtful quality of the cable to my house or
equipment damage from a lightning
strike. Since my job is research in immersive teleconferencing, I knew I had
to dig into the intermittent poor network problem, if only for myself.
An Enlightening Lunch. Suspecting
features of my cable provider to be
part of the problem, I met with Comcast’s Rich Woundy, who provided a
number of new issues to consider:
˲˲The “big-buffers” problem,
which David Clark (Internet network
architect, currently senior research
scientist at MIT) had warned about
several years earlier.
˲ ˲Broadband measurement studies have been indicating overly large
˲ ˲ AQM is MIA—many ISPs are running without any AQM even in circumstances where they really should.
˲ ˲A group at UC Berkeley’s ICSI
(International Computer Science Institute) had developed a very fine tool
Last 3 hours
Median Ping Rtt (855.5 ms avg) 0 1/20 2/20 3/20 4/20 10/20 19/20
Packet Loss: 33.34% average 83.91% maximum 62.05% current
Probe: 20 iCMP echo Pings (1024 Bytes) every 300 seconds created on fri Jul 16 14:34: 50 2010