( 100 Mbps–1Gbps)
When uplink is heavily utilized
the router buffers will fill
Rate mismatch will
cause CM buffer to fill
(bufferbloat) and may
cause priority inversion
a CeroWrt-enabled edge router to
make bufferbloat disappear. Unfortunately, large buffers are not always
located where they can be managed
but can be ubiquitous and hidden. 6, 7
Examples include consumer-edge
routers connected to cable modems
and wireless access points with ring
buffers. Many users access the Internet through a cable modem with
varying upstream link speeds: 2 Mbps
is a typical value. The home network
or home computer connects to the
cable modem with an Ethernet cable
in the range of 100Mbps–1Gbps (
Figure 10). The modem’s buffers are at
the fast-to-slow transition, and that is
where queues will build up: inside a
sealed device outside of user control.
Any differentiated services (DiffServ)
queuing and buffer management at
the router can be defeated by a full
queue in the cable modem. Three approaches have been suggested: limit
the Ethernet link to the upstream
rate; put buffer management and DiffServ queues in the cable modem with
a configuration interface for the DiffServ queues; and implement Ethernet
flow control between the modem and
the upstream router.
Option 1 has the advantage that it
can be implemented by a user with-
out cable-modem changes and the
disadvantage that it must rate limit to
the expected rate of the upstream. If
the rate drops, then a queue will still
build up in the cable modem, and any
additional short-term bandwidth can-
not be utilized. Option 2 puts buffer
management right at the bottleneck
link but requires the vendor to make
(possibly significant) changes to the
modem architecture and permit con-
figuration. Option 3 also requires
vendors to make changes but uses
Ethernet flow control to permit only
the number of packets in the modem
buffer needed for good transmission
utilization while pushing the queue
into the router where it can be man-
aged and where new algorithms can
be more readily deployed. Options 2
or 3 are preferable but require a cable-
modem vendor and/or a cable data
network service provider to make this
part of the modem requirements.
The open source project CeroWrt3 is
using Open Wrt to explore solutions to
bufferbloat. A CoDel implementation
is in the works, after which real-world
data can be studied. We plan to make
our ns- 2 simulation code available, as
well as some further results. 19
1. braden, r. et al. recommendations on queue
management and congestion avoidance in the
internet. rFC 2309 (1998).
2. bufferbloat Project; http://www.bufferbloat.net.
3. Cero Wrt Project; http://www.bufferbloat.net/projects/
4. dischinger, M. et. al. Characterizing residential
broadband networks. in Proceedings of the Internet
Measurement Conference, san diego, Ca, 2007.
5. Floyd, s. and Jacobson, v. random early detection
gateways for congestion avoidance. IEEE/ACM
Transactions on Networking, (1993).
6. gettys, J. bufferbloat: dark buffers in the internet.
backspace Column. IEEE Internet Computing 15, 3
7. gettys, J. and nichols, K. 2011. bufferbloat: dark
buffers in the internet. Commun. ACM 9, 11 (sept.
8. Jacobson, v. Congestion avoidance and control.
Proceedings of SIGCOMM ’ 88, (stanford, Ca, 1998).
9. Jacobson, v. reported in Minutes of the Performance
Working group. Proceedings of the Cocoa Beach
Internet Engineering Task Force. Corporation for
national research initiatives, reston, va, 1989.
10. Jacobson, v. notes on using red for queue
management and congestion avoidance. talk
presented at north american network operators’
group (1998); ftp://ftp.ee.lbl.gov/talks/vj-nanog-red.pdf.
11. Jacobson, v. a rant on queues. a talk presented at
Mit lincoln labs, lexington, Ma, 2006; http://www.
12. Jacobson, v., nichols, K. and Poduri, K. red in a
different light, 1999; http://www.cnaf.infn.it/~ferrari/
papers/ispn/red_light_ 9_ 30.pdf.
13. Kreibich, C. et. al. netalyzr: illuminating the edge
network. in Proceedings of the Internet Measurement
Conference, (Melbourne, australia, 2010).
14. li, t. and leith, d. adaptive buffer sizing for tCP
flows in 802.11e Wlans. in Proceedings of the 2008
Communications and Networking in China.
15. Mankin, a. random drop congestion control. in
Proceedings of SIGCOMM ’ 90.
16. Mathis, M., semke, J. and Mahdavi, J. the macroscopic
behavior of the tCP congestion avoidance algorithm.
ACM SIGCOMM Computer Communication Review 27,
17. nagle, J. Congestion control in iP/tCP internet works.
rFC 896 (1984); http://www.ietf.org/rfc/rfc896.txt,.
18. network simulator - ns- 2; http://nsnam.isi.edu/nsnam/
20. vu-brugier, g., et. al. 2007. a critique of recently
proposed buffer-sizing strategies. ACM SIGCOMM
Computer Communication Review 37( 1).
21. Weigle, M. C. 2002. Web traffic generation in ns- 2 with
22. Feng, W. et. al. 2002. the blue active Queue
Management algorithm. IEEE/ACM Transactions on
Networking 10, 4 (2002), 513–528.
Kathleen nichols is the founder and Cto of Pollere inc.,
a consulting company working in both government and
commercial networking. she has 30 years of experience
in networking, including a number of silicon valley
companies and as a cofounder of Packet design.
Van Jacobson is a research Fellow at ParC where he
leads its content-centric networking research program.
he has worked at lawrence berkeley national laboratory,
Cisco systems, and was a cofounder of Packet design.
Bufferbloat: Dark Buffers in the Internet
Jim Gettys, Kathleen Nichols
Revisiting network I/O APIs:
The netmap Framework
The Robustness Principle Reconsidered