default in Linux. CUBIC’s initial window growth is like Jacobson’s original
algorithm, but it “flattens out” at an
apparently stable pipe-size estimate
for a while. 8 If it does not detect a congestion event (that is, a packet drop),
then it ramps the window up quickly.
In the trace in Figure 3, about five seconds pass without a drop, whereupon
TCP ramps up the window in an attempt to find a new operating point;
after 10 seconds the buffer is full,
and packet loss occurs. (Because of
the offload engine, the RTTs are from
the last packet of the jumbogram
it receives.) Notice the RTT ramps
up quickly to about 1. 2 seconds and
mostly stays there. The three-sec-ond RTT spikes show where massive
dropping took place when the buffer
became full. These are followed by a
drop in window size and RTT. This,
in turn, causes TCP to shut down the
window size. The window-size curve
illustrates that sometimes the algorithm gets a drop before it goes into
CUBIC’s second probing stage.
Figure 4 shows the goodput (
determined from the ACKs) versus time. The
initial brief period of nearly 10Mbps is
a result of Comcast’s PowerBoost feature and is followed by a steady 2Mbps,
showing that I was getting all the bandwidth I expected. Figure 4b shows the
RTT seen at each window size. The
lower data set clearly results from the
PowerBoost phase and the upper data
set is the subsequent 2Mbps phase.
These points show exactly the situation shown abstractly in Figure 1: the
delay is about 10ms initially; then the
window size increases, the buffer fills
up, and the delay (as measured by the
RTT) increases linearly.
TCP should approximately share a
bottleneck link between competing
flows. The impact of bufferbloat on
TCP’s behavior is subtle and profound
in two ways:
˲ ˲ For TCP congestion avoidance to
be useful to people using that link,
a TCP connection causing conges-
tion must react quickly to changes in
demand at the bottleneck link, but
TCP’s reaction time is quadratic to the
amount of overbuffering. A link that is
10 times overbuffered not only imposes
10 times the latency, but also takes 100
times as long to react to the congestion.
Your short, interactive TCP connection
loses completely to any long-lived flow
that has saturated your link.
figure 4a. uplink bandwidth during trace.
figure 4b. scatterplot of Rtt experienced vs. window size during trace.
200 400 600 800 1000 1200
RT T (ms)
0 10 20 30 40 50 60
0 10 20 30 40 50 60 70
slope = 0.85 ms/KB ( 9. 4 mbps)
Window Size (KB)