Taking a Long Look at QUIC
An Approach for Rigorous Evaluation of Rapidly
Evolving Transport Protocols
By Arash Molavi Kakhki,* Samuel Jero, David Choffnes, Cristina Nita-Rotaru, and Alan Mislove
Google’s Quick UDP Internet Connections (QUIC) protocol, which implements TCP-like properties at the application layer atop a UDP transport, is now used by the vast
majority of Chrome clients accessing Google properties
but has no formal state machine specification, limited
analysis, and ad-hoc evaluations based on snapshots of the
protocol implementation in a small number of environments. Further frustrating attempts to evaluate QUIC is the
fact that the protocol is under rapid development, with
extensive rewriting of the protocol occurring over the scale
of months, making individual studies of the protocol obsolete before publication.
Given this unique scenario, there is a need for alternative techniques for understanding and evaluating QUIC
when compared with previous transport-layer protocols.
First, we develop an approach that allows us to conduct
analysis across multiple versions of QUIC to understand
how code changes impact protocol effectiveness. Next, we
instrument the source code to infer QUIC’s state machine
from execution traces. With this model, we run QUIC in a
large number of environments that include desktop and
mobile, wired and wireless environments and use the
state machine to understand differences in transport-and application-layer performance across multiple versions of QUIC and in different environments. QUIC
generally outperforms TCP, but we also identified performance issues related to window sizes, re-ordered packets,
and multiplexing large number of small objects; further,
we identify that QUIC’s performance diminishes on
mobile devices and over cellular networks.
Transport-layer congestion control is one of the most important elements for enabling both fair and high utilization of
Internet links shared by multiple flows. As such, new transport-layer protocols typically undergo rigorous design, analysis, and evaluation—producing public and repeatable
results demonstrating a candidate protocol’s correctness
and fairness to existing protocols—before deployment in
the Operating System (S) kernel at scale.
Because this process takes time, years can pass between
development of a new transport-layer protocol and its wide
deployment in operating systems. In contrast, developing
an application-layer transport (i.e., one not requiring OS The original version of this paper was published in the
Proceedings of the 2017 ACM Internet Measure Conference
(London, U.K., Nov. 1–3, 2017).
kernel support) can enable rapid evolution and innovation
by requiring only changes to application code, with the
potential cost due to performance issues arising from pro-
cessing packets in userspace instead of in the kernel.
The Quick UDP Internet Connections (QUIC) protocol,
initially released by Google in 2013, 4 takes the latter
approach by implementing reliable, high-performance,
in-order packet delivery with congestion control at the
application layer (and using UDP as the transport layer).a
Far from just an experiment in a lab, QUIC is supported
by all Google services and the Google Chrome browser; as
of 2016, more than 85% of Chrome requests to Google servers use QUIC. 21 b In fact, given the popularity of Google
services (including search and video), QUIC now represents a substantial fraction (estimated at 7% 15) of all
Internet traffic. While initial performance results from
Google show significant gains compared to TCP for the
slowest 1% of connections and for video streaming, 9
there have been very few repeatable studies measuring
and explaining the performance of QUIC compared with
standard HTTP/2+TCP. 8, 11, 17 In addition to Google’s
QUIC, an IETF working group established in 2016 is
working on standardizing QUIC and there are more than
20 QUIC implementations in progress. 2 Our study focuses
on Google’s QUIC implementation.
Our overarching goal is to understand the benefits and
tradeoffs that QUIC provides. In this work, we address a
number of challenges to properly evaluate QUIC and make
the following key contributions.
First, we identify a number of pitfalls for application-layer protocol evaluation in emulated environments and
across multiple QUIC versions. Through extensive calibration and validation, we identify a set of configuration
parameters that fairly compare QUIC, as deployed by
Google, with TCP-based alternatives.
Second, we develop a methodology that automatically
generates network traffic to QUIC- and TCP-supporting
servers in a way that enables head-to-head comparisons.
Further, we instrument QUIC to identify the root causes
Work done while at Northeastern University.
a It also implements TLS and SPDY, as described in the next section.
b Newer versions of QUIC running on servers are incompatible with older
clients, and ISPs sometimes block QUIC as an unknown protocol. In such
cases, Chrome falls back to TCP.