on the Internet
Today, the first mile—that is, origin infrastructure—
tends to get most of the attention when it comes to
designing Web applications. This is the portion of the
problem that falls most within an application architect’s
control. Achieving good first-mile performance and reliability is now a fairly well-understood and tractable problem. From the end user’s point of view, however, a robust
first mile is necessary, but not sufficient, for achieving
strong application performance and reliability.
This is where the middle mile comes in. Difficult to
tame and often ignored, the Internet’s nebulous middle
mile injects latency bottlenecks, throughput constraints,
and reliability problems into the Web application performance equation. Indeed, the term middle mile is itself
a misnomer in that it refers to a heterogeneous infrastructure that is owned by many competing entities and
typically spans hundreds or thousands of miles.
This article highlights the most serious challenges
the middle mile presents today and offers a look at the
approaches to overcoming these challenges and improving performance on the Internet.
S TUCK IN THE MIDDLE
While we often refer to the Internet as a single entity,
it is actually composed of 13,000 different competing
networks, each providing access to some small subset of
end users. Internet capacity has evolved over the years,
shaped by market economics. Money flows into the networks from the first and last miles, as companies pay for
hosting and end users pay for access. First- and last-mile
capacity have grown 20- and 50-fold, respectively, over
the past five to 10 years.
On the other hand, the Internet’s middle mile—made
up of the peering and transit points where networks
trade traffic—is literally a no man’s land. Here, economically, there is very little incentive to build out capacity.
If anything, networks want to minimize traffic coming
into their networks that they don’t get paid for. As a
result, peering points are often overburdened, causing
packet loss and service degradation.
The fragile economic model of peering can have
even more serious consequences. In March 2008, for
example, two major network providers, Cogent and
Telia, de-peered over a business dispute. For more than a
week, customers from Cogent lost access to Telia and the
networks connected to it, and vice versa, meaning that
Cogent and Telia end users could not reach certain Web
sites at all.
Other reliability issues plague the middle mile as well.
Internet outages have causes as varied as transoceanic
cable cuts, power outages, and DDoS (distributed denial
of service) attacks. In February 2008, for example, communications were severely disrupted in Southeast Asia
and the Middle East when a series of undersea cables
were cut. According to TeleGeography, the cuts reduced
bandwidth connectivity between Europe and the Middle
East by 75 percent.
Internet protocols such as BGP (Border Gateway
Protocol, the Internet’s primary internetwork routing
algorithm) are just as susceptible as the physical network
infrastructure. For example, in February 2008, when
Pakistan tried to block access to You Tube from within
the country by broadcasting a more specific BGP route,
it accidentally caused a near-global You Tube blackout,
underscoring the vulnerability of BGP to human error
(as well as foul play).
The prevalence of these Internet reliability and peering-point problems means that the longer data must
travel through the middle mile, the more it is subject to
congestion, packet loss, and poor performance. These
middle-mile problems are further exacerbated by current
trends—most notably the increase in last-mile capacity
and demand. Broadband adoption continues to rise, in
terms of both penetration and speed, as ISPs invest in
last-mile infrastructure. AT&T just spent approximately
$6.5 billion to roll out its U-verse service, while Veri-zon is spending $23 billion to wire 18 million homes
with FiOS (Fiber-optic Service) by 2010.7, 6 Comcast also
recently announced it plans to offer speeds of up to 100
Mbps within a year.
Demand drives this last-mile boom: Pew Internet’s
2008 report shows that one-third of U.S. broadband users
have chosen to pay more for faster connections.
Technologies’ data, shown in table 1, reveals that 59 percent of its global users have broadband connections (with
speeds greater than 2 Mbps), and 19 percent of global
users have “high broadband” connections greater than 5
Mbps—fast enough to support DVD-quality content.
high-broadband numbers represent a 19 percent increase
in just three months.