MODERN SERVER SOFTWARE is demanding to develop
and operate: It must be available at all times and in
all locations; it must reply within milliseconds to user
requests; it must respond quickly to capacity demands;
it must process a lot of data and even more traffic; it
must adapt quickly to changing product needs; and, in
many cases, it must accommodate a large engineering
organization, its many engineers the proverbial cooks
in a big, messy kitchen.
What’s more, the best computers for these applications—
whether you call them clouds, datacenters, or warehouse
computers—are really bad. They are complex and
unreliable, and prone to partial failures. They have
asynchronous interconnects and deep memory
hierarchies, and leave a lot of room for operator error.
Cloud computing thus forces us to confront the full
complexity of distributed computing, where seemingly
simple problems require complicated solutions.
While much of this complexity is inherent—the very
nature of the problem precludes simpler solutions—
much of it is also incidental, a simple consequence of
using tools unfit for the purpose.
At Twitter, we use ideas from functional programming to tackle many
of the complexities of modern server
software, primarily through the use
of higher-order functions and effects.
Higher-order functions, or those that
return other functions, let us combine simpler functions to define more
complex ones, building up application functionality in a piecemeal fashion. Effects are values that represent
some side-effecting operation; they
are used in conjunction with higher-order functions to build complex effects from simpler ones.
Higher-order functions and effects
help build scalable software in two
ways: First, building complex software
from simple parts makes it easy to understand, test, reuse, and replace individual components; second, effects
make side-effecting operations tractable, promoting modularity and good
separation of concerns.
This article explores three specific
abstractions that follow this style of
functional programming: futures are
effects that represent asynchronous
operations; services are functions that
represent service boundaries; and
filters are functions that encapsulate
application-independent behavior. In
turn, higher-order functions provide
the glue used to combine these to create complex systems from simple parts.
Futures, services, and filters are thus
combined to build server software in a
piecemeal fashion. They let programmers build up complex software while
preserving their ability to reason about
the correctness of its constituent parts.
By consistently applying these principles, programmers can construct
systems that are at once simpler, more
flexible, and performant.
“Concurrent programs wait faster.”
Concurrency is a central topic in
sources of concurrency prevail in this
Article development led by
Applying functional programming principles
to distributed computing projects.
BY MARIUS ERIKSEN