COATTA: So far, we have talked only
about general issues. Now I would like
to hear about some of your more specific engineering challenges.
BONÉR: One thing I would like to
know is whether you’re mostly doing
reactive scaling, predictive scaling, or
some combination of the two to optimize for your hardware.
BERUBE: For now at least, our loads
don’t really change a lot. Or perhaps
what I should say is that they change
throughout the day, but predictably so
from one day to the next. And the way
we have designed our services to run
behind brokers means we are able just
to spin off more workers as necessary.
In combination with some great tooling from AWS (Amazon Web Services),
we are able to adapt quickly to changing workloads.
STEEL: One thing we did decide to
do was to build a framework using ZeroMQ to enable process communication between our various PHP systems.
But then we saw later that we could have
just as easily pulled all that into Akka.
COATTA: And I’m assuming, with
Akka, it would have been easier for you
to achieve your goal of adhering to the
actor paradigm, while also taking advantage of better recovery mechanisms
and finer-grained control.
STEEL: Yes, but I think the key is that
because of our ability to change the
characteristics of actors by how we
configure them, we have been able to
adapt this core framework to all types
of payloads and traffic profiles for the
various services. We can say, “This ser-
vice is using a blocking database,” at
supported by the Lightbend stack sur-
faced almost immediately as the Hoot-
suite engineering team started discov-
ering opportunities for scaling down
on the underlying physical and virtual
infrastructure they had run previously
on their LAMP stack (where there had
been a process for each request). Ac-
cordingly, it soon became apparent
that operations under the reactive mi-
croservices model were going to put
much less strain on their resources.
In fact, if anything, the engineers
at Hootsuite quickly learned that by
continuing to employ some of the practices that had made sense with a LAMP
stack, they would actually be denying
themselves many of the benefits available by relying to a greater degree on
the Lightbend stack. For example, they
found there was a real advantage to
making greater use of the model classes supported by the Lightbend stack,
since those classes come equipped
with data-layer knowledge that can
prove to be quite useful in a dynamic
Similarly, they learned that by using individual actors to run substantial portions of their system instead of
decomposing those components into
groups of actors, they had been unwittingly depriving themselves of some
of the features Akka offers for tuning
parts of the system separately, parallelizing them, and then distributing work
efficiently among a number of different actors capable of sharing the load.
And then there were also a few other
things they learned ...
Because of our
ability to change
of actors by how
we configure them,
we have been able
to adapt this core
framework to all
types of payloads
and traffic profiles
for the various