troduced, the demand for the game so
outstripped the capacity that had been
built into the game that subscriptions
had to be closed off for months while
the code that distributed the game
was rewritten.
changing chip architectures
Scaling over a set of machines is a distributed computing problem, and the
game and virtual-world programming
culture has had little experience with
this set of problems. This is hardly the
only place where scaling requires the
game programmer to learn a new set
of skills. A change in the trend of chip
design also means these programmers
must learn skills they have never had to
exercise before.
With the possible exception of the
highest end of scientific computing, no
other kind of software has ridden the
advances of Moore’s Law as aggressively as game or virtual-world programs.
As chips have gotten faster, games
and virtual worlds have become more
realistic, more complex, and more immersive. Serious gameplayers invest in
the very best equipment that they can
obtain, and then use techniques such
as overclocking to push even more performance out of those systems.
Now, however, chip designers have
decided to exploit Moore’s Law in a different way. Rather than increasing the
speed of a chip, they are adding multiple cores to a chip running at the same
(or sometimes slower) clock speed.
There are many good reasons for this,
from simplified design to lower power
consumption and heat production,
but it means that the performance of a
single program will not automatically
increase when you run the program
on a new chip. Overall performance
of a group of programs may increase
(since they can all run in parallel) but
not the single program (unless it can
be broken into multiple, cooperating
threads). Games are written as single-threaded programs, however.
In fact, games and virtual worlds
(and especially the server side of these
programs) should be perfect vehicles
to show the performance gains possible with multicore chips and groups of
cooperating servers. Games and virtual
worlds are embarrassingly parallel, in
that most of what goes on in them is
independent of the other things that
By backing the
data in a persistent
fashion rather than
keeping it in main
memory, we gain
some inherent
reliability that has
not been exhibited
by games or worlds
in the past.
are happening. Of the hundreds of
thousands of players who are active in
World of Warcraft at any one time, only
a very small number will be interacting
with any particular player. The same is
true in Second Life and nearly all large-scale games or worlds.
The problem is that the culture that
has grown up around games and virtual
worlds is not one that understands or is
overly familiar with the programming
techniques that are required to exploit
the parallelism inherent in these systems. These are people who grew up on
a single (PC) machine, running a single
thread. Asking them to master the in-tricacies of concurrent programming
or distributed systems takes them away
from their concentration on the game
or world experience itself. Even when
they have the desire, they don’t have
the time or the experience to exploit
these new technologies.
Project Darkstar
It is for these reasons that we started
Project Darkstar ( http://www.pro-jectdarkstar.com), a research effort
attempting to build a server-side infrastructure that will exploit the multithreaded, multicore chips being produced and scaled over a large group
of machines while presenting the programmer with the illusion that he or
she is developing in a single-threaded,
single-machine environment. Hiding threading and distribution is, in
the general case, probably not a good
idea (see http://research.sun.com/
techrep/1994/ abstract- 29.ht ml for a
full argument). Game and world servers tend to follow a very restricted programming model, however, in which
we believe we can hide both concurrency and distribution.
The model is a simple event-based
one in which input from the client is received by the server, which then sets off
a task in response to that event. These
tasks can change the state of the world
(by moving a player, changing the state
of an object, or the like) and initiate
communication. The communication
can be to a single client or to a group
of clients that are all subscribed to the
same communication channel.
We chose this model largely because
this is the way most game and virtual-world servers are already structured.
The challenge was then to keep this