main obscured within Photoshop’s synchronization code for 10 years serves to
illustrate just how tricky parallel programming can be. But it also highlights
how much progress has been made in
terms of improved resources for managing some of this complexity. Had Photoshop’s synchronization been written
today using C++’s stack-based locking,
for example, it’s unlikely a bug of that
sort would have been introduced. As
processors get more cores and grow in
complexity, the need will only intensify
for new tools and better programming
primitives for hiding the complexity
from developers and allowing them to
code at higher levels of abstraction.
At the same time, software architects also need to keep an eye on some
other fundamental issues. For example,
despite using less-sophisticated synchronization primitives in the original
design, the Photoshop team was able
to essentially forget about complex
thread-synchronization problems, in
part because the image-processing
problem itself was so amenable to parallelization. Also, however, Photoshop’s
architecture made it possible to establish some very clean object boundaries,
which in turn made it easy for programmers to slice up objects and spread the
resulting pieces across multiple processors. Indeed, the architects of Photoshop were keenly aware of where their
best opportunities for parallelization
existed, and they designed the application accordingly.
Generalizing on this, it’s clear that—
with or without advances in tools and
programming abstractions—in order
for developers to fully leverage the multicore architectures that are coming,
they’ll need to be adept at identifying
those parts of a program that can benefit most from parallelization. It’s in
these portions of the code that new
tools, techniques, and parallel programming abstractions are likely to
have the greatest impact.
coLe: As operating-system designers, we
both grew up in a world where we had
to deal with parallelism. It’s not always
clear that the solutions we came up with
for our operating systems proved to be
the right ones. In an earlier conversation, you mentioned your experience
with creating and removing mutexes.
We’ve gotten smarter over the years.
RuSSeLL WiLLiamS
i think we’ve made
some incremental
gains in our ability
to deal with
parallelism over
the past 20 years,
just as we’ve
made stepwise
progress on all
other programming
fronts.
We’ve learned how to do things that are
more efficient, but that doesn’t mean it
has gotten any easier. What do we have
up our sleeves to make it easier?
WiLLiamS: Parallelism remains difficult in a couple of ways. It’s one thing
to ask for a new Photoshop filter for
processing a grid of pixels to do that in
parallel. It’s quite another thing to say,
“I’m going to parallelize and thus speed
up the way that Photoshop runs JavaScript actions.” For example, I’ve got a
JavaScript routine that opens 50 files
one after the other and then performs a
set of 50 steps on each one. I don’t have
control over that script. My job is just to
make that—or any other script the user
has to run—faster.
I could say, “Rewrite all your scripts
so we can design a completely new interface that will let you specify that all
these images are to be processed in parallel.” That’s one answer, but it would
require a lot of work on the user’s part,
as well as on our part. And it would still
leave us with the problems associated
with parallelizing the opening of an
image, parsing the image contents, interpreting the JavaScript, running the
key command object through the application framework, and updating the
user interface—all of which typically is
tied into an app framework and thus
involves calling some horrendously sequential script interpreter. Once you
start looking at the Amdahl’s Law numbers on something like that, it soon becomes apparent that trying to get that
to parallelize eight ways would just be
completely hopeless.
At the other end of the spectrum you
might find, for example, a mathematical algorithm that conceptually lends
itself to parallelism simply because it
has a bunch of data structures it needs
to share. So how hard would it be to correctly implement that mathematically
parallelizable algorithm?
I think we’ve made some incremental gains in our ability to deal with parallelism over the past 20 years, just as
we’ve made stepwise progress on all
other programming fronts. Remember
that back in the 1970s, there was a lot of
talk about the “software crisis,” regarding how software was getting more and
more complicated, to the point where
we couldn’t manage the bugs anymore.
Well, in response to that, there was no
huge breakthrough in software produc-