main obscured within Photoshop’s synchronization code for 10 years serves to illustrate just how tricky parallel programming can be. But it also highlights how much progress has been made in terms of improved resources for managing some of this complexity. Had Photoshop’s synchronization been written today using C++’s stack-based locking, for example, it’s unlikely a bug of that sort would have been introduced. As processors get more cores and grow in complexity, the need will only intensify for new tools and better programming primitives for hiding the complexity from developers and allowing them to code at higher levels of abstraction.

At the same time, software architects also need to keep an eye on some other fundamental issues. For example, despite using less-sophisticated synchronization primitives in the original design, the Photoshop team was able to essentially forget about complex thread-synchronization problems, in part because the image-processing problem itself was so amenable to parallelization. Also, however, Photoshop’s architecture made it possible to establish some very clean object boundaries, which in turn made it easy for programmers to slice up objects and spread the resulting pieces across multiple processors. Indeed, the architects of Photoshop were keenly aware of where their best opportunities for parallelization existed, and they designed the application accordingly.

Generalizing on this, it’s clear that— with or without advances in tools and programming abstractions—in order for developers to fully leverage the multicore architectures that are coming, they’ll need to be adept at identifying those parts of a program that can benefit most from parallelization. It’s in these portions of the code that new tools, techniques, and parallel programming abstractions are likely to have the greatest impact.

 

coLe: As operating-system designers, we both grew up in a world where we had to deal with parallelism. It’s not always clear that the solutions we came up with for our operating systems proved to be the right ones. In an earlier conversation, you mentioned your experience with creating and removing mutexes. We’ve gotten smarter over the years.

RuSSeLL WiLLiamS
i think we’ve made
some incremental
gains in our ability
to deal with
parallelism over
the past 20 years,
just as we’ve
made stepwise
progress on all
other programming
fronts.

We’ve learned how to do things that are more efficient, but that doesn’t mean it has gotten any easier. What do we have up our sleeves to make it easier?

WiLLiamS: Parallelism remains difficult in a couple of ways. It’s one thing to ask for a new Photoshop filter for processing a grid of pixels to do that in parallel. It’s quite another thing to say, “I’m going to parallelize and thus speed up the way that Photoshop runs JavaScript actions.” For example, I’ve got a JavaScript routine that opens 50 files one after the other and then performs a set of 50 steps on each one. I don’t have control over that script. My job is just to make that—or any other script the user has to run—faster.

I could say, “Rewrite all your scripts so we can design a completely new interface that will let you specify that all these images are to be processed in parallel.” That’s one answer, but it would require a lot of work on the user’s part, as well as on our part. And it would still leave us with the problems associated with parallelizing the opening of an image, parsing the image contents, interpreting the JavaScript, running the key command object through the application framework, and updating the user interface—all of which typically is tied into an app framework and thus involves calling some horrendously sequential script interpreter. Once you start looking at the Amdahl’s Law numbers on something like that, it soon becomes apparent that trying to get that to parallelize eight ways would just be completely hopeless.

At the other end of the spectrum you might find, for example, a mathematical algorithm that conceptually lends itself to parallelism simply because it has a bunch of data structures it needs to share. So how hard would it be to correctly implement that mathematically parallelizable algorithm?

I think we’ve made some incremental gains in our ability to deal with parallelism over the past 20 years, just as we’ve made stepwise progress on all other programming fronts. Remember that back in the 1970s, there was a lot of talk about the “software crisis,” regarding how software was getting more and more complicated, to the point where we couldn’t manage the bugs anymore. Well, in response to that, there was no huge breakthrough in software produc-