Programming

using a combination of static analysis and runtime monitoring. There has been a recent resurgence of interest in this problem with the arrival of multi-core processors.

In the mid 1990s, as distributed memory machines became the high-performance parallel systems of choice, my research focus shifted to developing optimizing compiler technology for the High Performance Fortran programming model. To this day, this work continues to play a strong influence in my group’s current research portfolio, which includes a project to refine the Coarray Fortran language (a partitioned global address space language) to turn it into a viable programming model for scalable parallel systems and the HPC Toolkit project to develop performance tools for parallel systems.

As far as teaching parallel computing, I’ve only been doing that since 2006, shortly after I transitioned from a research faculty position at Rice to a teaching faculty position. Teaching parallel computing was a natural choice since my research focus for my entire career has been on compilers and tools for parallel systems.

MAURICE HERLIHY: Briefly, concurrent data structures and algorithms are my main area of research. For a long time, parallel computing was a kind of nice area, respected but not taken seriously, of interest mostly to scientific programmers and people like me who thought that parallelism would someday become important.

The advent of multi-core means, to my astonishment, that parallelism has indeed become important. The curriculum is slow to reflect this change, mostly because curriculums change slowly. It takes a lot of work to change them.

William Gropp
“[Computational]
performance isn’t
something, by itself,
that many students
are interested in. It
needs to be related to
something that they
want to do, and that
without parallelism,
they won’t be able
to accomplish what
they want to do.”

XRDS: As Maurice Herlihy said, a lot of recent interest in parallel programming in the industry and research communities has been driven by the advent of multi-core. Is this also a good motivation for students? More broadly, how might you motivate a computer science undergraduate to look into parallel programming?

WG: Parallelism is widely touted as the next big problem, and there certainly is a lot of interest from students in the technologies, particularly for both multi-core and GPUs. But the primary reason to consider the technology of parallelism is for performance, and I’ve found that performance isn’t something, by itself, that many students are interested in (there are some, of course, who are attracted to the challenge). It needs to be related to something that they want to do, and that without parallelism, they won’t be able to accomplish what they want to do.

A problem is that the CS curriculum rarely includes any significant attention to performance modeling applied to application codes, so there often isn’t a basis from which to start. Even numerical analysis (which we require of our CS undergraduates) tends to emphasize floating-point counts rather than memory motion, and so the performance models have little predictive power, even as a rough estimate.

In any event, for most, parallelism is a means to an end, not the end in itself. On the up side, if you are willing to consider parallelism for more than one chip, you can get three to five orders of magnitude of greater performance on systems available now.

JM-C: With the arrival of multi-core processors, interest in parallel computing has increased significantly. These days, almost every student has a multi-core laptop. Undergraduates are becoming aware that adding parallelism to software is the key to improving code performance for future generations of chips. Those interested in software want to make sure that they don’t get left out. I think that making sure that students are aware of the current software crisis is the best way of encouraging them to invest some time studying parallel computing.