HPC References

Books

• The Sourcebook of Parallel Computing (Dongarra et. al.)
  Discusses high level ideas in parallelism, overviews of several applications, software technology, and enabling technologies (mostly numerical libraries). There appears to be little discussion of MPI, OpenMP, or the details of parallel programming. With seven editors, it might not be too surprising that this book doesn't have a uniform feel. This book served at least once as a recommended text for Berkeley's CS 267, and was also one of the required texts in the 2006 course at Columbia by David Keyes.
• Introduction to Parallel Computing (Grama, Karypis, Kumar, and Gupta)
  Some ideas of parallel program design and modeling, followed by sections on parallel programming (MPI and OpenMP), non-numerical algorithms, and numerical methods. I like the book, but the emphasis is not on scientific computing; the chapters on numerical methods are relatively short and few, and the discussion does not seem to be particularly application driven.
• Parallel Scientific Computing in C++ (Karniadakis and Kirby)
  This is a book about numerical methods, with parallel computing using MPI and OpenMP as a secondary theme.
• Scientific Parallel Computing (Scott, Clark, and Bagheri)
  Required text for the parallel computing course at Columbia in 2006.
• Parallel Programming in C with MPI and OpenMP (Quinn)
  Required text for the parallel computing course at Penn State.
• Parallel Programming (Wilkinson and Allen)
  See also the authors' website for the book. Required text for the parallel computing courses at UCSB and Utah.
• Designing and Building Parallel Programs (Foster)
  Available for free online or from Addison Wesley. It is what it says: a book about parallel program design, with relatively little time given to numerical methods.
• MPI: The Complete Reference
  First edition is available online.
• Using OpenMP: Portable Shared Memory Parallel Programming (Chapman, Jost, and van der Pas)
  Lanugage and performance issues, along with a few case studies.

Course pages

• MIT 18.337: Parallel Scientific Computing
• Berkeley CS267: Applications of Parallel Computers
• Stanford CS238: Parallel Methods in Numerical Analysis (inaccessible?)
• UIUC CS554: Parallel Numerical Algorithms
• Columbia AMCSE4302: Introduction to Parallel Scientific Computing (syllabus only)
• UCSB CS140: Parallel Scientific Computing
• Utah CS6230: High Performance Computing
• Berkeley CS294-2: Parallel Programming for Multicore

Tutorials and other references

• Code optimization and performance tuning (lecture notes from CSC in Finland)
• HPC training materials from LLNL
• HPC tutorials from NERSC
• DOE ACTS (Advanced CompuTational Software) Collection
• MPI tutorial material list from Argonne

CUDA references

• UIUC slides from lecture
• very nice slides with simple examples
• CUDA Programming from Nvidia
  You can find the CUDA 2.0 Programming Guide; also look at Cuda U -> Education.