Course topics
We will spend the first eight or nine weeks of the class on general ideas in high performance parallel program design, with examples drawn from scientific computing. We will loosely follow the book Parallel Programming by Wilkinson and Allen. Topics include:
- Single-processor performance, memory hierarchy, and pipelines (Chap 5).
- Overview of parallel system organization (Chap 1).
- Introduction to message passing and MPI programming (Chap 2).
- Embarrassingly parallel problems (Chap 3).
- Problem decomposition, graph partitioning, and load balancing (Chap 4-7).
- Introduction to shared memory and OpenMP programming (Chap 8-9).
For the rest of the class, we will discuss more examples from scientific computing and additional programming topics. The exact selection will depend in part on student background and interests, but may include:
- Programming with toolkits (PETSc, Trilinos, etc).
- Parallel programming in UPC.
- Iterative solution of sparse linear systems.
- Fast Fourier transforms.
- Multigrid methods.
- Adaptive mesh refinement.
Assignments
There will be three programming assignments and a final project, respectively emphasizing serial performance, MPI, OpenMP, and an application. If you work with simulations in your own research, you may propose a project based on those simulation tools. The final projects may be done individually or in small groups.