Numerical Computing, CSCI-UA.0421

Instructor: Denis Zorin, dzorin@cs.nyu.edu
Time/room: Mon, Wed 11:00 - 12:15 a.m., rm. 1221, 719 Broadway
Office Hours: 715 Broadway, rm. 1201, Mon, Wed 12:15-1:15, or by appointment
Questions/discussion: Piazza site
Final: Monday, May 19, 10:00-11:50AM, rm. 1221, 719 Broadway

Course Description

This class covers basic topics in numerical computation: floating-point arithmetics, types of error, methods for solving linear and nonlinear systems, interpolation and basic discretizations of ordinary differential equations. Numerical algorithms will be presented in the context of a variety examples, primarily drawn from interactive physics and animation techniques used in games and computer graphics.

Prerequisites

CS 102 (intro to computer science II) and Math 140 (linear algebra). Knowledge of Python is not expected, but helps.

Requirements

  1. Written and programming homeworks;
  2. project;
  3. final exam.

Textbook

Software

We will use a Python-based numerical computing environment SciPy, which provides most of the functionality found in Matlab, in particular efficient vector and matrix manipulation through NumPy, but based on a general-purpose high-level language, and seamlessly integrated with a large number of libraries provided by Python.

Installation. Please use Enthought Canopy Python distribution . It contains most of what you may need for the class. If you have problems installing and running it, please contact the instructor.

Assignments

Late penalty: 15% if submitted the week it is due, after the deadline (no later than the end of the day Sunday); beyond this, 10% per additional day late. Homework accepted until graded homework that was turned in on time is returned.

Assignment 0, due Wednesday, January 29. Install the Python distribution. Change student to your last name in this file version_info.py and run this program. It should create a png file yourname_version_info.png; please send it to dzorin@cs.nyu.edu

Assignment 1, due Wednesday, February 19. PDF If you typeset your assignment, you can send it to the instructor by e-mail. If parts of it are hand-written, please submit the whole assignment on paper. Derivative error code.

Assignment 2, due Monday, March 24. PDF LU code.

Assignment 3, due Monday, April 7. PDF

Assignment 4, due Monday, May 12. PDF (for those taking the homework option)

Assignment 5, due Monday, May 21. PDF (for those taking the homework option)

Project, report due May 21. You can use the ideas from the sources listed below, or propose your own. Timeline:
April 16: send an e-mail to me with one-two sentences describing the project topic to make sure it is acceptable.
April 21: 1-2 page project proposal describing the proposed problem in more detail and how you plan to implement your solution.
May 21: Report due. You will need to schedule a 10 min. appointment to discuss the results with the me. Projects ideas: simulation and image processing
Project ideas from 2012
Scientific Computing with Case Studies

Final questions.

Lectures

January 27Overview
January 29Python and numpy review. Tutorials:
General python, sections 3-4 and 5.3 (tuples); the question about main is discussed in 6.11
numpy sections 2-4
A summary on numpy arrays
February 3Types of errors. Book: Chapter 1.
February 5IEEE 754 floating-point. Book: Chapter 2, additional material: A brief overview of the standard by Steve Hollasch.
Minifloats are useful for understanding the details of f.p. operations.
Lecture notes
by. W. Kahan, the architect of IEEE 754.
A script to extract sign, mantissa and exponent from a single-precision float as strings in binary format.
February 10 Linear algebra review, data fitting example (similar to Ch.4, example 4.12). Linear systems with one, infinitly many or zero solutions.
February 12 Linear alegbra review continued. Algebraic matrix operations, matrix inversion, singular and nearly singular matrices (Ch. 4) Matrices in Python: arrays, operations on arrays A summary on numpy arrays What's under the hood in numpy: BLAS Reference
February 19 Linear algebra review, last part. Eigenvalues, eigenvectors. Vector norms, matrix norms. Review for quiz: questions on page 102.
February 24 Solving nonlinear equations in one variable: Bisection, Newton's method, secant method, fixed point iterations. (Ch. 3). bisection sample code Newton sample code
February 26 Linear algebra quiz. Direct methods for solving linear systems. LU factorization.
March 3 Linear algebra quiz. LU factorization continued. Sparse systems: 1D differential equation example. Banded systems. Sparse system storage. (Ch. 5.)
March 5 Solving sparse systems in scipy. 2D sparse system example -- Laplacian. (Ch. 5.)
March 10 2D sparse system example continued; 2D Laplacian code.
March 12 Iterative methods for linear systems: general form, Jacobi, Gauss-Seidel. (Ch. 7.)
March 24 Iterative methods for linear systems review. Newton's method for n-dimensional problems. (Ch. 9.)
March 26 Newton's method for n-dimensional problems: theory. Gradient descent. (Ch. 9.)
March 31 Minimization of functions of many variables, making Newton's method robust. Line search, descent directions.
April 2 OPtimization for n-dimensional problems in SciPy: choice of methods, interface SciPy minimize
April 7 Projects -- overview of project ideas. Projects ideas: simulation and image processing
Project ideas from 2012
Scientific Computing with Case Studies
April 9 Time-stepping methods: Euler, backward Euler in one dimension. (Chapter 17)
April 14 Time-stepping methods: n-dimensional systems of ODE. High-order methods.
April 16 Time-stepping continued: integration and time stepping, Runge-Kutta methods, leapfrog.
April 21 Project help.
April 23 Computing Eigenvalues (Section 8.1), power method, shift-invert.
April 28 Singular Value Decomposition, basic properties (Section 8.2)
April 30 Lagrange interpolation, Bezier curves (Chapter 11, K. Bakeer's notes
May 5 Discrete Fourier Transforms, FFT (Sections 14.2-14.3, wikipedia
May 7 Project help.
May 12 Review for the final. Final questions.