
This page contains the schedule, slide from the lectures, lecture notes, reading lists,
assigments, and web links.
I urge you to download the DjVu viewer
and view the DjVu version of the documents below. They display faster,
are higher quality, and have generally smaller file sizes than the PS and PDF.
Fulltext search is provided for the entire
collection of slides and papers. Click here to search
You can have a look at the schedule
and class material for the version of this course taught during the
Fall 2005 semester, but be warned that the new edition is
significantly different.
WARNING: The schedule below is almost certainly going to change.
09/05: Introduction and basic concepts 
Subjects treated: Intro, types of learning, nearest neighbor, how biology does it,
linear classifier, perceptron learning procedure, linear regression,
Slides: [DjVu  PDF]
Recommended Reading:
 Bishop, Chapter 1.
 Refresher on random variables and probabilites by
Andrew Moore: (slides 127) [DjVu  PDF]
 Refresher on joint probabilities, Bayes theorem by
Chris Willams: [DjVu  PDF]
 Refresher on statistics and probabilities by
Sam Roweis: [DjVu  PS]
 If you are interested in the early history of selforganizing
systems and cybernetics, have a look at this book available from the
Internet Archive's Million Book Project: SelfOrganizing
Systems, proceedings of a 1959 conference edited by Yovits and
Cameron (DjVu viewer required for full text).
09/12: EnergyBased Models, Loss Functions, Linear Machines 
Subjects treated: Energybased learning, minimumenergy
inference, loss functions.
Linear machines: least square, perceptron, logistic regression.
Slides: [DjVu  PDF]
Recommended Reading:
09/19: Basis Functions, Kernel Trick, Regularization, Generalization 
Subjects treated: Energybased models, minimumenergy
machines, loss functions. Linear machines: perceptron, logistic
regression. Linearly parameterized classifiers: Polynomial
classifiers, basis function expansion, RBFs, Kernelbased expansion.
Slides Basis Functions, Kernel Trick: [DjVu  PDF]
Slides Regularization, Generalization: [DjVu  PDF]
Homework Assignements: Linear Classifier: implementing the Perceptron
Algorithm, MSE Classifier (linear regression), Logistic Regression.
Details and datasets below:
 Download this tar.gz archive. It
contains the datasets and the homework description.
 Decompress it with "tar xvfz hwlinear.tgz" on Unix/Linux or
with Winzip in Windows.
 The file README.txt contains the questions and instructions.
 Most the of the necessary Lush code is provided.
 Due Date is Tuesday October 03th, before the lecture.
09/26: GradientBased Learning I, MultiModule Architectures and BackPropagation, Regularization 
Subjects treated: MultiModule learning machines. Vector
modules and switches. Multilayer neural nets. Backpropagation
Learning.
Slides MultiModule Learning machines, backprop: [DjVu  PDF]
Slides Special modules: [DjVu  PDF]
Recommended Reading: Bishop, Chapter 5.
10/03: GradientBased Learning II: Special Modules and Architectures 
10/10: Convolutional Nets, Image Recognition 
Required Reading:
If you haven't read it already: Gradientbased Learning Applied to
Document Recognition by LeCun, Bottou, Bengio, and Haffner; pages 1 to
the first column of page 18:
[ DjVu  .pdf ]
Optional Reading: FuJie Huang, Yann LeCun, Leon Bottou: "Learning Methods for Generic Object
Recognition with Invariance to Pose and Lighting.", Proc. CVPR 2004.
[DJVU,
PDF,
.PS.GZ,
Homework Assignements: Neural Nets and Backpropagation:
Click on this links to get the homework: hwbackprop.tgz.
Due Date is Tuesday October 31th, before the lecture.
10/17: Efficient Optimization 
Required Reading:
Efficient Backprop, by LeCun, Bottou, Orr, and Muller:
[ DjVu  .pdf ]
10/24: Probabilistic Learning, MLE, MAP, Bayesian Learning 
Slides Review of Probability and Statistics: [DjVu  PDF]
Slides Bayesian Learning: [DjVu  PDF]
10/31: Intro to unsupervised Learning 
Slides Unsupervised Learning, PCA, KMeans: [DjVu  PDF]
11/07: More on unsupervised Learning, Latent Variables, EM 
Slides Latent variables: [DjVu  PDF]
Slides EM, Mixture of Gaussians: [DjVu  PDF]
11/14: Learning Theory, Bagging, Boosting, VCDim 
Slides Latent variables: [DjVu  PDF]
Homework Assignements: KMeans and Mixture of Gaussians:
Click on this links to get the homework: hwbackprop.tgz.
Due Date is Tuesday November 28, before the lecture.
11/21: Intro to Graphical Models 
Suggested Reading:
11/28: Structured Outputs: HMMs, Graph Transformer Networks 
Suggested Reading:
The projects descriptions are here
Due date is December 30th (or December 23rd if you are graduating this semester).
12/05: Kernel Methods and Support Vector Machines 
Lecture given by Sumit Chopra and Fu Jie Huang.

