CSC2515 Fall 2006 - Lectures

Tentative Lecture Schedule

  • Sept 12 -- Machine Learning:
    Introduction to Machine Learning, Generalization and Capacity (notes [ps.gz] [pdf])

  • Sept 19 --Classification 1:
    KNN, linear discriminants, decision trees (notes [ps.gz] [pdf])

  • Sept 19 -- TUTORIAL (Prob/Stats/Linear Algebra Review/Questions)

  • Sept 25 -- Classification 2:
    probabilistic classifiers: class-conditional Gaussians, naive Bayes, logistic regression, neural nets for classification (notes [ps.gz] [pdf])

  • Oct 3: Assignment 1 (Classification) posted

  • Oct 3 -- Regression 1:
    constant model, linear models, generalized additive models (e.g. RBFs), locally weighted regression, multilayer perceptrons/neural networks (notes [ps.gz] [pdf])

  • Oct 10 -- Objective Functions and Optimization:
    error surfaces, weight space, gradient descent, stochastic gradient, conjugate gradients, second order methods, convexity, enforcing constraints (notes [ps.gz] [pdf])

  • Oct 10 -- TUTORIAL (A1 Questions)
  • Oct17: Assignment 1 due at the start of class

  • Oct 17 -- Regression 2 and Supervised Mixtures:
    credit assignment problem, neural networks, radial basis networks, kolmogorov's theorem, backprop algorithm for efficiently computing gradients, mixtures of experts, piecewise models (notes [ps.gz] [pdf])

  • Oct24: Assignment 2 (Regression) posted

  • Oct 24 -- Unsupervised Learning 1: Trees & Clustering
    K-means, heirarchical clustering (alglomerative and divisive), maximum likelihood trees, optimal tree structure (notes [ps.gz] [pdf])

  • Oct 31 -- Unsupervised Learning 2: Mixture models and the EM Algorithm:
    missing data, hidden variables, Jensen's inequality, lower bound on marginal likelihood, free energy interpretation, inference, (notes [ps.gz] [pdf])

  • Oct 31 -- TUTORIAL (A2 Questions)

  • Nov 7: Assignment 2 due

  • November 7 -- Unsupervised Learning 3:
    Continuous latent variable models, Factor Analysis, (Probabilistic) PCA, Mixtures of Factor Analyzers, Independent Components Analysis (notes [ps.gz] [pdf])

  • Nov 14: Assignment 3 posted

  • Nov 14 -- Time Series Models
    autoregressive/Markov models, aggregate Markove models, hidden Markov models, profile HMMs (notes [ps.gz] [pdf])

  • Nov 21 -- Capacity Control:
    generalization and overfitting, No free lunch theorems, high dimensional issues.
    capacity control methods: weight decay, early stopping, cross validation, model averaging, intro to Bayesianism (notes [ps.gz] [pdf])

  • Nov 21 -- TUTORIAL (A3 Questions)

  • Nov 28: Assignment 3 due at the start of class

  • Nov 28 -- Meta-Learning Methods:
    stacking, bagging, boosting (notes [ps.gz] [pdf])

  • Nov 28: Project Meetings during tutorial time after class

  • Dec 1 12:00-2:00pm, Bahen 1220 (NOTE UNUSUAL DATE/TIME/LOCATION) -- Kernel methods:
    the kernel trick, support vector machines, kernel perceptrons, sparsity, capacity control, dual problems (notes [ps.gz] [pdf])

  • Dec 5 -- NO CLASS (rescheduled to Dec1)

  • December 15 -- projects due by email before noon
    Send attachments or valid URL pointing to your report.
    POSTSCRIPT or PDF only. DO NOT SUBMIT WORD, HTML OR OTHER FORMAT FILES.

  • December 15 -- all readings must be completed by noon.
    Online Reading Submission

  • Extra topics we may or may not have time for


[ Home | Course Information | Lecture Schedule/Notes | Textbook/Readings | Assignments/Project | Computing | ]

CSC2515 - Machine Learning || www.cs.toronto.edu/~roweis/csc2515/