Foundations of Machine Learning
Instructor: Mehryar Mohri
Graders/TAs: Umar Syed and Afshin Rostami
This course introduces the fundamental concepts and methods of machine
learning, including the description and analysis of several modern
algorithms, their theoretical basis, and the illustration of their
applications. Many of the algorithms described have been successfully
used in text and speech processing, bioinformatics, and other areas in
real-world products and services. The main topics covered are:
It is strongly recommended to those who can to also attend
Machine Learning Seminar.
- Probability and general bounds
- PAC model
- Perceptron, Winnow
- Support vector machines (SVMs)
- Kernel methods
- Decision trees
- Regression problems and algorithms
- Ranking problems and algorithms
- Halving algorithm, weighted majority algorithm, mistake bounds
- Learning automata and transducers
- Reinforcement learning, Markov decision processes (MDPs)
Location and Time
Room 102 Warren Weaver Hall,
251 Mercer Street.
Mondays 5:00 PM - 6:50 PM.
Familiarity with basics in linear algebra, probability, and analysis
Projects and Assignments
There will be 3 to 4 assignments and a project. The final grade is
essentially the average of the assignment and project grades. The
standard high level of integrity
is expected from all students, as with all CS courses.
The following textbook covers all the material
presented in this course (and a lot more):
Parts of that material might be made available to the students at the
time of the lectures.
- Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar.
Foundations of Machine Learning.
MIT Press, 2012 (to appear).
Here is also a list of other books recommended for further reading:
- Martin Anthony and Peter Bartlett.
Neural Network Learning: Theoretical Foundations.
Cambridge University Press, 1999.
- Nicolo Cesa-Bianchi and Gabor Lugosi. Prediction, learning, and games. Cambridge University Press, 2006.
- Luc Devroye, Laszlo Gyorfi, Gabor Lugosi.
A Probabilistic Theory of Pattern Recognition.
- Michael J. Kearns and Umesh V. Vazirani.
An Introduction to Computational Learning Theory.
MIT Press, 1994.
- Bernhard Schoelkopf and Alex Smola.
Learning with Kernels.
MIT Press, Cambridge, MA, 2002.
- Vladimir N. Vapnik.
Estimation of Dependences Based on Empirical Data.
Springer NY (new edition), 2006.
- Vladimir N. Vapnik.
Statistical Learning Theory.
An extensive list of recommended papers for further reading is
provided in the lecture slides.