E0 270: Machine Learning

January - April 2013

Department of Computer Science & Automation
Indian Institute of Science



[Announcements]  [Course Description]  [Assignments]  [Academic Honesty]  [Tutorials/Discussions]  [Schedule] 

Course Information

Class Meetings

Lectures: Tue-Thu 11:30am-1:00pm, CSA 252 (Multimedia Classroom) CLH L1 CSA 252 (Multimedia Classroom)
First lecture: Tue Jan 8

Tutorial/discussion sessions will be scheduled on an on-going basis.

Instructors

Prof. Shivani Agarwal (shivani@csa)
Prof. Chiranjib Bhattacharyya (chiru@csa)

TAs

Harikrishna Narasimhan (harikrishna@csa)
Adway Mitra (adway@csa)

Course Registration

If you wish to take this course for credit, please fill in the registration form here.

Mailing list

http://groups.google.com/group/iisc-e0-270-machine-learning-jan-apr-2013


Announcements


Course Description

With the increasing amounts of data being generated in diverse fields such as astronomical sciences, health and life sciences, financial and economic modeling, climate modeling, market analysis, and even defense, there is an increasing need for computational methods that can automatically analyze and learn predictive models from such data. Machine learning, the study of computer systems and algorithms that automatically improve performance by learning from data, provides such methods; indeed, machine learning techniques are already being used with success in a variety of domains, for example in computer vision to develop face recognition systems, in information retrieval to improve search results, in computational biology to discover new genes, and in drug discovery to prioritize chemical structures for screening. This course aims to provide a sound introduction to both the theory and practice of machine learning, with the goal of giving students a strong foundation in the subject, enabling them to apply machine learning techniques to real problems, and preparing them for advanced coursework/research in machine learning and related fields.

Syllabus (Tentative)

Introduction to machine learning. Supervised learning: classification (generative and discriminative probabilistic models, support vector machines, nearest neighbor, decision trees, VC-dimension, generalization error bounds); regression (least squares regression, support vector regression). Unsupervised learning: mixture models, EM algorithm. Directed and undirected graphical models (hidden Markov models, Bayesian networks, Markov random fields, conditional random fields). Additional learning settings: online learning (perceptron, winnow); semi-supervised and active learning; reinforcement learning. Gaussian processes. Ensemble learning (boosting). Advanced lectures on selected topics.

References

Recommended textbooks: Additional textbooks:

Prerequisites

Anyone taking the course for credit must have taken E0 232: Probability and Statistics (or equivalent course elsewhere) and earned a grade of B or higher. In addition, some background in linear algebra and optimization will be helpful.

Grading


Assignments

Assignment Policy

The following assignment policy will be strictly followed: * Except in the case of a documented medical/personal emergency, which must be supported by a medical certificate or signed letter submitted to the instructors and to the CSA office.

Academic Honesty

As students of IISc, we expect you to adhere to the highest standards of academic honesty and integrity.

Assignments in the course are designed to support your learning of the subject. Copying will not help you (in the exams or in the real world), so don't do it. If you have difficulties learning some of the topics or lack some background, try to form study groups where you can bounce off ideas with one another and try to teach each other what you understand. You're also welcome to talk to any of us and we'll be glad to help you.

If any assignment/exam is found to be copied, it will automatically result in a zero grade for that assignment/exam and a warning note to your advisor. Any repeat instance will automatically lead to a failing grade in the course.


Tutorials/Discussions

  1. Tutorial: Optimization I. Fri Jan 11, 5:30-7pm, CSA 252. [HN]
  2. Tutorial: Optimization II. Fri Jan 18, 5:30-7pm, CSA 252. [HN]
  3. Tutorial: Optimization III. Sat Jan 19, 10:30am-12:00, CSA 254. [HN]

Tentative Schedule

 
  Date Topic Lecture Notes Additional Recommended Reading Notes
1 Tue Jan 8 Introduction and course overview
Binary classification and Bayes error
[1.pdf] Bishop Ch 1  
2 Thu Jan 10 Generative probabilistic models for classification [2.pdf] Bishop Ch 4  
3 Tue Jan 15 Discriminative probabilistic models for classification [3.pdf] Bishop Ch 4  
4 Thu Jan 17 Least squares regression [4.pdf] Bishop Ch 3  
5 Tue Jan 22 Support vector machines for classification and regression [5.pdf] Bishop 7.1  
6 Thu Jan 24 Kernel methods - Bishop Ch 6  
7 Tue Jan 29 Decision trees and nearest neighbor methods - Bishop 14.4, 2.5.2 Assignment 1 due
8 Thu Jan 31 Model selection, generalization error bounds, and VC-dimension - Bishop 1.3
E0 370 Lecture 3 notes, Sections 1-2
E0 370 Lecture 4 notes, Section 2, Corollary 3.3
Bishop 3.4
 
9 Tue Feb 5 Principal component analysis (PCA). - Bishop Ch 12  
10 Thu Feb 7 Kernel PCA and canonical correlation analysis (CCA) [CCA] Further reading  
11 Tue Feb 12 Probabilistic PCA - Bishop Ch 12
PPCA
Project preferences/proposals due
12 Thu Feb 14 EM algorithm and mixture distribution - Bishop Ch 9  
13 Tue Feb 19 Introduction to HMM - Bishop Ch 13  
14 Thu Feb 21 Introduction to HMM - Bishop Ch 13  
  Tue Feb 26 Midterm (in class)      
15 Thu Feb 28 Introduction to Bayesian networks (BNs) - Bishop 8.1, 8.2  
  Tue Mar 5 Project formulation presentations (in class)      
16 Thu Mar 7 Belief propagation on singly connected BNs - A Tutorial on Belief Propagation
Original papers:
[Pearl, 82]
[Kim & Pearl, 83]
Project formulation reports due
17 Tue Mar 12 Markov networks and variational methods - Bishop 8.3
A Tutorial Introduction to Variational Methods
 
18 Thu Mar 14 Metropolis-Hastings and Gibbs sampling - Bishop 11.2  
19 Tue Mar 19 Online learning - E0 370 Lecture 18 notes  
20 Thu Mar 21 Semi-supervised and active learning - Semi-supervised learning literature survey
Nigam et al, 2000

Active learning literature survey
Cesa-Bianchi et al, 2004
 
21 Tue Mar 26 Reinforcement learning
Guest lecture by Dr. Shivaram Kalyanakrishnan
[slides]    
22 Thu Mar 28 Boosting and ensemble methods - E0370 lecture 15 notes
Section 3
Bishop 14.3
Project milestone reports due
23 Tue Apr 2 Gaussian processes for regression and classification - Bishop 6.4  
24 Thu Apr 4 Statistical consistency of binary classification algorithms based on risk minimization [24.pdf]    
  Tue Apr 9 Project presentations (in class)      
  Thu Apr 11 Project presentations (in class)      
  Tue Apr 16 Final project reports due     Final project reports due
  Mon Apr 29 Final exam (9 AM - 12 noon, CLH L1)