` E0 270: Machine Learning, January Term 2015

E0 270: Machine Learning

January - April 2015

Department of Computer Science & Automation
Indian Institute of Science



[Announcements]  [Course Description]  [Assignments]  [Academic Honesty]  [Tutorials/Discussions]  [Schedule] 

Course Information

Class Meetings

Lectures: Tue-Thu 11:00am-12:30pm, CSA 252 (Multimedia Classroom) CLH L1
First lecture: Tue Jan 13

Tutorial/discussion sessions will be scheduled on an on-going basis.

Instructors

Prof. Shivani Agarwal (shivani@csa)
Prof. Chiranjib Bhattacharyya (chiru@csa)

TAs

Harikrishna Narasimhan (harikrishna@csa)
Lavanya Sita Tekumalla (lavanya@csa)
Rohit Vaish (rohit.vaish@csa)

TA Hours

Harikrishna Narasimhan: Fri, 6:00-7:00pm, CSA 230
Rohit Vaish: Wed, 6:00-7:00pm, CSA 230
Lavanya Tekumalla: Wed, 5:00-6:00pm, CSA 251 (MLLab)

Course Registration

If you wish to take this course for credit, please fill in the registration form here.

Mailing list

http://groups.google.com/group/iisc-e0-270-machine-learning-jan-apr-2015


Announcements


Course Description

With the increasing amounts of data being generated in diverse fields such as astronomical sciences, health and life sciences, financial and economic modeling, climate modeling, market analysis, and even defense, there is an increasing need for computational methods that can automatically analyze and learn predictive models from such data. Machine learning, the study of computer systems and algorithms that automatically improve performance by learning from data, provides such methods; indeed, machine learning techniques are already being used with success in a variety of domains, for example in computer vision to develop face recognition systems, in information retrieval to improve search results, in computational biology to discover new genes, and in drug discovery to prioritize chemical structures for screening. This course aims to provide a sound introduction to both the theory and practice of machine learning, with the goal of giving students a strong foundation in the subject, enabling them to apply machine learning techniques to real problems, and preparing them for advanced coursework/research in machine learning and related fields.

Syllabus (Tentative)

Introduction to machine learning. Supervised learning: classification (generative and discriminative probabilistic models, support vector machines, nearest neighbor, decision trees, VC-dimension, generalization error bounds); regression (least squares regression, support vector regression). Unsupervised learning: mixture models, EM algorithm. Directed and undirected graphical models (hidden Markov models, Bayesian networks, Markov random fields, conditional random fields). Additional learning settings: online learning (perceptron, winnow); semi-supervised and active learning; reinforcement learning. Gaussian processes. Ensemble learning (boosting). Advanced lectures on selected topics.

References

Recommended textbooks: Additional textbooks:

Prerequisites

Anyone taking the course for credit must have taken E0 232: Probability and Statistics (or equivalent course elsewhere) and earned a grade of B or higher. In addition, some background in linear algebra and optimization will be helpful.

Grading


Assignments

Assignment Policy

The following assignment policy will be strictly followed: * Except in the case of a documented medical/personal emergency, which must be supported by a medical certificate or signed letter submitted to the instructors and to the CSA office.

Academic Honesty

As students of IISc, we expect you to adhere to the highest standards of academic honesty and integrity.

Assignments in the course are designed to support your learning of the subject. Copying will not help you (in the exams or in the real world), so don't do it. If you have difficulties learning some of the topics or lack some background, try to form study groups where you can bounce off ideas with one another and try to teach each other what you understand. You're also welcome to talk to any of us and we'll be glad to help you.

If any assignment/exam is found to be copied, it will automatically result in a zero grade for that assignment/exam and a warning note to your advisor. Any repeat instance will automatically lead to a failing grade in the course.


Tutorials/Discussions

  • Tutorial: Optimization I.
  • Fri Jan 16, 6:00pm - 7.30pm, CSA 117. [HN]   [lecture-notes]
  • Tutorial: Optimization II.
  • Sat Jan 17, 11:00am - 12:30pm, CSA 117. [HN]   [lecture-notes]
  • Tutorial: Optimization III.
  • Fri Jan 23, 6:00pm - 7.30pm, CSA 117. [HN]   [lecture-notes]






    Tentative Schedule

      Date Topic Lecture Notes Additional Recommended Reading Notes
    1 Tue Jan 13 Introduction and course overview
    Binary classification and Bayes error
    [1.pdf] Bishop Ch 1 Assignment 1 out
    2 Thu Jan 15 Generative probabilistic models for classification [2.pdf] Bishop Ch 4  
    3 Tue Jan 20 Discriminative probabilistic models for classification [3.pdf] Bishop Ch 4  
    4 Thu Jan 22 Least squares regression [4.pdf] Bishop Ch 3  
    5 Tue Jan 27 Support vector machines for classification and regression [5.pdf] Bishop 7.1 Assignment 1 due
    Assignment 2 out
    6 Thu Jan 29 Kernel methods [6.pdf] Bishop Ch 6  
    7 Tue Feb 3 Decision trees and nearest neighbor methods   Bishop 14.4, 2.5.2  
    8 Thu Feb 5 Model selection, generalization error bounds, and VC-dimension   Bishop 1.3
    E0 370 Lecture 3 notes
    (Sections 1-2)
    E0 370 Lecture 4 notes
    (Section 2, Corollary 3.3)
    Bishop 3.4
     
    9 Tue Feb 10 Principal component analysis (PCA)   Bishop Ch 12 Assignment 2 due
    Assignment 3 out, due on Tue Mar 10
    10 Thu Feb 12 Kernel PCA and canonical correlation analysis (CCA) [CCA] Further reading  
      Tue Feb 17 Insitute Holiday (Maha Shivaratri)      
      Thu Feb 19 No class     Project guidelines out
      Tue Feb 24 Midterm      
    11 Thu Feb 26 Introduction to latent variable models, EM algorithm   Bishop Ch 9  
    12 Tue Mar 3 Mixture of Gaussians, K-mean algorithm, Introduction to HMM   Bishop Ch 9, 13  
      Thu Mar 5 No class     Project proposals due
    13 Tue Mar 10 HMM (continued)   Bishop Ch 13 Assignment 3 due
    14 Thu Mar 12 Bayesian networks (BNs)   Bishop 8.1, 8.2 Assignment 4 out
    15 Mon Mar 16 Extra class (at 5 PM)
    Belief propagation in singly connected BNs
      Pearl Ch 4
    Loopy Belief propagation:
    [Murphy et al., 99]
    [McEliece et al., 98]
     
    16 Tue Mar 17 Markov networks   Bishop 8.3  
    17 Thu Mar 19 Variational methods and Boltzmann machines   [Saul and Jordan, 95]
    [Ackley et al., 85]
    Restricted Boltzmann Machines
     
    18 Tue Mar 24 Metropolis-Hastings algorithm   Sheldon Ross, "Introduction to Probability Models", Ch 4 Assignment 4 due
    19 Wed Mar 25 Extra class (at 5 PM, CSA 254)
    Gibbs sampling
      Bishop 11.3  
    20 Thu Mar 26 Bayesian linear regression and Gaussian process regression   Bishop 6.4  
    21 Tue Mar 31 Online learning   E0 370 Lecture 18 notes
     
    22 Thu Apr 2 Boosting   Section 3 of E0 370 Lecture 15 notes
    Bishop 14.3
    Assignment 5 out
    23 Tue Apr 7 Semi-supervised learning
    Guest lecture by Prof. Partha Talukdar
    Venue: CSA 252
    [slides]    
    24 Thu Apr 9 Machine learning in natural language processing
    Guest lecture by Prof. Partha Talukdar
    Venue: CSA 252
      An Introduction to Conditional Random Fields for Relational Learning  
      Tue Apr 14 Project reports due     Assignment 5 due
      Tue Apr 28 Final exam
    9am-12noon, CLH L1