COMP 5212: Machine Learning [Fall 2022]

Monday, Wednesday 12:00-13:20 @ Room 2503

Overview

Announcements

Description

In this course, we will cover some classical and advanced algorithms in machine learning. Topics include: Linear models (linear/logistic regression, support vector machines), Non-linear models (tree-based methods, kernel methods, neural networks), learning theory (hypothesis space, bias/variance tradeoffs, VC dimensions). The course will also discuss some advanced topics of machine learning such as testing-time integrity in trustworthy machine learning and neural architecture search in AutoML.

Prerequisites

Basic knowledge in numerical linear algebra, probability, and calculus.

Grading Policy

Late submission policy:

Late submissions are accepted up to 2 days after the due date, with 10% (of the total grade of the item) penalty per day.

Term projects

Students will work on a open-topic research project with groups. Each group could only be consisted with less or equal than 4 members (<=4). Feel free to discuss with me offline for the topic choice.

Tentative Schedule and Material

DateTopicSlidesReadings&linksAssignments
Mon 5/9Overview of Machine Learninglecture_0  
Wed 7/9Math Basicslecture_1Matrix Calculus:Derivation and Simple Application HU Pili, DL Chapter 2.1 & 2.2 &2.3 
Wed 14/9Linear modelslecture_2  
Mon 19/9Optimizationlecture_3Convex Optimization Boyd and Vandenberghe Chapter 3.1, Numerical Optimization Nocedal and Wright Chapter 3.1 
Wed 21/9Stochastic gradient descent and its variantslecture_4 Written_HW1 out
Mon 26/9Support Vector Machine, Polynomail nonlinear mapping, Kernel method,lecture_5Stanford CS 229 notes 
Wed 28/9Learning theorylecture_6 Programming_HW1 out
Mon 3/10Uniform convergence, growth functionlecture_7Symmetrization 
Wed 5/10VC Dimensionlecture_8  
Mon 10/10Regularizationlecture_9  
Wed 12/10Clusteringlecture_10  
Mon 17/10Tree-based methodslecture_11Xgboost 
Wed 19/10Neural networkslecture_12 Written_HW2 out
Mon 24/10Neural networks for computer visionlecture_13  
Wed 26/10Dropout, Batch Norm, ResNet, Neural networks for NLP: basiclecture_14 Programming_HW2 out
Mon 31/10Neural networks for NLP: Modellecture_15  
Wed 2/11Transformer & Unsupervised pertaining for NLPlecture_16  
Mon 7/11Vision Transformerlecture_17  
Wed 9/11Semi-supervised learning, graph convolution networklecture_18 HW3 out
Mon 14/11AutoML(Neural architecture search)lecture_19  
Wed 16/11Recent progress in Neural architecture searchlecture_20  
Mon 21/11Limitations of deep learning: Testing-time integritylecture_21  
Wed 23/11Limitations of deep learning: Training-time integrity & Reviewlecture_22  
Mon 28/11Final project presentation-part 1   
Wed 30/11Final project presentation-part 2   

References

There is no required textbook for this course. Some recommended readings are