COMP 5212: Machine Learning [Fall 2022]
Monday, Wednesday 12:00-13:20 @ Room 2503
Overview
- Instructor: Minhao Cheng (minhaocheng@cse.ust.hk)
- Office hours: Tuesday 13:00-14:30 @ Room 2542
- Teaching assistants:
- Mingxuan Fan’s (fanae@connect.ust.hk) office hours: Thursday 14:00-15:00 @ Room 3661
- Shuowei Hang’s (scaiak@cse.ust.hk) office hours: Friday 16:00-17:00 @ Room 3661
- Canvas: COMP5212
Announcements
Description
In this course, we will cover some classical and advanced algorithms in machine learning. Topics include: Linear models (linear/logistic regression, support vector machines), Non-linear models (tree-based methods, kernel methods, neural networks), learning theory (hypothesis space, bias/variance tradeoffs, VC dimensions). The course will also discuss some advanced topics of machine learning such as testing-time integrity in trustworthy machine learning and neural architecture search in AutoML.
Prerequisites
Basic knowledge in numerical linear algebra, probability, and calculus.
Grading Policy
- Homework (40%)
- 3 Written homeworks
- 2 Programming homeworks
- Term project (35%)
- Final exam (25%)
Late submission policy:
Late submissions are accepted up to 2 days after the due date, with 10% (of the total grade of the item) penalty per day.
Term projects
Students will work on a open-topic research project with groups. Each group could only be consisted with less or equal than 4 members (<=4). Feel free to discuss with me offline for the topic choice.
Tentative Schedule and Material
| Date | Topic | Slides | Readings&links | Assignments |
|---|---|---|---|---|
| Mon 5/9 | Overview of Machine Learning | lecture_0 | ||
| Wed 7/9 | Math Basics | lecture_1 | Matrix Calculus:Derivation and Simple Application HU Pili, DL Chapter 2.1 & 2.2 &2.3 | |
| Wed 14/9 | Linear models | lecture_2 | ||
| Mon 19/9 | Optimization | lecture_3 | Convex Optimization Boyd and Vandenberghe Chapter 3.1, Numerical Optimization Nocedal and Wright Chapter 3.1 | |
| Wed 21/9 | Stochastic gradient descent and its variants | lecture_4 | Written_HW1 out | |
| Mon 26/9 | Support Vector Machine, Polynomail nonlinear mapping, Kernel method, | lecture_5 | Stanford CS 229 notes | |
| Wed 28/9 | Learning theory | lecture_6 | Programming_HW1 out | |
| Mon 3/10 | Uniform convergence, growth function | lecture_7 | Symmetrization | |
| Wed 5/10 | VC Dimension | lecture_8 | ||
| Mon 10/10 | Regularization | lecture_9 | ||
| Wed 12/10 | Clustering | lecture_10 | ||
| Mon 17/10 | Tree-based methods | lecture_11 | Xgboost | |
| Wed 19/10 | Neural networks | lecture_12 | Written_HW2 out | |
| Mon 24/10 | Neural networks for computer vision | lecture_13 | ||
| Wed 26/10 | Dropout, Batch Norm, ResNet, Neural networks for NLP: basic | lecture_14 | Programming_HW2 out | |
| Mon 31/10 | Neural networks for NLP: Model | lecture_15 | ||
| Wed 2/11 | Transformer & Unsupervised pertaining for NLP | lecture_16 | ||
| Mon 7/11 | Vision Transformer | lecture_17 | ||
| Wed 9/11 | Semi-supervised learning, graph convolution network | lecture_18 | HW3 out | |
| Mon 14/11 | AutoML(Neural architecture search) | lecture_19 | ||
| Wed 16/11 | Recent progress in Neural architecture search | lecture_20 | ||
| Mon 21/11 | Limitations of deep learning: Testing-time integrity | lecture_21 | ||
| Wed 23/11 | Limitations of deep learning: Training-time integrity & Review | lecture_22 | ||
| Mon 28/11 | Final project presentation-part 1 | |||
| Wed 30/11 | Final project presentation-part 2 |
References
There is no required textbook for this course. Some recommended readings are
- Deep Learning (by Ian Goodfellow, Yoshua Bengio, Aaron Courville)
- CS 229: Machine Learning, Stanford University