UBC CPEN 400D (2022 Winter Term 2): Deep Learning
Overview
Deep Learning has revolutionized many fields, e.g., computer vision, speech recognition, and natural language processing. It has become the vital pillar underpinning the modern machine learning and AI and is one of the most highly sought after skills in industries.
In this course, we will study the fundamentals of deep learning, including architectures (e.g., MLPs, CNNs, RNNs, Transformers, and GNNs) and learning algorithms under different paradigms (supervised / unsupervised / reinforcement learning), with an emphasis on motivating applications, design principles, and practical and or theoretical limitations.
Course Information
| Instructor | Renjie Liao |
|---|---|
| TA | Qi Yan, Sadegh Mahdavi, Jiahe Liu |
| Time | 12:30pm to 2:00pm, Tue. and Thu. |
| Location | Hugh Dempster Pavilion 310 |
| Piazza | https://piazza.com/ubc.ca/winterterm22022/cpen400d |
| Canvas | CPEN 400D 206 2022W2 |
| Office Hour | 1:00pm to 2:00pm Wed. KAIS 3047 (Ohm) |
| rjliao@ece.ubc.ca |
Announcements
- Jan. 9 2023: The course will be in person in this semester by default unless specially announced.
Pre-requisites
- Basic knowledge in linear algebra, probability, and calculus.
- Proficiency in a programming language: preferably Python.
- Proficiency in a deep learning library: preferably PyTorch, JAX, Tensorflow, etc.
- Proficiency in LaTex.
Grading
- [30%] Homework (i.e., Written Assignments, 2X, each with 15%). Guideline & Policy
- [30%] Programming Assignments (2X, each with 15%). Guideline & Policy
- [40%] Project (1X). Guideline & Policy
- [3% Extra Credits] Participation Guideline & Policy
Important Notes
-
All course-related questions should be sent and handled via Piazza. Canvas is only used for submitting homework, assignments, and projects. Try to avoid sending me emails directly as it is likely to be buried in my inbox.
-
All homework, assignments, and projects must be done individually. A 20% (non-hourly based) penalty is applied to any late submission. Any submission that is later than 3 days after the deadline will not be evaluated. E.g., if your homework is late but within 3 days after the deadline, your receive 80% of the grade for the homework. If it is beyond 3 days, then you get 0 grade.
-
UBC values academic integrity. Therefore, all students must understand the meaning and consequences of cheating, plagiarism and other academic offences under the Code of Student Conduct and Discipline.
Schedule
This is a tentative schedule, which will likely change as the course goes on. Changes will be announced on Piazza and this website.
| Lecture | Dates | Topic | Slides | Suggested Readings |
|---|---|---|---|---|
| 1 | Jan. 10 Jan. 12 Jan. 17 Jan. 19 |
Introduction & ML Basics & Linear Models | slides 1.1 slides 1.2 |
Chapter 5 of DL book & Chapter 10 and 11 of PML1 book |
| Assignment 1 | Jan. 13 (out) Feb. 3 (due) |
1st homework | ||
| 2 | Jan. 24 Jan. 26 Feb. 2 Feb. 7 Feb. 9 |
Multilayered Perceptron & Back-Propagation | slides 2.1 slides 2.2 slides 2.3 |
Chapter 6, 7 and 8 of DL book & Chapter 13 of PML1 book & Chapter 16 of PML2 book |
| Assignment 2 | Feb. 5 (out) Mar. 3 (due) |
1st Programming Assignment | ||
| 3 | Feb. 14 Feb. 16 |
Pytorch and Autograd | slides 3 | PyTorch Tutorials |
| 4 | Feb. 28 Mar. 2 Mar. 7 |
Convolutional Neural Networks | slides 4.1 slides 4.2 |
Chapter 9 of DL book & Chapter 14 of PML1 book & Chapter 16 of PML2 book |
| Assignment 3 | Mar. 3 (out) Mar. 24 (due) |
2nd Homework | ||
| 5 | Mar. 9 Mar. 14 |
Transformers | slides 5 | Chapter 15 of PML1 book & Chapter 16 of PML2 book |
| 6 | Mar. 16 | Recurrent Neural Networks | slides 6 Guest Lecture by Prof. Xiaoxiao Li |
Chapter 10 of DL book & Chapter 15 of PML1 book & Chapter 16 of PML2 book |
| 7 | Mar. 21 Mar. 23 |
Graph Neural Networks | slides 7 | Chapter 23 of PML1 book & Chapter 16 of PML2 book |
| Assignment 4 | Mar. 24 (out) Apr. 14 (due) |
2nd Programming Assignment | ||
| 8 | Mar. 28 Mar. 30 Apr. 4 |
Autoencoders & Denoising Autoencoders & Variational Autoencoders (VAEs) | slides 8 | Chapter 14 of DL book & Chapter 20 of PML1 book & Chapter 21 of PML2 book |
| 9 | Apr. 6 | Generative Advesarial Networks (GANs) | slides 9 | Chapter 20 of DL book & Chapter 26 of PML2 book |
| 10 | Apr. 11 Apr. 13 |
Deep Reinforcement Learning: Q-Learning & Policy Gradient | slides 10 | Chapter 35 of PML2 book |
| Project | Apr. 21 (due) |
FAQ
Can I audit or sit in?
I am very open to auditing guests if you are a member of the UBC community (registered student, staff, and/or faculty). I would appreciate that you first email me. If the in-person class is too full and running out of space, I would ask that you please allow registered students to attend.
Is there a textbook for this course?
While there is no required textbook, I recommend the following closely relevant ones for further reading:
- DL book: “Deep Learning” by Ian Goodfellow, Yoshua Bengio, Aaron Courville. Free online version
- PML1 book: “Probabilistic Machine Learning: An Introduction” by Kevin Murphy. Free online version
- PML2 book: “Probabilistic Machine Learning: Advanced Topics” by Kevin Murphy. Free online version
- PRML book: “Pattern Recognition and Machine Learning” by Christopher Bishop. Free online version
I also recommend students who are self-motivated to take a look at similar courses taught at other universities:
- UofT CSC413/2516, Winter 2022: Neural Networks and Deep Learning
- University of Amsterdam, 2022: Deep Learning
- EPFL EE559, Spring 2022: Deep Learning
- Stanford CS231n, Spring 2022: Deep Learning for Computer Vision
Previous Version
UBC CPEN400D 2022 W1 Taught by Prof. Brad Quinton and Prof. Scott Chin.