UBC CPEN 455 (2026 Winter Term 1): Deep Learning
Overview
Deep Learning has transformed fields like computer vision, speech recognition, and natural language processing, becoming a cornerstone of modern machine learning and artificial intelligence.
This course provides a comprehensive exploration of foundational and advanced topics in deep learning. You will develop a solid understanding of statistical learning, generalization, backpropagation, and optimization, and study modern neural network architectures, including multilayered perceptrons, convolutional networks, recurrent networks, graph neural networks, and transformers.
A particular highlight of the course is its focus on generative models, including autoregressive models such as large language models (LLMs), Variational Autoencoders (VAEs), and diffusion models, with an introduction to latent diffusion, diffusion transformers, and flow matching. The LLM topics include efficient inference, model adaptation, and evaluation.
Reinforcement learning foundations, including returns, value functions, and policy gradients, support a focused treatment of LLM post-training and reasoning. We will distinguish instruction fine-tuning, preference optimization, and reinforcement learning from human feedback or verifiable rewards, and examine the role of test-time computation.
The course emphasizes hands-on learning through implementation, model adaptation, and reproducible experiments. You will practice comparing models against appropriate baselines, evaluating on held-out data, interpreting results, and communicating findings, connecting theoretical foundations with practical applications.
Learning Outcomes
By the end of this course, students should be able to:
- Explain the statistical learning principles behind training and evaluating neural networks.
- Derive gradients and apply backpropagation and optimization methods.
- Implement, adapt, and compare neural network architectures, including generative models.
- Explain reinforcement learning foundations and their role in LLM post-training and reasoning.
- Design experiments with appropriate baselines and held-out data, interpret results, and communicate findings with reproducible code and clear reports.
Course Information
| Instructor | Renjie Liao |
|---|---|
| TA | Felix Fu |
| Lecture Time | 1:00 p.m. - 2:30 p.m., Wed. and Fri. |
| Lecture Dates | Sep. 9 - Dec. 4, 2026 |
| Lecture Location | UBCV, Food, Nutrition and Health Building (FNH), Floor: -1, Room: 60 |
| Piazza | https://piazza.com/ubc.ca/winterterm12026/cpen455 |
| Canvas | CPEN_V 455 101 2026W1 Deep Learning |
| Tutorial Time | 9:00 a.m. - 10:00 a.m., Mon. |
| Tutorial Dates | Sep. 14 - Dec. 7, 2026 |
| Tutorial Location | UBCV, Food, Nutrition and Health Building (FNH), Floor: -1, Room: 60 |
| Office Hour | 2:00pm to 3:00pm, Tuesday, KAIS 3047 (Ohm) |
| renjie.liao@ubc.ca |
Announcements
- Lectures begin Wed., Sep. 9, 2026; tutorials begin Mon., Sep. 14, 2026.
- Lecture 2 notes are now available: Please read the notes before the lecture if possible, or review them afterward; the material may be relevant to homework and quizzes. Please report any errors or unclear explanations to the TA or instructor for consideration toward the active participation bonus.
Please pay attention to announcements on Piazza and Canvas!
Pre-requisites
- Basic knowledge in linear algebra, probability, and calculus.
- Proficiency in a programming language: preferably Python.
- Proficiency in a deep learning library: preferably PyTorch, JAX, Tensorflow, etc.
- Proficiency in LaTex.
Grading
- [20%] Homework (i.e., Written Assignments, 2X, each with 10%). Guideline & Policy
- [30%] Programming Assignments (3X, each with 10%). Guideline & Policy
- [20%] In-class Quiz (2X, each with 10%). Both quizzes will take place no later than November 27, 2026. Guideline & Policy
- [30%] Project (1X). Guideline & Policy
- [Up to 3 percentage points extra credit] Participation. The final course grade is capped at 100%. Guideline & Policy
Course Policies
Please read the Course Policies, covering individual work, submissions and late work, academic concessions, GenAI, and student support.
Schedule
This tentative 2026 schedule follows the UBC Vancouver calendar1. Changes will be announced on Piazza and this website.
Lectures
Slide PDFs will be posted as they are finalized. TBA marks slides to be added.
| Lectures | Dates | Topic | Slides | Suggested Readings |
|---|---|---|---|---|
| Lecture 1 | Sep. 9 | Introduction | slides 1 | DL book: ch. 1 |
| Lecture 2 | Sep. 11 | Linear Models for Regression & Classification, Generalization, Model Evaluation | slides 2 notes |
DL book: ch. 5; PML1 book: ch. 10 & 11; PRML book: ch. 3 & 4 |
| Sep. 11 | Release Homework 1 (due Oct. 2) | |||
| Lecture 3 | Sep. 16 | Multilayered Perceptron (MLP), Activation Functions (ReLU, GELU, SiLU), Batch Normalization, Dropout | slides 3 | DL book: ch. 6 & 7; PML1 book: ch. 13; PML2 book: ch. 16; PRML book: ch. 5 |
| Lecture 4 | Sep. 18 Sep. 23 |
Back-Propagation, Weight Initialization | slides 4.1 notes |
DL book: ch. 7 & 8; PML1 book: ch. 13; PML2 book: ch. 6; PRML book: ch. 5 |
| Sep. 25 Oct. 2 |
Optimization Methods w. Adaptive Learning Rate, AdamW, Learning Rate Schedules, Weight Decay, Early Stopping | slides 4.2 notes |
DL book: ch. 7 & 8; PML1 book: ch. 13; PML2 book: ch. 6; PRML book: ch. 5 | |
| Oct. 2 | Release Homework 2 (due Oct. 23) | |||
| Lecture 5 | Oct. 7 Oct. 9 |
Invariance, Equivariance, Convolutions and Variants (Transposed, Dilated, Grouped, Separable), Pooling, CNNs (UNet, ResNet, MobileNet) | slides 5.1 slides 5.2 |
DL book: ch. 9; PML1 book: ch. 14; PML2 book: ch. 16 |
| Lecture 6 | Oct. 14 Oct. 16 |
Recurrent Neural Networks, Back-Propagation Through Time, LSTMs | slides 6 | DL book: ch. 10; PML1 book: ch. 15; PML2 book: ch. 16 |
| Lecture 7 | Oct. 21 Oct. 23 |
Graph Neural Networks: Message Passing, Graph Readout, Graph Attention | slides 7 | PML1 book: ch. 23; PML2 book: ch. 16 |
| Oct. 23 | Release Programming Assignment 1 (due Nov. 6) | |||
| Oct. 23 | Release Course Project (due Dec. 10) | |||
| Lecture 8 | Oct. 28 Oct. 30 |
Transformers: Sinusoidal Positional Encoding; Encoder (Multi-Head Self-Attention, Residual Connections & LayerNorm); Decoder (Masked Self-Attention & Cross-Attention); Variants: Efficient Attention (FlashAttention, Sparse & Linear), RoPE, LayerNorm/RMSNorm, Pre-/Post-Norm, SwiGLU, ViT & Swin | slides 8 | PML1 book: ch. 15; PML2 book: ch. 16; Llama 3 |
| Lecture 9 | Nov. 4 Nov. 6 |
Large Language Models (LLMs): Tokenization (BPE, WordPiece & Unigram); Autoregressive Pretraining, Scaling Laws & Mixture of Experts; Data Mixtures, Mid-Training & Context Extension; Supervised Fine-Tuning; Parameter-Efficient Fine-Tuning (LoRA & QLoRA) | slides 9 | PML1 book: ch. 15; PML2 book: ch. 22; Chinchilla; LoRA |
| Nov. 6 | Release Programming Assignment 2 (due Nov. 20) | |||
| Lecture 10 | Nov. 13 | Reinforcement Learning Foundations for LLM Post-Training: MDPs (Overview), Policies, Rewards & Returns, Policy Gradient (REINFORCE), Baselines & Advantages | slides 10: TBA | PML2 book: ch. 35 (selected sections) |
| Lecture 11 | Nov. 18 Nov. 20 |
LLM Post-Training & Reasoning: Instruction Fine-Tuning (Recap), Reward Modeling, RLHF (PPO Overview), DPO, Verifiable Rewards, Test-Time Computation & Evaluation | slides 11.1: TBA slides 11.2: TBA |
DPO; DeepSeek-R1 |
| Nov. 20 | Release Programming Assignment 3 (due Dec. 4) | |||
| Lecture 12 | Nov. 25 | Autoencoders, Denoising Autoencoders, Variational Autoencoders (VAEs) | slides 12: TBA | DL book: ch. 14; PML1 book: ch. 20; PML2 book: ch. 21 |
| Lecture 13 | Nov. 27 Dec. 2 Dec. 4 |
Diffusion Models: DDPMs, Classifier-Free Guidance, Latent Diffusion, Diffusion Transformers (DiT), Introduction to Flow Matching | slides 13.1: TBA slides 13.2: TBA slides 13.3: TBA |
PML2 book: ch. 25; Latent Diffusion; DiT; Flow Matching |
Q-Learning/DQN, Dyna-Q, detailed value-based planning and actor-critic methods, NPG/TRPO derivations, and detailed score-SDE derivations are optional reading.
Tutorials
| Tutorials | Dates | Topic | Slides | Suggested Readings |
|---|---|---|---|---|
| Tutorial 1 | Sep. 14 | Probability & Statistics | slides | Intro to Probability |
| Tutorial 2 | Sep. 21 | Linear Algebra & Matrix Calculus | slides | |
| Tutorial 3 | Sep. 28 | Tensor Operations & Dataset & DataLoader, Data Splits & Reproducibility | slides Colab |
Pytorch Official Tutorials |
| Tutorial 4 | Oct. 5 | Autograd & Build Your Models, Mixed Precision | slides Colab |
|
| Tutorial 5 | Oct. 19 | Explain HW1 | ||
| Tutorial 6 | Oct. 26 | Discussion on Quiz 1 | ||
| Tutorial 7 | Nov. 2 | Hugging Face Transformers: Tokenization, Causal Masking, Sampling & KV Caching | slides: TBA | |
| Tutorial 8 | Nov. 16 | Small-Model Adaptation & Evaluation: LoRA, Baselines & Error Analysis | slides: TBA | |
| Tutorial 9 | Nov. 23 | Discussion on PA1 | ||
| Tutorial 10 | Nov. 30 | Discussion on PA2 | ||
| Tutorial 11 | Dec. 7 | Discussion on Course Project and Office Hour |
1 Excluded: Sep. 30, National Day for Truth and Reconciliation (no lecture); Oct. 12, Thanksgiving (no tutorial); and Nov. 9–11, midterm break including Remembrance Day (no Nov. 9 tutorial or Nov. 11 lecture). Lectures resume Fri., Nov. 13. The final lecture is Fri., Dec. 4; the final tutorial is Mon., Dec. 7, UBC's last day of Term 1 classes. ↩
FAQ
Can I audit or sit in?
I am very open to auditing guests if you are a member of the UBC community (registered student, staff, and/or faculty). I would appreciate that you first email me. If the in-person class is too full and running out of space, I would ask that you please allow registered students to attend.
Is there a textbook for this course?
While there is no required textbook, I recommend the following closely relevant ones for further reading:
- DL book: “Deep Learning” by Ian Goodfellow, Yoshua Bengio, Aaron Courville. Free online version
- PML1 book: “Probabilistic Machine Learning: An Introduction” by Kevin Murphy. Free online version
- PML2 book: “Probabilistic Machine Learning: Advanced Topics” by Kevin Murphy. Free online version
- PRML book: “Pattern Recognition and Machine Learning” by Christopher Bishop. Free online version
I also recommend students who are self-motivated to take a look at similar courses taught at other universities:
- UofT CSC413/2516, Fall 2025: Neural Networks and Deep Learning
- University of Amsterdam, Fall 2025: Deep Learning 1, Notebook Tutorials
- University of Geneva 14X050 (formerly EPFL EE559): Deep Learning
- Stanford CS231n, Spring 2026: Deep Learning for Computer Vision
- Stanford CS336, Spring 2026: Language Modeling from Scratch
- MIT 6.S184, IAP 2026: Introduction to Flow Matching and Diffusion Models