Theory of Deep Learning
Informal seminar for PhD students with background in mathematics, physics, statistics, etc. The goal is to formulate core open problems at the intersection of theory and practice of deep learning.
For current Princeton affiliates only. Email bhanin 'at' princeton.edu if you're interested in attending.
Schedule
| Introduction | ||
| Thu Sep 10 | 1 | Overview what is a neural network; how it is used; the big questions of the course |
| Scaling laws | ||
| Thu Sep 17 | 2 | Compactify the moduli space of neural networks → hyperparameter transfer opentransfer over the token horizon; the large-data regime |
| Thu Sep 24 | 3 | High-dimensional limits of SGD openpower law exponents; optimizer design |
| Feature learning | ||
| Thu Oct 1 | 4 | Near-linear windows: one gradient step; Bayesian posteriors openwhat gets learned, and when |
| Diffusion Models | ||
| Thu Oct 8 | 5 | The score Hamiltonian opendiscrete time; schedules you would actually use |
| Thu Oct 15 | 6 | Sampling ≲ score estimation (Chewi et al.) openupper vs. lower bounds |
| Thu Oct 22 | no class — fall recess (Oct 17–25) | |
| To be determined | ||
| Thu Oct 29 | 7 | TBD |
| Thu Nov 5 | 8 | TBD |
| Thu Nov 12 | 9 | TBD |
| Thu Nov 19 | TBD | |
| Thu Nov 26 | no class — Thanksgiving recess (Nov 25–29) | |
| Thu Dec 3 | 10 | TBD |