Module I · Gaussians
13 sessions · Sep 2 – Oct 14
1Sep 2Gaussians: definition and linear structure
- Definition by the Fourier transform, and why not the density
- Linear images, existence, rotational invariance
- Uncorrelated and jointly Gaussian implies independent
- Maximum entropy at fixed covariance
2Sep 7Gaussians: conditioning as projection
- Conditional expectation as orthogonal projection
- The Schur complement, derived
- Bayesian posteriors and ridge regression
- Gaussian process regression
3Sep 9CLT: Lindeberg's method
- Maximum entropy, recalled from the homework; the entropic CLT stated
- Lindeberg's swapping argument, proved
- The Lindeberg condition as the Gaussian/Poisson boundary
- NNGP: a wide network at initialization
4Sep 14Stein's method
- The Gaussian characterized by an identity, not a limit
- Stein's equation; the CLT with a Berry–Esseen rate
- Chen–Stein for Poisson; dependence
5Sep 16Gaussian integration by parts: Wick's theorem
- Stein's identity
- Wick's theorem, by induction
- Pairings, and which of them cross
6Sep 21Semicircle law: the moment method
- The empirical spectral distribution; traces as moments
- Wick applied to tr M⁴; crossings suppressed by 1/N
- Catalan numbers and the limit
7Sep 23Semicircle law: resolvents
- The Stieltjes transform and its inversion
- Schur complement and the self-consistent equation
- The density, obtained directly
- Marchenko–Pastur
8Sep 28Cumulants: distance from Gaussian
- Cumulants; vanishing beyond the second characterizes the Gaussian
- Additivity, and scaling under normalized sums
- Rates for the CLT; cumulants as a perturbation parameter
9Sep 30Gaussian processes: the geometric picture
- A process on an index set T as a curve in a Hilbert space
- Existence from any positive semidefinite kernel
- The Karhunen–Loève expansion
10Oct 5Gaussian processes: the canonical metric and maxima
- The canonical pseudometric is distance in the Hilbert space
- Suprema as support functions; Gaussian width
- Sudakov and Dudley, stated
11Oct 7Kac–Rice and zero sets
- The area formula, invoked without proof
- Rice's formula for stationary processes
- Random trigonometric polynomials; Kac polynomials
12Oct 12Random landscapes: counting critical points
- Conditioned on ∇f = 0, the Hessian is a GOE matrix
- log|det| as a spectral integral
- Exponentially many, overwhelmingly saddles
13Oct 14Midterm examination
- Sessions 1–12, closed book
Module II · Dynamics, Deviations, Diffusion
12 sessions · Oct 26 – Dec 7
1Oct 26Martingales: conditional expectation and definitions
- Conditional expectation as projection
- Filtrations and the tower property
- The Doob decomposition
2Oct 28Martingales: limit theorems
- Doob's maximal inequality; convergence in L²
- Optional stopping
- The martingale central limit theorem
3Nov 2Martingales: concentration
- Azuma–Hoeffding and Freedman
- Bounded differences
4Nov 4SGD: stochastic approximation
- Drift plus martingale difference
- Robbins–Monro and the ODE method
5Nov 9SGD: fluctuations and averaging
- Asymptotic normality of the iterates
- Polyak–Ruppert averaging
- Concentration along the trajectory
6Nov 11SGD: high-dimensional dynamics
- A deterministic ODE plus a fluctuation term
- The data covariance spectrum, through the resolvent
7Nov 16Large deviations: energy versus entropy
- Cramér's theorem by exponential tilting
- The rate function as a competition
8Nov 18Large deviations: Sanov's theorem
- The method of types
- Relative entropy as the rate function
- The contraction principle
9Nov 23Large deviations: Varadhan and examples
- Varadhan's lemma and the Gibbs variational principle
- Curie–Weiss and its phase transition
- Hypothesis testing; escape from a basin
10Nov 30Diffusion models: the forward process
- Noising as a Gaussian channel
- Tweedie's formula from integration by parts
- Denoising as score estimation
11Dec 2Diffusion models: score matching
- The reverse chain
- Denoising score matching
- The DDPM objective
12Dec 7Diffusion models: the PDE picture
- Fokker–Planck and the heat equation
- The Gaussian as fundamental solution
- Time reversal; the variational bound