36-705 Intermediate Statistics
This course covers the fundamentals of theoretical statistics. Topics include: concentration of measure, basic empirical process theory, convergence, point and interval estimation, maximum likelihood, hypothesis testing, Bayesian inference, nonparametric statistics and bootstrap re-sampling. This course is excellent preparation for advanced work in Statistics and Machine Learning. See below for a detailed schedule.
Course Syllabus
The syllabus provides information on grading, class policies etc.
Lecture Notes
- Lecture 1: (8/26) A very brief review
- Lecture 2: (8/28) Concentration inequalities
- Lecture 3: (8/30) More concentration inequalities
- Lecture 4: (9/4) Even more concentration inequalities
- Lecture 5: (9/6) Convergence of random variables
- Lecture 6: (9/9) More Convergence and the CLT
- Lecture 7: (9/11) CLT and variants
- Lecture 8: (9/13) Uniform Laws
- Lecture 9: (9/16) Applications of Uniform Convergence
- Lecture 10: (9/18) Rademacher Complexity
- Lecture 11: (9/23) Sufficient Statistics
- Lecture 12: (9/25) Minimal Sufficiency
- Lecture 13: (9/27) Exponential Families
- Lecture 14: (9/30) Constructing Estimators
- Lecture 15: (10/2) Cramer-Rao Lower Bound
- Lecture 16: (10/4) Decision Theory Basics
- Lecture 17: (10/7) Bounding the Minimax Risk
- Lecture 18: (10/9) Consistency of the MLE
- Lecture 19: (10/11) Asymptotic Normality of the MLE
- Lecture 20: (10/14) Hypothesis Testing and the Neyman-Pearson Lemma
- Lecture 21: (10/16) General Purpose Tests
- Lecture 22: (10/21) LRT, Two-Sample Testing, Permutation Test
- Lecture 23: (10/23) Multiple Testing
- Lecture 24: (10/28) More Multiple Testing
- Lecture 25: (10/30) Confidence Intervals
- Lecture 26: (11/4) Confidence Intervals and the Bootstrap
- Lecture 27: (11/6) More on the Bootstrap
- Lecture 28: (11/8) Causal Inference
- Lecture 29: (11/11) More Causal + Non-Parametric Regression
- Lecture 30: (11/13) High-dimensional Statistics
- Lecture 31: (11/15) Linear Regression
- Lecture 32: (11/18) Non-parametric Regression
- Lecture 33: (11/22) Bayesian Inference
- Lecture 34: (12/2) Model Selection
- Lecture 35: (12/4) Distances Between Distributions
- Lecture 36: (12/6) Fano’s inequality and more distances