36-410 Introduction to Probability Modeling

Stochastic processes are ways of quantifying the dynamic relationships of sequences of random events. Stochastic models play an important role in elucidating many areas of the natural, managerial, and engineering sciences. They can be used to analyze the variability inherent in biological and medical processes, to deal with uncertainties affecting managerial decisions, and with the complexities of psychological and social interactions, and to provide new perspectives, methodology, models and intuition to aid in other mathematical and statistical studies. This course in intended as an introduction to stochastic models for students familiar with elementary probability. The course aims to bridge the gap between a first course in mathematical probability and an intermediate level course in stochastic processes.

Syllabus

The syllabus provides information on grading, class policies etc.

My consolidated lecture notes are available here. These lecture notes cover the following topics.

Topics

• Probability Overview
• Expectations, Conditional Expectations and Common Distributions
• Markov Chain Basics
• Branching Processes
• Time Reversible Markov Chains and MCMC
• PageRank
• Markov Decision Processes
• Estimating Markov Chains
• Poisson Processes
• Continuous Time Markov Chains
• Markov Chain Mixing
• Martingales
• Brownian Motion
• Hidden Markov Models