--- title: Prediction Processes author: 36-740 date: 24 September 2026 (Lecture 10) output: slidy_presentation: math_method: engine: mathjax url: https://bactra.org/mathjax/tex-svg.js bibliography: locusts.bib --- \[ \newcommand{\Prob}[1]{\mathbb{P}\left( #1 \right)} \newcommand{\Expect}[1]{\mathbb{E}\left[ #1 \right]} \newcommand{\LinearExpect}[1]{\mathbb{L}\left[ #1 \right]} \newcommand{\indep}{\perp} \] ## Prediction - We've emphasized _point_ prediction, $\Expect{Y|Z}$ + Or just the linear approximation, $\LinearExpect{Y|Z}$ - Supplemented with a variance - Last few lectures have opened up _distributional_ prediction, $\Prob{Y|Z}$ + Especially $\Prob{X_{t+1}|X_{1:t}}$ - Let's push that idea ## Prediction in Hidden Markov Models {.smaller} - Suppose we have an HMM with state $S_t$ and observable $X_t$ - $\Prob{X_t=x|S_t=s}$ and $\Prob{S_{t+1}=r|S_t=s}$ are both given - This fixes the conditional distribution for the next _two_ observations \[ \Prob{X_{t:t+1}=x_0 x_1|S_t=s} = \Prob{X_t=x_0|S_t=s} \sum_{r}{\Prob{X_{t+1}=x_1|S_{t+1}=r}\Prob{S_{t+1}=r|S_t=s}} \] - And so on for the distribution of $\Prob{X_{t:t+k}|S_t}$ for any $k$ - The current state fixes the distribution of all present and future observables + This is pretty much what "state" means in physics / dynamical systems ## Prediction without Hidden Markov Models - There is _some_ distribution for $X_{t+1}$ given $X_{1:t}$, $\Prob{X_{t+1}|X_{1:t}}$ + Sometimes it's mathematically convenient to make the past infinite, then we condition on $X_{-\infty;t}$ - $\therefore$ $\Prob{X_{t+1:t+k}|X_{1:t}}$ also exists - $\therefore$ $\Prob{X_{t+1:\infty}|X_{1:t}}$ also exists + Proving the second "$\therefore$" is the Ionescu-Tulcea theorem - The history of the process fixed the distribution of all future observables ## Prediction Process - Define the current **predictive state** as the current distribution over the future: \[ S_t \equiv \Lawof{X_{t+1:\infty}|X_{1:t}} \] - The sequence of predictive states is the **prediction process** of the original process - Knowing $S_t$ implies knowing $\Prob{X_{t+1}|X_{1:t}}$, etc. ## Sufficient Statistics - A **statistic** is a (measurable) function of the data - A statistic $S=\sigma(Z)$ is **sufficient** for predicting $Y$ when $Y \indep Z | S$, i.e., when \[ \Prob{Y|S=\sigma(z)} = \Prob{Y|Z=z} \] - Implication: If $\sigma(z_1) = \sigma(z_2)$ then $\Prob{Y|Z=z_1} = \Prob{Y|Z=z_2}$ ## Predictive States and Sufficient Statistics [@Inferring-stat-compl] - Histories $a$ and $b$ are equivalent iff \[ \Prob{X_{t+1:\infty}|X_{1:t} = a} = \Prob{X_{t+1:\infty}|X_{1:t} = b} \] - $[a] \equiv$ all histories equivalent to $a$ - Define \[ \epsilon(x_{1:t}) \equiv [x_{1:t}] \] - Set $s_t \equiv \epsilon(x_{1:t})$ - These are our predictive states - A predictive state is an equivalence class of histories _and_ a distribution over future events - IID = 1 state, periodic = $p$ states ## ![](history-space.png) set of histories, color-coded by conditional distribution of futures ## ![](histories-partitioned-into-causal-states.png) Partitioning histories into predictive states ## Sufficiency [@CMPPSS] - The predictive states are sufficient: \begin{eqnarray*} \Prob{X_{t+1:\infty}|S_t = \epsilon(x_{1:t})} & = & \int_{y \in [x_{1:t}]}{\Prob{X_{t+1:\infty}|X_{1:t}=y} \Prob{X_{1:t}=y|S_t = \epsilon(x_{1:t})} dy}\\ & = & \Prob{X_{t+1:\infty}|X_{1:t}=x_{1:t}} \end{eqnarray*} because all the histories in the equivalence class have the same predictive distribution ## ![](non-sufficienct-partition.png) A non-sufficient partition of histories ## ![](effect-of-insufficiency-on-prediction.png) Effect of insufficiency on predictive distributions ## Markov Properties (I) - Future observations are independent of the past given the causal state: \[ X_{t+1:\infty} \indep X_{1:t} | S_{t} \] - by sufficiency: \begin{eqnarray*} \Prob{X_{t+1:\infty}|X_{1:t}=x_{1:t}, S_{t} = \epsilon(x_{1:t})} & = & \Prob{X_{t+1:\infty}|X_{1:t}=x_{1:t}}\\ & = & \Prob{X_{t+1:\infty}|S_{t} = \epsilon(x_{1:t})} \end{eqnarray*} ## Recursive Updating/Deterministic Transitions - Recursive transitions for predictive states: \[ \epsilon(x_{1:t+1}) = T(\epsilon(x_{1:t}), x_{t+1}) \] + Automata theory: "deterministic transitions" (even though there are probabilities) ## Recursive Updating from Sufficiency {.smaller} - Claim: If $u \sim v$, then $ua \sim va$ for any next observation $a$ - Fix any set of future events $F$ \begin{eqnarray*} \Prob{X_{t+1:\infty} \in aF|X_{1:t} = u} & = & \Prob{X_{t+1:\infty} \in aF|X_{1:t} = v}\\ \Prob{X_{t+1}= a, X_{t+2:\infty} \in F|X_{1:t} = u} & = & \Prob{X_{t+1}= a, X_{t+2:\infty} \in F|X_{1:t} = v} \end{eqnarray*} \begin{eqnarray*} \Prob{X_{t+2:\infty} \in F|X_{1:t+1}= ua}\Prob{X_{t+1}= a|X_{1:t} = u} & = & \Prob{X_{t+2:\infty} \in F|X_{1:t+1}= va}\Prob{X_{t+1}= a|X_{1:t} = v} \end{eqnarray*} \begin{eqnarray*} \Prob{X_{t+2:\infty} \in F|X_{1:t+1}= ua} & = & \Prob{X_{t+2:\infty} \in F|X_{1:t+1}= va}\\ ua & \sim & va \end{eqnarray*} + (same for continuous values or time but need more measure theory) ## Markov Properties (II) - The prediction process is a Markov process: \[ S_{t+1:\infty} \indep S_{1:t-1}|S_t \] - because (recursive updating) \[ S_{t+1:\infty} = T(S_t, X_{t+1:\infty}) \] and (sufficiency) \[ X_{t+1:\infty} \indep \left\{ X_{1:t}, S_{1:t-1}\right\} | S_t \] ## Minimality - $\epsilon$ is **minimal sufficient** $=$ can be computed from any other sufficient statistic - $\therefore$ for any sufficient $\eta$, exists a function $g$ such that \[ \epsilon(X_{1:t}) = g(\eta(X_{1:t})) \] - By construction, the predictive states are the minimal sufficient statistic for the future of the original process ## ![](sufficient-but-not-minimal.png) Sufficient, but not minimal, partition of histories ## ![](coarser-but-not-sufficient.png) Coarser than the predictive states, but not sufficient ## Uniqueness - There is really no other minimal sufficient statistic - If $\eta$ is minimal, there is an $h$ such that \[ \eta = h(\epsilon) ~\mathrm{w.p.1} \] - but $\epsilon = g(\eta)$ (w.p.1) - so \begin{eqnarray*} g(h(\epsilon)) & = & \epsilon\\ h(g(\eta)) & = & \eta \end{eqnarray*} - $\therefore$ $\epsilon$ and $\eta$ partition histories in the same way (w.p.1) ## Minimal Markovian Representation - The observed process $(X_t)$ is non-Markovian and ugly - But it is generated from a homogeneous Markov process $(S_t)$ - After minimization, this representation is (essentially) unique + Can exist smaller Markovian representations, but then always have distributions over those states\ldots + \ldots and those distributions correspond to predictive states ## What Sort of Markov Model? - Common-or-garden HMM: \[ S_{t+1} \indep X_{t+1}|S_t \] - But here \[ S_{t+1} = T(S_t, X_{t+1}) \] - This is a **chain with complete connections** [@intro-of-chains-with-complete-connections;@Iosifescu-Grigorescu-complete-connections] ## HMM ![](hmm-graph.png) CCC ![](csm-graph.png) ## Example of a CCC: Even Process ![](even_process.png) - Blocks of As of any length, separated by even-length blocks of Bs + Histories ending $AB$ have different implications than histories ending $ABB$ than $ABBB$ than $ABBBB$ + $\Rightarrow$ Not Markov at any finite order ## Inventions {.smaller} - "Statistical relevance basis" [@Salmon-1971;@Salmon-1984] + Based on a manuscript circulated in the 1960s - "Measure-theoretic prediction process" [@Knight-predictive-view] - Forecasting/true measure complexity [@Grassberger-1986] - "Causal states", "$\epsilon$ machine" [@Inferring-stat-compl] + Not generally causal + Though maybe sometimes? [@What-is-a-macrostate] - "Observable operator model" [@Jaeger-operator-models] - "Predictive state representations" [@predictive-representations-of-state] - "Sufficient posterior representation" [@Langford-Salakhutdinov-Zhang] ## How Broad Are These Results? @Knight-predictive-view gave most general constructions - Non-stationary $X$ - $t$ continuous (but discrete works as special case) - $X_t$ with values in continuous spaces + Lusin space = image of a complete separable metrizable space under a measurable bijection - $S_t$ is a Markov process with recursive updating ## Connecting to Data - Everything so far has been math/probability + The Oracle tells us the infinite-dimensional distribution of $X$ - Can we do some statistics and find the states? - Two senses of "find": learn in a fixed model vs. discover the right model ## Learning **Problem**: Given states and transitions ($\epsilon, T$), realization $x_{1:n}$, estimate $\Prob{X_{t+1}=x|S_t=s}$ - Just estimation for stochastic processes + Easier than ordinary HMMs because $S_t$ is a function of trajectory + Exponential families in the all-discrete case, very tractable ## Discovery **Problem**: Given $x_{1:n}$, estimate $\epsilon, T, \Prob{X_{t+1}=x|S_t=s}$ - Much harder! + Why should this be possible at all? + Can't cover every possible approach (and will neglect @Langford-Salakhutdinov-Zhang;@Pfau-Bartlett-Wood-prob-det-inf-automata both) ## CSSR: Causal State Splitting Reconstruction - Key observation: Recursion + one-step-ahead predictive sufficiency $\Rightarrow$ general predictive sufficiency + Get next-step distribution right by independence testing + Then make states recursive - Assumes discrete observations, discrete time, finite number of predictive states - Paper: @CSSR-for-UAI; code, [http://bactra.org/CSSR/] + R interface (not by us, haven't checked it beyond making sure it compiles): [https://github.com/szhan/CSSR-R] ## One-Step Ahead Prediction - Start with all histories in the same state - Given current partition of histories into states, test whether going one step further back into the past changes the next-step conditional distribution + Use a hypothesis test to control false positive rate + If yes, split that cell of the partition, but see if it matches an existing distribution + Must allow this merging or else lose minimality - If no match, add new cell to the partition - Stop when no more divisions can be made or a maximum history length $\Lambda$ is reached + For consistency, $\Lambda < \frac{\log{n}}{h + \iota}$ for some $\iota$ [@Marton-Shields] * See backup for definition of the entropy rate $h$ ## Ensuring Recursive Transitions - Need to determinize a probabilistic automaton - Several ways of doing this, but a solved technical problem in automata theory + Textbooks: @Lewis-Papadimitriou-computation; @Hopcroft-Ullman + Brilliant but bizarre math approach: @Conway-finite-machines - Trickiest coding in the algorithm and can influence the finite-sample behavior ## Convergence - $\mathcal{S} =$ true predictive state structure - $\widehat{\mathcal{S}}_n$ = structure reconstructed from $n$ data points - Assume: finite \# of states, every state has a finite history, using long enough histories, technicalities: \[ \Prob{\widehat{\mathcal{S}}_n \neq \mathcal{S}} \rightarrow 0 \] - $\mathcal{D} =$ true distribution, $\widehat{\mathcal{D}}_n$ = inferred \[ \Expect{{\|\widehat{\mathcal{D}}_n - \mathcal{D}\|}_{1}} = O(n^{-1/2}) \] + Same order of convergence as for IID samples ## Hand-waving - Empirical conditional distributions for histories converge + ergodic theorem for Markov chains - Histories in the same state become harder to accidentally separate - Histories in different states become harder to confuse - Each state's predictive distribution converges $O(n^{-1/2})$ + Convergence rate in the ergodic theorem for Markov chains ## Example: The Even Process ![](even_process.png) ![](reconstructed_even.png) reconstruction with $\Lambda = 3$, $n=1000$, $\alpha = 0.005$ N.B., CSSR did not know that there were 2 states, or how they were connected --- it discovered this ## On-Going Work [@MW-S-CRS-prediction] {.smaller} - Suppose $S_t$ is continuous but $\dim{S_t} = p$, and $X_t \in \mathbb{R}^d$ > Theorem (roughly) [@MW-S-CRS-identification]: If a family $M_{\theta}$ of distributions on $\mathbb{R}^d$ depends smoothly on a $p$-dimensional parameter $\theta$, and $\theta_1 \neq \theta_2 \Rightarrow M_{\theta_1} \neq M_{\theta_2}$, then there is almost surely a diffeomorphism between $\theta$ and the expected values of $k = 2p+1$ _randomly chosen_ Fourier basis functions, say $f(x) = (f_1(x), \ldots f_k(x))$ - Apply to $\Prob{X_{t+1}|S_t=s}$ (with $s$ in the role of $\theta$) + By recursive updating, we only need the one-step-ahead distributions - $Z_t \equiv \Expect{f(X_{t+1}|S_t}$ is (almost surely) diffemorphic to $S_t$ - $\therefore$ $Z_{t+1} = \tau(Z_t, X_{t+1})$ - Do a pilot estimate of $\Expect{f(X_{t+1})|X_{t-m:t}}$ to get initial values of $Z_t$ + This is regression - Estimate the update mapping $\tau$ + This is also regression - Estimate $\Prob{X_{t+1}|Z_t=z}$ + This is conditional density estimation ## Some Uses - Neural spike train analysis [@spike-train-complexity], fMRI analysis (Merriam, Genovese and Shalizi in prep.) - Geomagnetic fluctuations [@British-Antarctic-Survey-comp-mech-of-geomag] - Natural language processing [@Padro-Padro-FSMNLP05;@Padro-Padro-PLN05;@Padro-Padro-RANLP05;@Padro-Padro-FSMNLP07;@Padro-Padro-PLN07] - Anomaly detection [@Friedlander-Phoha-Brooks-vehicle-behavior;@Friedlander-et-al-anomaly-prediction;@Ray-anomaly-detection] - Information sharing in networks [@Info-Coh-for-NIPS;@Functional-communities-for-ICML] - Social media propagation [@Cointent-Faure-Roth-ICWSM-2007] ## Extension to Spatiotemporal Systems {.smaller} [@CRS-prediction-on-networks;@QSO-in-PRL;@Automatic-Filters;@Janicke-et-al-multifield-visualization;@LICORS-package;@LSC-package;@LICORS;@Mixed-LICORS;@Montanez-CRS-LICORS-cabinet] - Dynamic random field $X(\vec{r},t)$ + Assume a finite maximum speed $c$ at which information can propagate - Past cone: points in space-time which could matter to $X(\vec{r},t)$ - Future cone: points in space-time for which $X(\vec{r},t)$ could matter ![](light-cones.png) - Equivalence-class past cone configurations by conditional distributions over future cones - $S(\vec{r},t)$ is a Markov field - Minimal sufficiency, recursive updating, etc., all go through ## Filtering for Spatio-Temporal Systems [@Automatic-Filters] {.smaller} - Time evolution of a dynamic Markov random field: starting from an IID initial condition (left), spiral waves self-organize around randomly-located cores (right) + Specifically this is a "cyclic cellular automaton" [@Fisch-Gravner-Griffeath; @Fisch-Gravner-Griffeath-threshold-range] * If $X(\vec{r}, t) = j$ and enough neighbors are in state $j+1 \bmod k$, then $X(\vec{r}, t+1) = j+1 \bmod k$ ```{r, fig.retina=NULL, out.width=700, echo=FALSE} knitr::include_graphics("ca-evolution.png") ``` - The "order parameter" for the spiral waves, characterizing their final distribution, as worked out by exact theory (left), vs. $-\log{\Prob{S(\vec{r}, t)}}$, the "local statistical complexity" (right) + The algorithm which reconstructed the local predictive states didn't know anything about the "physics" of the original process, let alone the order parameter ```{r, fig.retina=NULL, out.width=700, echo=FALSE} knitr::include_graphics("op-vs-local-complexity.png") ``` ## Summary - Your stochastic process has a unique, minimal Markovian representation - This representation has nice predictive properties - Can reconstruct from sample data in some cases\ldots + and a lot more could be done in this line ## Backup: Why Care About Sufficiency? - Optimal strategy, under any loss function, only needs a sufficient statistic (Blackwell \& Girshick) - Strategies using insufficient statistics can generally be improved (Blackwell \& Rao) - Excuse for not worrying about particular loss functions ## Backup: Crash Course in Information Theory: Entropy - The **entropy** of a random variable $X$: \[ H[X] \equiv -\sum_{x}{\Prob{X=x}\log_2{\Prob{X=x}}} \] - Some mathematical properties: + $H[X] \geq 0$ + $H[X] = 0$ if and only if $\Prob{X=x_0} = 1$ for some $x_0$ + If $X$ has $k$ possible values, $H[X] \leq \log_2{k}$ + $H[X] = \log_2{k}$ if and only if $\Prob{X=x} = 1/k$ for all $x$ - **Data processing inequality**: $H[X] \geq H[\sigma(X)]$ + $H[X] = H[\sigma(X)]$ iff $\sigma$ is 1-1 - $H[X] \approx$ number of bits needed to encode the value of $X$ - $H[X]$ is the uncertainty in the distribution of $X$ ## Joint and Conditional Entropy - Conditional entropy: \begin{eqnarray} H[Y|X=x] & \equiv & -\sum_{y}{\Prob{Y=y|X=x}\log_2{\Prob{Y=y|X=x}}}\\ H[Y|X] & \equiv & \Expect{H[Y|X=X]} \end{eqnarray} + Condition on multiple variables in the natural way - Joint entropy: \[ H[X,Y] = H[X] + H[Y|X] = H[Y] + H[X|Y] \] - Same interpretations as entropy ## Entropy Rate - Limiting entropy per time-step: \[ h \equiv \lim_{n\rightarrow\infty}{\frac{1}{n}H[X_{1:n}]} \] - Equivalently, limiting conditional entropy: \[ h = \lim_{n\rightarrow\infty}{H[X_{n+1}|X_{1:n}]} \] - Both limits exist and are equal for all stationary processes - Interpretations: + How many bits do we need to code the next observation, given the complete history? ("source coding") + How much uncertainty is there about what happens next, given the complete history? + $h=0$ for deterministic processes ## Information - The information $X$ gives about $Y$ is the reduction in uncertainty: \[ H[Y] - H[Y|X] \] - Some algebra shows that \[ H[Y] - H[Y|X] = H[X] - H[X|Y] \] - Define the **mutual information** \[ I[X;Y] \equiv H[Y] - H[Y|X] \] - Some properties: + $I[X;Y] \geq 0$ + $I[X;Y] = 0$ if and only if $X$ and $Y$ are independent - Data processing: $I[X;Y] \geq I[\sigma(X);Y]$ + Can be preserved even if $\sigma$ isn't 1-1 ## Predictive Sufficiency (Again) - For any statistic $\sigma$, \[ I[X_{t+1:\infty};X_{1:t}] \geq I[X_{t+1:\infty};\sigma(X_{1:t})] \] + (Data-processing inequality) - $\sigma$ is **predictively sufficient** iff \[ I[X_{t+1:\infty};X_{1:t}] = I[X_{t+1:\infty};\sigma(X_{1:t})] \] + This is equivalent to our earlier definition (can you show this?) - Sufficient statistics retain all predictive information in the data - The predictive state maximizes the predictive information ## Minimality (Again) - Recall: $\epsilon$ is **minimal sufficient** $=$ can be computed from any other sufficient statistic - $\therefore$ if $\eta$ is sufficient \[ I[\epsilon(X_{1:t}); X_{1:t}] \leq I[\eta(X_{1:t}); X_{1:t}] \] + (Data-processing inequality) ## Minimal stochasticity [@CMPPSS] - If $R_t = \eta(X_{1:t-1})$ is also sufficient, then \[ H[R_{t+1}|R_t] \geq H[S_{t+1}|S_t] \] - $\therefore$ the predictive states are the closest we get to a deterministic model, without losing power ## Entropy Rate - For a stationary process, \begin{eqnarray*} h & \equiv & \lim_{n\rightarrow\infty}{H[X_{n+1}|X_{1:n}]}\\ & = & \lim_{n\rightarrow\infty}{H[X_{n+1}|S_n]} ~\text{(sufficiency)}\\ & = & H[X_2|S_1] ~\text{(stationarity)} \end{eqnarray*} - so the predictive states lets us calculate the entropy rate + and do source coding ## Backup: Statistical Forecasting Complexity - **Statistical forecasting complexity** is [@Grassberger-1986;@Inferring-stat-compl] \[ C \equiv I[\epsilon(X_{1:t});X_{1:t}] \] - $=$ amount of information about the past needed for optimal prediction - $=H[\epsilon(X_{1:t})]$ for discrete predictive states - $=$ expected algorithmic sophistication [@Gacs-Tromp-Vitanyi] - $=\log$(period) for period processes - $=\log$(geometric mean(recurrence time)) for stationary processes - Property of the _process_, not our models ## Backup: A Cousin: The Information Bottleneck [@Tishby-Pereira-Bialek-bottleneck] - For inputs $X$ and outputs $Y$, fix $\beta > 0$, find $\eta(X)$, the **bottleneck variable**, maximizing \[ I[\eta(X);Y] - \beta I[\eta(X);X] \] - give up 1 bit of predictive information for $\beta$ bits of memory - Predictive sufficiency comes as $\beta \rightarrow \infty$, unwilling to lose _any_ predictive power ## Backup: Extension to Input-Output Systems [@predictive-representations-of-state; @CRS-thesis] - System output $(X_t)$, input $(Y_t)$ - Histories $x_{1:t}, y_{1:t}$ have distributions of output $x_{t+1}$ for each further input $y_{t+1}$ - Equivalence class these distributions and enforce recursive updating - Internal states of the system, not trying to predict future inputs ## References