House-keeping

• HW 13 due tomorrow (Friday) at 6 pm
• FCEs

Causal vs. Ordinary Prediction

• In ordinary statistical prediction, we look at \(\mathbb{E}\left[ Y|X=x \right]\)
  • or \(Y\)’s distribution, etc.
• This is equivalent to picking out the sub-population where \(X=x\) and then calculating the average \(Y\)
• Genuine differences between sub-populations can be very far from causal
  • Sub-population of bright red cars is driven faster than the others
  • Sub-population of middle-aged men who have recently bought bright red cars is more likely to be recently divorced
  • Sub-population of people who have seen an ad for \(Z\) online are more likely to buy \(Z\) within the next month (say)
  • Sub-population of people whose friends became obese within the last year are more likely to become obese themselves
• Causality is counter-factual: what would happen if, hypothetically, something (or Someone) set \(X=x\), over-riding the usual processes?

Graphical Causal Models

• Every variable gets its own node in the graph or network
• There are directed edges or arrows from node to node
• The nodes you get arrows from are your parents; the nodes you send arrows to are your children
  • Parents’ parents, etc. = ancestors; childrens’ children, etc = descendants
• Every variable is a function of its parents and independent noise
• Assume no cycles (directed acyclic graph or DAG)
• Then: every variable is independent of its non-descendants given its parents (Markov property) \[ \mathbb{P}\left( V_1 = v_1, V_2=v_2, \ldots V_p=v_p \right) = \prod_{k=1}^{p}{\mathbb{P}\left( V_k = v_k| V_{\mathrm{Parents}(k)} = v_{\mathrm{Parents}(k)} \right)} \]

Some Time Series Models in DAG Form

• Markov chain:

• Hidden Markov model or state-space model:

• Observation-driven model (or “chain with complete connections”):

Expressing Causality in DAGs

• We want \(\mathbb{P}\left( V_i|do(V_j=x) \right)\), i.e., we want to know the distribution of \(Y\) if we somehow reached in and set \(V_j\) to the value \(x\), whatever \(V_j\) might normally be
• We calculate this from the graph by surgery:
  • Delete all the edges into \(V_j\)
  • Fix \(V_j\) at the value \(x\)
  • Leave everything else alone
  • Calculate the joint distribution in the new, surgically-altered graph
  • Add up over all the variables other than \(V_i\)
• Similarly if we set multiple variables at once
• Setting descendants of \(V_i\) has no effect on its distribution
  • But conditioning on descendants does give information about \(V_i\)

Graphical Models for Variables Evolving over Time

• We just looked at a few!
• Usually a good idea to unpack the variables, so things look more like this:

(from Chu and Glymour (2008), Fig. 1)

Graphical Models for Variables Evolving over Time

• Usually assume a constant, repetitive structure over time, so we can “roll up” the big graph into repetitions of a template:

(from Chu and Glymour (2008), Fig. 1)

So Where Does the Graph Come From?

• Intuition / dreams / visions
• Scientific theories
• Data analysis

Granger “Causality”: A Diversion

• Back in the 1960s, C. W. J. Granger proposed a test for causality between time series: \(X\) Granger-causes \(Y\) if \(Y(t)\) is better predicted by the past of \(Y\) and \(X\) than by the past of \(Y\) alone
• The original test used linear predictors but there are plenty of nonlinear variants, if you want them
  • I helped commit one a long time ago (Klinkner, Shalizi, and Camperi 2006)
• Granger was a great econometrician but this wasn’t one of his better ideas
• If \(Y\) really causes \(X\), but we measure \(Y\) imprecisely, \(X\) can Granger-cause \(Y\)!
• Granger causality is (IMHO) a waste of time, if you are interested in causality
  • It’s exactly what you want if you just want to predict…

What’s Really Needed Is Conditional Independence

• Remember: Any \(V\) is independent of its non-descendants, given its parents (Markov property)
• Implies: Some additional conditional independence relations (which can be read off from the graph by rules)
  • Basically: \(V_i \perp V_j | V_k\) if conditioning on \(V_k\) “blocks” all the paths between \(V_i\) and \(V_j\) (with complications in the rules about when paths are blocked or not)
• Assume: There are no other conditional independence relations (faithfulness to the graph)
• If we can find \(V_k\) where \(V_i \perp V_j | V_k\), then there can’t be an edge between \(V_i\) and \(V_j\)
• If nothing makes \(V_i\) and \(V_j\) independent, then there’s an edge between them (or there’s a hidden common cause)
• Start with edges between everything, then remove them if we can make variables independent by conditioning (Spirtes, Glymour, and Scheines 1993, 2001; Kalisch and Bühlmann 2007; Kalisch, Mächler, and Colombo 2010; Maathuis, Kalisch, and Bühlmann 2009; Colombo et al. 2012)
  • There are tricks for then getting the direction of the edges

Unfortunately, Conditional Independence Testing Is Hard

• Testing whether \(V_i \perp V_j | V_k\) is easy for categorical variables (\(\chi^2\) test)
• Testing whether \(V_i \perp V_j | V_k\) is easy if everything’s Gaussian and linear (regress \(V_i\) on \(V_j\) and \(V_k\), look at coefficient on \(V_j\))
  • Or do something more symmetric-looking with partial correlation
• The general case is very hard, though there are ways to do it (Zhang et al. 2011)

Additive Autoregressions

• Think back to our additive autoregressions from Lecture 24 \[ X(t+1) = \epsilon(t+1) + \alpha + \sum_{s=1}^{d}{f_i(X(t+1-s))} \]
• It’s easy to extend this to multiple variables, so we’d say something like \[ V_i(t+1) = \epsilon_i(t+1) + \alpha_i + \sum_{s=1}^{d}{\sum_{j=1}^{p}{f_{sj}(V_j(t+1-s)}} \] with \(f_{sj} = 0\) unless \(V_j(t+1-s)\) is a parent of \(V_i(t+1)\)
• Chu and Glymour (2008) gave a search procedure for causal graphs for time series, assuming an additive model is right

Additive Autoregressions

Summing Up Today

• Causal inference is about what-would-happen-if-this-were-changed, not about what-is-associated-with-this
• Causal structure is represented by graphical models
• Making this work over time involves “unrolling”
• Causal structure can be discovered, or constrained, from observations

Summing Up the Course

Spatiotemporal dependence is great

• It lets us predict there and then from here and now (or vice versa)
• It lets us learn about structure and process and how the world works
• The dependence is often what we most want to infer, statistically

Spatiotemporal dependence is a drag

• It messes up our basic statistical theory
• It (usually) reduces our effective sample size
• It forces us to learn more complicated theory and methods

Spatiotemporal dependence can be dealt with

• We need to recognize the dependence
• We need to build models of dependence
• We need to simulate those models