Data over Space and Time (36-740)

Fall 2026, Mini 1

Overview

This course is a survey of methods for spatial, temporal, and spatio-temporal statistics, with an emphasis on tools which can be applied in all three domains. The first part of the course will focus on linear predictors and covariance functions (which are inter-related), and the second on approximate methods for non-linear models, including resampling. Assignments will be a mix of theory and computational analysis of real data.

Prereqs

There are no formal pre-requisites, but students are assumed to have a good grasp of probability and mathematical statistics, at the level of Wasserman's All of Statistics, Casella and Berger, or Davison, and comfort working with vectors and matrices, including eigendecompositions. You should also be comfortable with at least one system for numerical statistical computing, either R or the appropriate parts of Python.

If you are unsure of whether you have the right background, please ask.

740 vs. 620

740 is for students in the Ph.D. program in statistics; MS. students, and students from other departments, should register for 620. The assignments will be somewhat different for the two classes.

Goals and Learning Outcomes

(Accreditation officials look here)

The goal of this class is to train you in using statistical models and methods to analyze interdependent data spread out over space, time, or both, using the models as data summaries, as predictive instruments, and as tools for scientific inference. We will build on the theory of statistical inference for independent data taught in earlier statistics courses, and complement the theory and applications of the linear model. After taking the class, when you're faced with a new temporal, spatial, or spatio-temporal data-analysis problem, you should be able to (1) describe the statistical challenges the problem presents, (2) select appropriate methods, (3) use statistical software to implement those methods, (4) critically evaluate the resulting statistical models, and (5) communicate the results of your analyses to collaborators and to non-statisticians.

Course Mechanics

Lectures

Do not record lectures. Exceptions to this will be made only for students with written accommodation plans, authorized by CMU's Office of Disability Resources, requiring recordings.

(The value of class meetings lies precisely in your chance to ask questions, discuss, and generally interact. Recordings interfere with this in two ways:

1. They tempt you to skip class and/or to zone out and/or try to multi-task during it. (Nobody is really any good at multi-tasking.) Even if you do watch the recording later, you will not learn as much from it as if you had attended in the first place.
2. People are understandably reluctant to participate when they know they're being recorded. (It's only too easy to manipulate recordings to make anyone seem dumb and/or obnoxious.) Maybe this doesn't bother you; it doesn't bother me, much, because I'm protected by academic freedom and by tenure, but a good proportion of your classmates won't participate if they're being recorded, and that diminishes the value of the class for everyone.

Taking notes during class is strongly encouraged; taking notes forces you to think about what you are hearing and how to organize it, which helps you understand and remember the content.

Office Hours

Times TBD; office hours will be held in my actual office, Baker Hall 229C.

If you cannot make the regular office hours, or have concerns you'd rather discuss privately (e.g., grades), please e-mail me to make an appointment.

Textbooks

Gidon Eshel, Spatiotemporal Data Analysis (Princeton, New Jersey: Princeton University Press, 2011, ISBN 978-0-691-12891-7)

Full text available (as PDFs of chapters) through JSTOR.

Bärbel Finkenstädt, Leonhard Held and Valerie Isham (eds.), Statistical Methods for Spatio-Temporal Systems (Boca Raton, Florida: Chapman & Hall / CRC Press, 2007, ISBN 978-1-584-88593-1)

No electronic access through the library; scans will be made available via Canvas as needed

Peter Guttorp, Stochastic Modeling of Scientific Data (Boca Raton, Florida: Chapman & Hall / CRC Press, 1995, ISBN 978-0-412-99281-0)

No electronic access through the library; scans will be made available via Canvas as needed.

Assignments

1. Practice. Practice is essential to developing the skills you are learning in this class. It also actually helps you learn, because some things which seem murky clarify when you actually do them, and sometimes trying to do something shows what you only thought you understood.
2. Feedback. By seeing what you can and cannot do, and what comes easily and what you struggle with, I can help you learn better, by giving advice and, if need be, adjusting the course.
3. Evaluation. The university is, in the end, going to stake its reputation (and that of its faculty) on assuring the world that you have mastered the skills and learned the material that goes with your degree. Before doing that, it requires an assessment of how well you have, in fact, mastered the material and skills being taught in this course.

To serve these goals, there will be four kinds of assignment in this course.

Homework

Most weeks will have a homework assignment, divided into a series of questions or problems. These will have a common theme, and will usually build on each other, but different problems may involve statistical theory, analyzing real data sets on the computer, and communicating the results.

All homework will be submitted electronically through Gradescope. Homework will be due at 6:00 pm on Thursdays.

Scribing

Starting with Lecture 3 (in week 2), at every lecture, one or more students will act as "scribes", taking their own notes on the lecture, and then turning that into a written document, to share with the class, within a week. Students will sign up to scribe on specific dates, on a first-come, first-served basis. (If you can't scribe on a date you signed up for, contact me ASAP to re-schedule.) Scribes will also have access to my slides, and the grading rubric will be posted by the end of week 1.

If some people end up having to scribe more than others, the extra work will count as extra credit.

Research questions

Students in 740 will work on developing a research question related to the course, in a short document 1--5 pages. This question can relate either to our methods themselves, or to their application to a concrete problem of data analysis. This is solely about developing, and refining, the question, not about doing the research to answer it. Students will submit a first draft, receive feedback within a week, and then submit a second draft at the end of the course.

Final oral exam

During the last week of the course, students will meet one-on-one with me for an oral exam of no more than 20 minutes. During this exam, students will be initially asked to describe the mathematical basis of one of the methods we have gone over, or how it could be applied to a concrete problem, with further questions following from there. At least one week before the exam, I will provide a list of the (8--12) topics you might be asked about. The exam will be open notes and open book, but with no computing devices, since it is supposed to assess your understanding, and your ability to communicate that understanding. (There will be no trick questions.) The exam will take place during our regular class time, with 20-minute slots reserved on a first-come, first-served basis.

Time Expectatons

Grading

Grade boundaries will be as follows:

A	[90, 100]
B	[80, 90)
C	[70, 80)
D	[60, 70)
R	< 60

To be fair to everyone, these boundaries will be held to strictly.

The grade components will be broken down as follows:

• Homework: 60%. The lowest 2 grades will be replaced by 80/100, no questions asked (unless your lowest grades are already over 80/100). This includes 0s for not turning in anything.
• Final exam: 20%.
• Scribing: 20% for 620, 10% for 740.
• Research question notes: 10% for 740, 0% for 620.

No late work will be accepted for any reason. (That's what replacing your lowest grades is for.)

Grade changes and regrading: If you think that particular assignment was wrongly graded, tell me as soon as possible. Direct any questions or complaints about your grades to me; the teaching assistants have no authority to make changes. (This also goes for your final letter grade.) Complaints that the thresholds for letter grades are unfair, that you deserve a higher grade, etc., will accomplish much less than pointing to concrete problems in the grading of specific assignments.

As a final word of advice about grading, "what is the least amount of work I need to do in order to get the grade I want?" is a much worse way to approach higher education than "how can I learn the most from this class and from my teachers?".

Collaboration, Cheating and Plagiarism

Except for explicit group exercises, everything you turn in for a grade must be your own work, or a clearly acknowledged borrowing from an approved source; this includes all mathematical derivations, computer code and output, figures, and text. Any use of permitted sources must be clearly acknowledged in your work, with citations letting the reader verify your source. You are free to consult the textbooks and recommended class texts, lecture slides and demos, any resources provided through the class website, solutions provided to this semester's previous assignments in this course, books and papers in the library, or legitimate online resources, though again, all use of these sources must be acknowledged in your work. (Websites which compile course materials are not legitimate online resources.)

In general, you are free to discuss homework with other students in the class, though not to share or compare work; such conversations must be acknowledged in your assignments. You may not discuss the content of assignments with anyone other than current students, the instructors, or your teachers in other current classes at CMU, until after the assignments are due. (Exceptions can be made, with prior permission, for approved tutors.) You are, naturally, free to complain, in general terms, about any aspect of the course, to whomever you like.

Any use of solutions provided for any assignment in this course, or in other courses, in previous semesters is strictly prohibited. This prohibition applies even to students who are re-taking the course. Do not copy the old solutions (in whole or in part), do not "consult" them, do not read them, do not ask your friend who took the course last year if they "happen to remember" or "can give you a hint". Doing any of these things, or anything like these things, is cheating, it is easily detected cheating, and those who thought they could get away with it in the past have failed the course. Even more importantly: doing any of those things means that the assignment doesn't give you a chance to practice; it makes any feedback you get meaningless; and of course it makes any evaluation based on that assignment unfair.

If you are unsure about what is or is not appropriate, please ask me before submitting anything; there will never be a penalty for asking. If you do violate these policies but then think better of it, it is your responsibility to tell me as soon as possible to discuss how to rectify matters. Otherwise, violations of any sort will lead to severe, formal disciplinary action, under the terms of the university's policy on academic integrity.

Using AI is Discouraged

AI tools have their place (maybe), but they are most useful when they are doing boring and repetitive, "mechanical" work which you can check. The point of assignments in a class like this is to help you acquire the knowledge and skills you need to evaluate the work of someone else --- or of something else, like the output of an AI. Therefore, using generative AI for any part of assignments in this class which require you to think is actively harmful to your own learning. (This is backed up by professional studies of the effect of AI on student learning, which I am happy to discuss at length.)

I recognize that it is going to be very hard to stop you from using these machines to do mechanical tasks (like writing boilerplate code or graph formatting). I would also be failing in my duty as a teacher if I let you replace your own thought and learning with a random sample of the lowest common denominator of the Internet. Since drawing a bright line between those two extremes is hard, I am willing, on a trial basis, to permit the use of generative AI tools in this class, on the following conditions:

1. Whenever possible, don't use AI. (You're at a university, try to learn for yourself.)
2. If you do use AI on an assignment, you must acknowledge it, and include a complete transcript of your session(s) with the AI. (Put this at the end, after the main body of your assignment.)
3. If I judge that you are over-relying on the AI, you will receive feedback to that effect, and your grade will be marked down for that assignment. (My decisions on this are final.)
4. Using AI without acknowledgment will be treated as plagiarism.
5. I reserve the right to forbid its use for everyone altogether.

Finally, if you absolutely must use these tools in this course, this seems like a good way to do it.

Accommodations for Students with Disabilities

Inclusion and Respectful Participation

The university is a community of scholars, that is, of people seeking knowledge. All of our accumulated knowledge has to be re-learned by every new generation of scholars, and re-tested, which requires debate and discussion. Everyone enrolled in the course has a right to participate in the class discussions. This doesn't mean that everything everyone says is equally correct or equally important, but does mean that everyone needs to be treated with respect as persons, and criticism and debate should be directed at ideas and not at people. Don't dismiss (or enhance) anyone in the course because of where they come from, and don't use your participation in the class as a way of shutting up others. (Don't be rude, and don't go looking for things to be offended by.) While methods for spatio-temporal data analysis don't usually lead to heated debate, some of the subjects we'll be applying them to might. If someone else is saying something you think is really wrong-headed, and you think it's important to correct it, address why it doesn't make sense, and listen if they give a counter-argument.

The classroom is not a democracy; as the teacher, I have the right and the responsibility to guide the discussion in what I judge are productive directions. This may include shutting down discussions which are not helping us learn about statistics, even if those discussions are important to have elsewhere. I will do my best to guide the course in a way which respects everyone's dignity as a human being and as a member of the university.

Lecture / Reading / Assignment Schedule

Readings marked with a star (*) are optional, because they're more peripheral, demand more mathematical or scientific background, and/or simply old. Readings marked with more than one star are, as it were, especially optional.

Topics after Lecture 9 are currently somewhat provisional, and may change (with plenty of notice); assignment due dates will not change.

Tuesday, 25 August (Lecture 1): Introduction to the Course

• Processes which unfold over space and/or time give rise to dependent data. Probabilistic dependence means that what happens here and now is related to, and gives us predictive information about, what happens there and then; we would like to study those relations and use that predictive information. Alas, the most basic forms of statistical theory and methods, about independent and identically distributed observations, can only look at this and say "Well, that's a single really high-dimensional observation, can't say anything about it. Have you tried resetting the world and running it again multiple times?" The subject of this course is methods which can break this impasse, by giving us tools to learn relations in the face of dependence. A concrete and cheerful example --- the history of cherry blossoms in Kyoto since the 800s --- gives us a first taste of the tasks we would like to do: interpolation, extrapolation, smoothing, noise reduction, trend identification, event detection, uncertainty quantification. All of this will follow in due course, beginning with the linked subjects of smoothing and trends.
• Reading:
  • Guttorp, Chapter 1 (on Canvas)
  • Eshel, Chapters 7 and 8
  • For students in 740, who should start thinking about research questions:
    • Kate L. Turabian (et al.), A Manual for Writers of Research Papers, Theses and Dissertations (8th edition, Chicago: University of Chicago Press, 2013), chs. 1 and 2 (on Canvas)
    • Peter Medawar, "Is the Scientific Paper a Fraud?", originally delivered as a public lecture in 1963, reprinted as ch. 3 of The Strange Case of the Spotted Mice, and Other Classic Essays on Science (Oxford: Oxford University Press, 1996) [PDF]
• Slides (.Rmd)

Thursday, 27 August (Lecture 2): Smoothing, Trends, Detrending

• Smoothing by local averaging. The idea of a trend, and de-trending. Smoothing as exploratory data analysis. Some of the math of smoothing: the influence (or "hat") matrix, degrees of freedom. Expanding in eigenvectors. Residuals after de-trending as estimates of the fluctuations. The Yule-Slutsky effect. Picking how much to smooth by cross-validation. (We will revisit cross-validation.) Special considerations for ratios.
• Reading:
  • Handout: "Smoothing, Trends, Detrending" (PDF, Rmd)
  • (*) Karen Kafadar, "Smoothing Geographical Data, Particularly Rates of Disease", Statistics in Medicine 15 (1996): 2539--2560
• Slides (Rmd), incorporating some discussion from lecture
• Assignments:
  • Homework 1: assignment, smoother.matrix.R

Tuesday, 1 September (Lecture 3): Expanding in Basis Functions I --- Principal Components Analysis

• The goal of principal components is, at a high level, to approximate the original data (which is complicated and messy) as a linear combination of a few vectors or "components", i.e., to approximate the data by a linear sub-space. Setting thjs up as a problem of minimizing mean squared error leads to the conclusion that we get the same result by finding the directions through the data with maximum variance. We can in turn find those by finding the eigenvectors of the data's variance matrix. We can apply this to multivariate spatial data, to multivariate temporal data, and to spatio-temporal data. Because PCA is a purely geometric technique, it does not assume a model and we can always do it, but by the same token it's not, generally, inferential statistics.
• Reading:
  • Eshel, chapter 11 (skipping 11.8 and 11.11--11.12)
  • (*) sec. 3 of Fuentes, Guttorp, and Sampson, "Using Transforms to Analyze Space-Time Processes", ch. 3 in Finkenstädt et al. [PDF preprint]
  • (*) Luigi L. Cavalli-Sforza, Paolo Menozzi and Alberto Piazza, "Demic Expansions and Human Evolution", Science 259 (1993): 639--646 [JSTOR]
  • (*) Luigi L. Cavalli-Sforza, Genes, Peoples, and Languages (New York: North Point Press, 2000)
  • (***) Luigi L. Cavalli-Sforza, Paolo Menozzi and Alberto Piazza, The History and Geography of Human Genes (Princeton: Princeton University Press, 1994)
  • (**) Karl Pearson, "On lines and planes of closest fit to systems of points in space", Philosophical Magazine 2 (series 6) (1901): 559--572
  • (**) Harold Hotelling, "Analysis of a complex of statistical variables into principal components", parts 1 and 2, Journal of Educational Psychology 24 (1933), 417--441 and 498--520
  • (***) Michel Loéve, Probability Theory, 1st edition (New York: D. Van Nostrand Company, 1955), ch. X
• Slides (.Rmd)

Thursday, 3 September (Lecture 4): Expanding in Basis Functions II --- Fourier Analysis

• Analyzing complicated, irregular functions into linear combinations of simple, periodic functions. Why symmetry under translation suggests using sines and cosines as our basis functions. The Fourier transform and the inverse Fourier transform. Convolution and smoothing. Connection between the Fourier transform and the autocovariance function for stationary processes (Wiener-Khinchin theorem). Estimating the power spectrum. Low-pass filtering and smoothing. Simulating new processes from the Fourier transform.
• Reading:
  • sec. 1 of Fuentes, Guttorp, and Sampson, "Using Transforms to Analyze Space-Time Processes", ch. 3 in Finkenstädt et al. [PDF preprint]
• Assignments:
  • Homework 1 due
  • Homework 2: assignment, soccomp.irep1.csv data file
• Slides (.Rmd)

Tuesday, 8 September (Lecture 5): Optimal Linear Prediction, Especially Over Time

• Mathematics of prediction. Mathematics of optimal linear prediction, in any context whatsoever. Ordinary least squares as an estimator of the optimal linear predictor. The PCA view. The Karhunen-Loeve view (a.k.a. "It's really all much simpler once you introduce a reproducing-kernel Hilbert space".) Applying the linear-predictor idea to time series: interpolating between observations; extrapolating into the future (or past). The concept of stationarity. Auto- and cross- covariance functions and their estimation. Covariance functions as EDA. Removing trends; stationary fluctuations after detrending. Historical notes: Wiener and Kolmogorov.
• Reading:
  • Eshel, section 9.5 (skipping 9.5.3 and 9.5.4)
  • (**) Emanuel Parzen, "An Approach to Time Series Analysis", The Annals of Mathematical Statistics 32 (1961): 951--989
  • (****) Norbert Wiener, Extrapolation, Interpolation and Smoothing of Stationary Time-Series: with Engineering Applications (Cambridge, Massachusetts: The Technology Press, 1949 [but originally published as a classified technical report, National Defense Research Council, 1942])
  • (*****) A. N. Kolmogorov, "Interpolation und Extrapolation von stationären zufälligen Folgen", Izvestiya Akademii Nauk SSSR. Seriya Matematicheskaya 5 (1941): 3--14 [In Russian; translated as "Interpolation and extrapolation of stationary random sequences", in Selected Works of A. N. Kolmogorov, volume II, Probability Theory and Mathematical Statistics (ed. A. N. Shiryayev; Dordrecht: Kluwer Academic, 1992), pp. 272--280]
• Slides (corrected and amplified after lecture); .Rmd

Thursday, 10 September (Lecture 6): Optimal Linear Prediction over Space and Space-Time

• Applying the linear-predictor idea to data spread over space or over space and time ("kriging"). The importance of estimating covariance between spatial locations. Assumptions restricting the form of the covariance and so enabling estimation: stationarity, isotropy, separability. Estimating parametric covariance functions. "Variograms". Historical notes: Krige and Matheron.
• Reading:
  • Gneiting, Genton, and Guttorp, "Geostatistical Space-Time Models, Stationarity, Seperability, and Full Symmetry", ch. 4 in Finkenstädt et al. [PDF preprint
  • Optional reading:
    • (*) Brian D. Ripley, Spatial Statistics (New York: Wiley, 1981), ch. 4 and (less importantly) 5
    • (***) D. G. Krige, Lognormal-de Wijsian Geostatistics for Ore Evaluation (2nd edition; Johannesburg: South African Institute of Mining and Metallurgy, 1981)
    • (**) Georges Matheron, Matheron's Theory of Regionalised Variables (Based on lecture notes from 1970, translated from French into English by Charles Huijbregts, edited by Vera Pawlowsky-Glahn and Jean Serra; Oxford: Oxford University Press, 2019)
• Slides (.Rmd)
• Assignments:
  • Homework 2 due
  • Homework 3: assignment

Tuesday, 15 September (Lecture 7): Separating Signal and Noise with Linear Methods

• Observational noise: using the linear-predictor idea to remove observational noise, a.k.a. "the Wiener filter". The myseriously-named "nugget effect" (accounting for measurement noise that's not auto-correlated). Periodicity: noticing periodicity from time series; from autocorrelation functions. Extracting periodic components with a known period by averaging. "Climate" and "anomaly". Seasonal adjustment of time series.
• Reading:
  • (****) Norbert Wiener, Extrapolation, Interpolation and Smoothing of Stationary Time-Series: with Engineering Applications (Cambridge, Massachusetts: The Technology Press, 1949 [but originally published as a classified technical report, National Defense Research Council, 1942])
• Slides (.Rmd)

Thursday. 17 September (Lecture 8): Factor Models, and Linear State-Space Models

• Factor models: observables are linear in a low-dimensional latent (the "factor"), plus noise. Implications for the covariance matrix and covariance estimation. Factor models and linear state-space models. Linear dynamical systems (=more fun with eigenvalues and eigenvectors). Kalman filter (= recursive form of the Wiener filter). Factor models for high-dimensional spatio-temporal data.
• Assignments:
  • Homework 3 due
  • First draft of research-question note due (36-740 only)
  • Homework 4 assigned
• Slides (.Rmd)

Tuesday, 22 September (Lecture 9): Nonlinear State-Space Models

• Nonlinearity in the state dynamics and/or in the observation function. The theoretical solution to the filtering / state-estimation problem. Particle filters as numerical approximations. Alternatively: "geometry from a time series" / "state-space reconstruction".

Thursday, 24 September (Lecture 10): Prediction Processes

• The predictive distribution of the future, given the whole past. Predictive states. Properties of the prediction process. Reconstruction algorithms. Local predictive states for spatio-temporal processes.
• Assignments:
  • Homework 4 due
  • Homework 5 assigned
  • Feedback on initial research-question notes returned (740 only)

Tuesday, 29 September (Lecture 11): Point Processes

• A point process is a random process which places distinct events at points in continuous time --- or space, or space-time. (If there is more than one kind of event, we get a marked point process; the mark on a point tells us what kind of event it was.) We will approach point processes as the limit of binary-valued discrete-time processes, which makes it easy to see why the most basic point process is the homogeneous Poisson process. It will also make it easy to branch out to inhomogeneous Poisson processes, and the concept of an intensity function, which can be estimated via smoothing (among other ways). The intensity function lets us do model checking, by re-scaling to a homogeneous Poisson process. From there we will look at second-order descriptive statistics, and some models which are not Poisson processes, and their estimation. Also, some applications. If any time is left over, we'll glance at the more mathematical view of point processes, as discrete random measures.
• Reading:
  • Guttorp, ch. 5

Thursday, 1 October (Lecture 12): Cross-Validation and Resampling over Space and Time

• Cross-validation and the bootstrap have been key parts of statistics for IID data since the 1970s, but they both get more complicated for dependent data. The issue with leave-one-out cross-validation is that if we just hold out one data point, it's dependent on all the neighboring points, which were in the training set, so we're not really looking at generalization. (Similarly for k-fold CV.) There are several different proposed solutions, involving placing buffers (of various sizes and shapes) around the testing set; we'll look at how well they do. Turning to bootstrapping, model-based bootstraps, where we simulate from a fitted model, have no special issues with dependent data, but resampling certainly does. Randomly sampling the original data points with replacement generates a realization of an IID process, which is not at all what we want. For time series, the usual solution is to resampling contiguous blocks of data points; space is a bit more complicated. We will close by looking at bootstrap estimates of generalization error, as an alternative to cross-validation.
• Reading (to be prioritized by the instructor before we get here):
  • Prabir Burman, Edmond Chow and Deborah Nolan, "A Cross-Validatory Method for Dependent Data", Biometrika 81 (1994): 351--358
  • Jeff Racine, "Feasible Cross-Validatory Model Selection for General Stationary Processes", Journal of Applied Econometrics 12 (1997): 169--179 [JSTOR]
  • Jeff Racine, "Consistent cross-validatory model-selection for dependent data: $hv$-block cross-validation", Journal of Econometrics 99 (2000): 39--61
  • S. N. Lahiri, Resampling Methods for Dependent Data (New York and Berlin: Springer-Verlag, 2003)
  • Elizaveta Levina and Peter J. Bickel, "Texture Synthesis and Nonparametric Resampling of Random Fields", Annals of Statistics 34 (2006): 1751--1773
• Assignments:
  • Homework 5 due

Tuesday, 6 October: Final Exams I

• In Baker Hall 229C, as scheduled

Thursday, 8 October: Final Exams II

• In Baker Hall 229C, as scheduled
• Assignments:
  • Second draft of research question note due (740 only)

Image credits: Pictures on this page are from my teacher David Griffeath's Particle Soup Kitchen website, except for Umberto Boccioni's Riot in the Galleria.