Statistics of Inequality and Discrimination

36-313, Fall 2022

Cosma Shalizi
Tuesdays and Thursdays, 1:25 -- 2:45 pm, Wean Hall (WEH) 5409

Many social questions about inequality, injustice and unfairness are, in part, questions about evidence, data, and statistics. This class lays out the statistical methods which let us answer questions like "Does this employer discriminate against members of that group?", "Is this standardized test biased against that group?", "Is this decision-making algorithm biased, and what does that even mean?" and "Did this policy which was supposed to reduce this inequality actually help?" We will also look at inequality within groups, and at different ideas about how to explain inequalities between and within groups. The class will interweave discussion of concrete social issues with the relevant statistical concepts.

Pre-requisites

36-202 ("Methods for Statistics and Data Science") (and so also 36-200, "Reasoning with Data")

Learning Objectives (accreditation bureaucrats look here)

By the end of the course, students will be be able to calculate, adjust, and communicate standard statistical measures of inequality within and between groups, and discuss their relation to social concepts of discrimination and disparity. More specifically, students will learn to

Course Mechanics

Lectures

Lectures will be used to amplify the readings, provide examples and demos, and answer questions and generally discuss the material. You will usually find the readings more rewarding if you do the readings before lecture, rather than after (or during).

No Recordings: I will not be recording lectures. This is because the value of class meetings lies precisely in your chance to ask questions, discuss, and generally interact. (Otherwise, you could just read a book.) Recordings interfere with this in two ways:

  1. They tempt you to skip class and/or to zone out and/or try to multi-task during it. (Nobody, not even you, is really any good at multi-tasking.) Even if you do watch the recording later, you will not learn as much from it as if you had attended in the first place.
  2. People are understandably reluctant to participate when they know they're being recorded. (It's only too easy to manipulate recordings to make anyone seem dumb and/or obnoxious.) Maybe this doesn't bother you; it doesn't bother me, much, because I'm protected by academic freedom and by tenure, but a good proportion of your classmates won't participate if they're being recorded, and that diminishes the value of the class for everyone.

Recording someone without their permission is illegal in many places, and more importantly is unethical everywhere, so don't make your own recordings of the class.

(Taking notes during class is fine and I strongly encourage it; taking notes forces you to think about what you are hearing and how to organize it, which vhelps you understand and remember the content.)

No textbooks, lots of readings

There is no one textbook which covers the material we'll go over at the required level. You will, instead, get very detailed lecture notes after each lecture. There will also be a lot of readings from various books and articles. (I will not agree with every reading I assign.)

You will see, when you look at the class schedule below, that there are usually no more than two (shorter) readings per class. There are also however a lot of optional readings. I don't expect you to do all those readings, but they do give you pathways to go deeper into particular subjects, to explore the history of ideas about some matter, or point you at related topics. You may notice that lots of the readings aren't about statistics; this is because doing good statistics about any subject requires knowing lots about the subject-matter.

Assignments

There are three reasons you will get assignments in this course. In order of decreasing importance:
  1. Practice. Practice is essential to developing the skills you are learning in this class. It also actually helps you learn, because some things which seem murky clarify when you actually do them, and sometimes trying to do something shows you what you only thought you understood.
  2. Feedback. By seeing what you can and cannot do, and what comes easily and what you struggle with, I can help you learn better, by giving advice and, if need be, adjusting the course.
  3. Evaluation. The university is, in the end, going to stake its reputation (and that of its faculty) on assuring the world that you have mastered the skills and learned the material that goes with your degree. Before doing that, it requires an assessment of how well you have, in fact, mastered the material and skills being taught in this course.

To serve these goals, there will be two kinds of assignment in this course.

After-class comprehension questions and exercises
Following every lecture, there will be a brief set of questions about the material covered in lecture. Sometimes these will be about specific points in the lecture, sometimes about specific aspects of the reading assigned to go with the lecture. These will be done electronically, and will be due the day after each lecture. These should take no more than 10 minutes, but will be untimed (so no accommodations for extra time are necessary). If the questions ask you to do any math (and not all of them will!), a scan or photograph of hand-written math is OK, so long as the picture is clearly legible. (Black ink or dark pencil on unlined white paper helps.)
Homework
Most weeks will have a homework assignment, divided into a series of questions or problems. These will have a common theme, and will usually build on each other. Each problem set will involve some combination of (very basic) statistical theory, (possibly less basic) calculations using the theory we've gone over, and analysis of real data sets using the methods discussed in class.
All homework will be submitted electronically through Gradescope. Most weeks, homework will be due at 6:00 pm on Thursdays (Pittsburgh time). Any exceptions will be clearly noted on the syllabus and at the beginning of the assignment. When this results in less than seven days between an assignment's due date and the previous due date, the homework will be shortened.

Time Expectations

You should expect to spend 5--7 hours on assignments every week, averaging over the semester. (This follows from the university's rules about how course credits translate into hours of student time.) If you find yourself spending significantly more time than that on the class, please come to talk to me.

Grading

Grades will be broken down as follows:

You can submit assignments as many times as you like; the last version you submit is the one that will be graded. Submit early, submit often.

Grade boundaries will be as follows:
A [90, 100]
B [80, 90)
C [70, 80)
D [60, 70)
R < 60

To be fair to everyone, these boundaries will be held to strictly.

Grade changes and regrading: If you think that particular assignment was wrongly graded, tell me as soon as possible. Direct any questions or complaints about your grades to me; the teaching assistants have no authority to make changes. (This also goes for your final letter grade.) Complaints that the thresholds for letter grades are unfair, that you deserve a higher grade, that you need a higher grade, etc., will accomplish much less than pointing to concrete problems in the grading of specific assignments.

As a final word of advice about grading, "what is the least amount of work I need to do in order to get the grade I want?" is a much worse way to approach higher education than "how can I learn the most from this class and from my teachers?".

Canvas, Gradescope and Piazza

Homework will be submitted electronically through Gradescope. Canvas will be used as a calendar showing all assignments and their due-dates, to distribute some readings, and as the official gradebook.

We will be using Piazza for question-answering. You will receive an invitation within the first week of class. Anonymous-to-other-students posting of questions and replies will be allowed, at least initially. Anonymity will go away for everyone if it is abused. During Piazza office hours, someone will be online to respond to questions (and follow-ups) in real time. You are welcome to post at any time, but outside of normal working hours you should expect that the instructors have lives.

Office Hours

TBD
During Piazza office hours, I'll be checking the site continually, and responding ASAP, so you can get very quick feedback, and there's a record which you (and others in the class) can consult later.

Collaboration, Cheating and Plagiarism

Except for explicit group exercises, everything you turn in for a grade must be your own work, or a clearly acknowledged borrowing from an approved source; this includes all mathematical derivations, computer code and output, figures, and text. Any use of permitted sources must be clearly acknowledged in your work, with citations letting the reader verify your source. You are free to consult the textbooks and recommended class texts, lecture slides and demos, any resources provided through the class website, solutions provided to this semester's previous assignments in this course, books and papers in the library, or legitimate online resources, though again, all use of these sources must be acknowledged in your work. (Websites which compile course materials are not legitimate online resources.)

In general, you are free to discuss homework with other students in the class, though not to share or compare work; such conversations must be acknowledged in your assignments. You may not discuss the content of assignments with anyone other than current students, the instructors, or your teachers in other current classes at CMU, until after the assignments are due. (Exceptions can be made, with prior permission, for approved tutors.) You are, naturally, free to complain, in general terms, about any aspect of the course, to whomever you like.

Any use of solutions provided for any assignment in this course, or in other courses, in previous semesters is strictly prohibited. This prohibition applies even to students who are re-taking the course. Do not copy the old solutions (in whole or in part), do not "consult" them, do not read them, do not ask your friend who took the course last year if they "happen to remember" or "can give you a hint". Doing any of these things, or anything like these things, is cheating, it is easily detected cheating, and those who thought they could get away with it in the past have failed the course. Even more importantly: doing any of those things means that the assignment doesn't give you a chance to practice; it makes any feedback you get meaningless; and of course it makes any evaluation based on that assignment unfair.

If you are unsure about what is or is not appropriate, please ask me before submitting anything; there will never be a penalty for asking. If you do violate these policies but then think better of it, it is your responsibility to tell me as soon as possible to discuss how to rectify matters. Otherwise, violations of any sort will lead to severe, formal disciplinary action, under the terms of the university's policy on academic integrity.

On the first day of class, you will be assigned a "homework 0" on the content of these policies. This assignment will not factor into your grade, but you must complete it before you can get any credit for any other assignment.

Accommodations for Students with Disabilities

If you need accommodations for physical and/or learning disabilities, please contact the Office of Disability Resources, via their website, www.cmu.edu/disability-resources. They will help you work out an official written accommodation plan, and help coordinate with me. Discussion is vital to learning

Inclusion and Respectful Participation

The university is a community of scholars, that is, of people seeking knowledge. All of our accumulated knowledge has to be re-learned by every new generation of scholars, and re-tested, which requires debate and discussion. Everyone enrolled in the course has a right to participate in the class discussions. This doesn't mean that everything everyone says is equally correct or equally important, but does mean that everyone needs to be treated with respect as persons, and criticism and debate should be directed at ideas and not at people. Don't dismiss (or credit) anyone in the course because of where they come from, and don't use your participation in the class as a way of shutting up others. Don't be rude, and don't go looking for things to be offended by. Statistical methods don't usually lead to heated debate, but the subjects to which we'll apply the methods notoriously do. If someone else is saying something you think is really wrong-headed, and you think it's important to correct it, address why it doesn't make sense, and listen if they give a counter-argument.

The classroom is not a democracy; as the teacher, I have the right and the responsibility to guide the discussion in what I judge are productive directions. This may include shutting down discussions which are not helping us learn about statistics, even if those discussions might be important to have elsewhere. (You can have them elsewhere.) I will do my best to guide the course in a way which respects everyone's dignity as a human being, as a scholar, and as a member of the university.

Detailed course calendar

Links to lecture notes, assignments, etc., will go here as they become relevant.

Readings will be finalized a week before each course meeting. Links on readings point to electronic versions accessible through the university library. (You may need to authenticate yourself with the library and/or use the VPN, if you're trying to access them from off campus.) Optional readings really are optional, but the non-optional ones really are not optional. Readings marked with one or more stars (*) are, as it were, especially optional, because of some combination of being long, difficult, old, etc.

The order of topics after about October 15 is currently somewhat tentative. The due dates for assignments, however, are fixed.

Lecture 1 (Tuesday, 30 August): Introduction to the course

There was a certain rich man, which was clothed in purple and fine linen, and fared sumptuously every day; and there was a certain beggar named Lazarus, which was laid at his gate, full of sores, and desiring to be fed with the crumbs which fell from the rich man's table; moreover the dogs came and licked his sores

Lecture 2 (Thursday, 1 September): Describing income and wealth inequality within a single population

Lecture 3 (Tuesday, 6 September): Income and wealth inequality: modeling I

Lecture 4 (Thursday, 8 September): Modeling wealth and income distributions II

Lecture 5 (Tuesday, 13 September): Speed-run through social and economic stratification CANCELED

Lecture 6 (Thursday, 15 September): Speed-run through social and economic stratification

Lecture 7 (Tuesday, 20 September) Income (and wealth) disparities: comparing central tendencies and typical values

Lecture 8 (Thursday, 22 September) Income (and wealth) disparities: Comparing whole distributions

Lecture 9 (Tuesday, 27 September): Explaining, or explaining away, inequality I

Lecture 10 (Thursday, 29 September): Explaining, or explaining away, inequality II

Lecture 11 (Tuesday, 4 October): Detecting and interpreting inequalities in hiring, admissions, etc.

Lecture 12 (Thursday, 6 October): Inequalities in health, disease and mortality

Lecture 13 (Tuesday, 11 October): Mobility and Transmission of Inequality

Lecture 14 (Thursday, 13 October): Measuring Spatial Segregation and Its Consequences

  • Homework:

    Tuesday, 18 October and Thursday, 20 October: NO CLASS

    Enjoy fall break!

    Lecture 15 (Tuesday, 25 October): Algorithmic Bias and/or Fairness

    Lecture 16 (Thursday, 27 October): Algorithmic Fairness continued

    Trade-offs between different forms of fairness. Trade-offs between forms of fairness and accuracy. Techniques for mitigating algorithmic unfairness: changing the estimation procedure; changing the data. Some critiques of these notions of "fairness".
  • Notes; See notes from last time.
  • Reading:
  • Homework:

    Lecture 17 (Tuesday, 1 November): Admissions tests

    Lecture 18 (Thursday, 3 November): Intelligence tests

    Tuesday, 8 November: NO CLASS

    Lecture 19 (Thursday, 10 November) Measuring attitudes and prejudice

    Lecture 20 (Tuesday, 15 November): Evaluating inequality-reducing interventions I

    Lecture 21 (Thursday, 17 November): Evaluating interventions II

    Lecture 22 (Tuesday, 22 November): Policing and Crime

    Thursday, 24 November: NO CLASS

    Happy Thanksgiving!

    Lecture 23 (Tuesday, 29 November): Self-organizing, structural and/or systemic inequalities

    Lecture 24 (Thursday, 1 December): Statistics and its history

    Lecture 25 (Tuesday, 6 December): How do we know what we do about inequalities?

    Lecture 26 (Thursday, 8 December) Review of the course

    (Most of the illustrations are from the great German-American artist George Grosz, via ARTSTOR. Clicking on any of the images will take you to its source.)