# Data sources — MaDS Fall 2026 storm/weather project

Every dataset used in either course, with a stable link, the access method, update
cadence, and licensing. **Reproducibility rule for the project: nothing goes into a
notebook, query, or slide without a row in this table.** When you add a source,
record the exact URL and the file's "constant" name (NOAA appends a processing-date
stamp like `c20260421` that changes on each refresh — note both).

---

## Primary datasets

| Dataset | What it is | Link | Access | Update cadence | License |
|---|---|---|---|---|---|
| **NOAA Storm Events** | Per-event storm records (type, location, damage $, casualties, magnitude, narrative). 1950–present. | https://www.ncdc.noaa.gov/stormevents/ — bulk CSV index: https://www.ncei.noaa.gov/pub/data/swdi/stormevents/csvfiles/ | Bulk gzipped CSV, one file per year (`details`, `locations`, `fatalities`). | Monthly | U.S. Gov public domain |
| **HURDAT2 (Atlantic)** | Atlantic hurricane best-track database — 6-hourly storm fixes (lat/lon, max wind, pressure). 1851–present. | https://www.nhc.noaa.gov/data/#hurdat — current file: https://www.nhc.noaa.gov/data/hurdat/hurdat2-1851-2023-051124.txt | Single plain-text file. | After each season (post-Nov) | U.S. Gov public domain |
| **GHCN-Daily** | Global land surface station observations (TMAX/TMIN, precip, snow). | https://www.ncei.noaa.gov/products/land-based-station/global-historical-climatology-network-daily | Bulk (`.dly` / per-station CSV) + by-year CSV. | Daily | U.S. Gov public domain |
| **NWS API** | Live forecasts, active alerts, observations. | https://api.weather.gov/ | REST/JSON, no key. | Real-time | U.S. Gov public domain |

### File-naming note (NOAA Storm Events)
`StormEvents_details-ftp_v1.0_d2024_c20260421.csv.gz`
- `d2024` = **data year** (constant — this is what you pin to).
- `c20260421` = **processing/creation date** (changes every monthly refresh; do **not** hard-code it — glob `*_d2024_*.csv.gz` or look it up from the index).

We have committed `data/StormEvents_details-ftp_v1.0_d2024_c20260421.csv.gz` (12.7 MB,
69,801 events) as the working subset for the proof-of-concept.

---

## Join / enrichment datasets (for the "what can we do with it" story)

| Dataset | Why join it | Link | Access | Key? |
|---|---|---|---|---|
| **Census ACS 5-year** | County/tract population, median income, median home value → turns raw damage into *exposure* and *vulnerability* per capita. | https://www.census.gov/data/developers/data-sets/acs-5year.html — API base: https://api.census.gov/data/2022/acs/acs5 | REST/JSON | Free key: https://api.census.gov/data/key_signup.html |
| **Census TIGER/Line** | County & tract polygons → spatially assign each storm point to a county/tract. | https://www.census.gov/geographies/mapping-files/time-series/geo/tiger-line-file.html | Shapefile / GeoJSON | No |
| **Zillow ZHVI** | Typical home value by county/ZIP, monthly → dollar-exposure proxy where assessor data is unavailable. | https://www.zillow.com/research/data/ | Bulk CSV | No |
| **xView2 (xBD)** | Pre/post-disaster satellite imagery with building-level damage labels (no/minor/major/destroyed). Ground-truth for storm damage. | https://xview2.org/dataset | Bulk download (registration) | Registration |
| **Census geocoder** | Free batch lat/lon → county/tract FIPS (alternative to a spatial join). | https://geocoding.geo.census.gov/geocoder/ | REST/JSON, batch CSV | No |

> **xView2 vs. xView3:** xView2 (xBD) is building-damage from *disasters incl. storms* —
> the right fit here. xView3 is *dark-vessel / maritime* detection (not damage); noted so
> we don't send students down the wrong path.

---

## How each course uses these

- **datavis-36613** → Storm Events as the core narrative dataset; HURDAT2 as the hurricane
  overlay. See `datavis-36613/explore-storm-events.qmd`.
- **computing-36614** → Storm Events + GHCN + HURDAT2 ingested into a relational DB
  (Postgres target, DuckDB for the zero-setup demo); ACS / Zillow / TIGER / xView2 as
  enrichment joins. See `computing-36614/sql-database-poc.qmd`.

_Last updated: 2026-06-15._