Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
34 lines (28 loc) · 1.05 KB

CONTRIBUTING.md

File metadata and controls

34 lines (28 loc) · 1.05 KB

Contributing

Data Strategy

  • CSVs (data/*.csv) are committed to git — compact (~400 KB per race)
  • Raw JSON (data/raw/) is gitignored — too large for git (~10 MB per race)
  • Race manifest (data/races.json) tracks all available races and is auto-generated by the scraper

Adding New Races

  1. Add the race to scripts/race-registry.json with its group URL from labs-v2.competitor.com
  2. Run discovery to find event IDs: 
    node scripts/discover.js <ironman-results-url-or-group-url>
  3. Scrape results: 
    node scripts/scrape-all.js --slug=<race-slug> --year=2025
    Or scrape a single race directly: 
    node scripts/scrape.js <slug> <event-id> --no-raw
  4. The app reads from data/races.json — no code changes needed

Data Pipeline

ironman.com results page
  → discover.js extracts group UUID → fetches subevent IDs
  → scrape.js fetches API results → raw JSON → CSV
  → scrape-all.js generates data/races.json manifest
  → Next.js app reads races.json + CSVs at build time