Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 0 additions & 73 deletions .github/workflows/rust-ci.yml

This file was deleted.

139 changes: 47 additions & 92 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,82 +5,51 @@ SPDX-FileCopyrightText: Copyright the Vortex contributors

# AGENTS.md - `benchmarks-website/`

Read [`README.md`](README.md) first for the architecture. Then this file. The
root [`CLAUDE.md`](../CLAUDE.md) covers Rust style, test layout, commit
conventions. Everything about the legacy generations (v2, v3) is in
[`docs/legacy.md`](docs/legacy.md).

## The legacy v2 site

The v2 Vite/React site is retired from serving (`bench.vortex.dev` now points at
v4) and was deployed from elsewhere — its source never lived in this repo. The
migrator still reads the v2 S3 dump as its historical source (see
[`migrate/README.md`](migrate/README.md)).

The v3 deploy lives entirely under `server/`, `migrate/`, and `ops/`.
The operator runbook is [`ops/README.md`](ops/README.md).

## v3 specifics

- **Wire shapes are a coordinated change.** [`server/src/records.rs`](server/src/records.rs),
[`vortex-bench/src/v3.rs`](../vortex-bench/src/v3.rs), and (until cutover)
[`migrate/src/classifier.rs`](migrate/src/classifier.rs) must agree.
Bumping a shape means changing all three plus the snapshot fixtures in
one commit. `SCHEMA_VERSION` is the version literal coupled across
[`server/src/schema.rs`](server/src/schema.rs) (in-repo source of
truth), [`web/lib/schema-version.ts`](web/lib/schema-version.ts) (the
in-repo web mirror), and [`scripts/post-ingest.py`](../scripts/post-ingest.py)
(the monorepo CI ingest wrapper, which hardcodes it as a Python literal); a
bump also coordinates with the monorepo producer wire shape in
`vortex-bench/src/v3.rs`. Bump in lockstep or every CI ingest run 400s/409s
(400 if the server is ahead, 409 if the emitter is ahead). The server-side validation in
`records.rs` + `ingest.rs` and the echo in `/health` all consume the
constant through `crate::schema`. The full versioned contract lives in
[`CONTRACT.md`](CONTRACT.md).
- **Numeric `?n=` is clamped to 1000; `?n=all` is the uncapped escape
hatch.** HTML routes hydrate from the materialized latest-100 shard
artifact by default; `?n=all` is an explicit opt-in
(chart-init.js's full-history zoom-out hop uses it once, and curl
power users can request it). The numeric `?n=` path is bounded by
`MAX_NUMERIC_COMMIT_WINDOW` in [`server/src/api/window.rs`](server/src/api/window.rs)
as a DoS-protection floor against `curl ...?n=99999999`. If you need
full history, use `?n=all`. Do NOT raise the numeric cap or remove it
without thinking about the DoS surface.
- **`measurement_id` is server-internal.** Never put it on the wire. It is
a deterministic hash over `commit_sha` plus the dim tuple, computed in
[`server/src/db.rs`](server/src/db.rs) and reused by the migrator via
the same crate.
- **Don't write a server-side classifier for live ingest.** The emitter
produces v3-shape records directly; the migrator's classifier only
exists to translate v2 records once and goes away after cutover.
- **Don't reach for WASM.** SSR + a thin hydration script in
[`server/static/chart-init.js`](server/static/chart-init.js) is the
whole client.
- **v3 ingest is no longer best-effort in CI.** The `Ingest results to v3
server` step in [`bench.yml`](../.github/workflows/bench.yml),
[`sql-benchmarks.yml`](../.github/workflows/sql-benchmarks.yml), and
[`v3-commit-metadata.yml`](../.github/workflows/v3-commit-metadata.yml)
no longer carries `continue-on-error: true`. A v3-server outage on a
develop push now fails the bench workflow and triggers the existing
`incident.io` alert. The gate is `vars.V3_INGEST_URL != ''` so forks
and unconfigured environments are unaffected.
- **Don't re-introduce a server-side commit cap on `?n=all`.** `?n=all`
is the uncapped escape hatch (chart-init.js fetches it once for the
zoom-out path); visual downsampling happens client-side via LTTB on
the visible commit range only. Numeric `?n=` is clamped per the bullet
above. Default fetches from chart-init.js use the materialized
latest-100 shard artifact, not `?n=all`.
- **Don't refetch on every scope change.** Once a chart's payload is in
memory, pan/zoom/slider/range-strip all rebuild in place via the
in-memory LTTB pass on the cached payload. The single exception is the
latest-100 to full-history zoom-out path: charts initially hydrate from
the materialized latest-100 group shard artifact (served from
`/api/artifacts/{generation}/groups/{slug}/shards/{i}`); when the user
zooms past that window for the first time, `chart-init.js` lazy-fetches
`?n=all` once and replaces the latest-100 payload in place.
Read [`README.md`](README.md) first for the architecture. Then this file. The web app has its own
deep-dive in [`web/README.md`](web/README.md).

## The legacy generations

The previous site generations (v2 and v3) are documented in [`docs/legacy.md`](docs/legacy.md).
The v3 Rust tree (`server/`, `migrate/`, `ops/`, the Cargo workspace) has been deleted from the
working tree; recover it from git history if ever needed. Do not resurrect it piecemeal — the
rules below are the parts that were worth keeping.

## Load-bearing rules

- **`SCHEMA_VERSION` is a coordinated cross-repo change.**
[`web/lib/schema-version.ts`](web/lib/schema-version.ts) is the in-repo source of truth; the
monorepo's `scripts/post-ingest.py` hardcodes the same integer, and its `vortex-bench/src/v3.rs`
owns the producer wire shape. Bump in lockstep across both repos. The versioned contract lives
in [`CONTRACT.md`](CONTRACT.md), and
[`web/lib/schema-version.test.ts`](web/lib/schema-version.test.ts) asserts the in-repo anchors
agree.
- **`measurement_id` is internal — never on the wire.** It is a deterministic xxhash64 over
`commit_sha` plus the dim tuple, computed by the monorepo's `scripts/_measurement_id.py` just
before INSERT and used as the `ON CONFLICT` upsert key. Its output is frozen forever: the
monorepo's golden-vector test (`scripts/tests/test_measurement_id.py`) pins it, because a
drifted hash would insert duplicates next to the rows already in production.
- **Numeric `?n=` is clamped to 1000; `?n=all` is the uncapped escape hatch.** Pages hydrate from
the materialized latest-100 shard artifact by default; `?n=all` is an explicit opt-in (the
chart's full-history zoom-out hop uses it once, and curl power users can request it). The
numeric path is bounded by `MAX_NUMERIC_COMMIT_WINDOW` in
[`web/lib/window.ts`](web/lib/window.ts) as a DoS-protection floor against `?n=99999999`. Do
NOT raise the numeric cap or remove it without thinking about the DoS surface, and do NOT
re-introduce a server-side commit cap on `?n=all` — visual downsampling happens client-side via
LTTB on the visible commit range only.
- **Don't refetch on every scope change.** Once a chart's payload is in memory,
pan/zoom/slider/range-strip all rebuild in place via the in-memory LTTB pass on the cached
payload ([`web/components/Chart.tsx`](web/components/Chart.tsx)). The single exception is the
latest-100 to full-history zoom-out path: when the user first zooms past the materialized
window, the chart lazy-fetches `?n=all` once and replaces the payload in place.
- **Don't write a server-side classifier for live ingest.** The emitter produces structured
records directly. Classifying loose name strings at read time was the v2-era weakness every
later generation existed to escape; it belongs nowhere in the live pipeline.

## Footguns we have already hit

All in [`web/components/Chart.tsx`](web/components/Chart.tsx):

- **Reverse predecessor walk in the tooltip.** `payload.commits[]` is
sorted oldest-first by SQL - `commits[0]` is the oldest, `commits[N-1]`
is the newest. For per-row delta the predecessor of `commits[idx]` is
Expand All @@ -96,25 +65,11 @@ The operator runbook is [`ops/README.md`](ops/README.md).
## Local dev

```bash
# Public-only run (read API + ingest only, admin routes 404):
INGEST_BEARER_TOKEN=dev cargo run -p vortex-bench-server
# The web app needs BENCH_DB_* env vars for a real database — see web/README.md.
cd web && pnpm install && pnpm dev

# With admin endpoints mounted on a separate loopback listener:
INGEST_BEARER_TOKEN=dev ADMIN_BEARER_TOKEN=dev \
cargo run -p vortex-bench-server

cargo nextest run -p vortex-bench-server -p vortex-bench-migrate
INSTA_UPDATE=auto cargo nextest run -p vortex-bench-server # update snapshots
pnpm test # vitest unit tests
pnpm build # deliberately works without a database, so CI needs no secrets
```

For the full env-var contract (admin bind, snapshot dir, extension dir,
logging spec, PaaS `PORT` fallback) see [`ops/config/vortex-bench.env.example`](ops/config/vortex-bench.env.example)
and the lib-level `//!` doc on [`server/src/main.rs`](server/src/main.rs).

For the migrator end-to-end against the real S3 dump:

```bash
cargo run -p vortex-bench-migrate -- run --output ./bench.duckdb
VORTEX_BENCH_DB=./bench.duckdb INGEST_BEARER_TOKEN=dev \
cargo run -p vortex-bench-server
```
For the full env-var contract see [`web/README.md`](web/README.md).
Loading
Loading