Ci/metrics history

[orionpapadakis]

Performance metrics history pipeline

Adds end-to-end collection of benchmark metrics from every CI run into a persistent docs/perf-history.jsonl file committed back to main.

What changed

scripts/write_metrics_sidecar.py (new)
Helper script that writes a *.meta.json sidecar from KEY=VALUE shell arguments. Coerces JSON-native types (booleans, numbers) automatically.

scripts/process_metrics.py (rewritten)

• Discovers metric+sidecar pairs recursively via rglob("*.json")
• Schema is fully open-ended — no required fields in either file
• Missing fields become null (not 0) in the JSONL output
• Each row includes flat compat fields plus nested "benchmark" and "metrics" objects for forward compatibility

.github/workflows/build-and-run.yml

• Every ./llama-tornado step now sets JAVA_TOOL_OPTIONS to emit a metrics JSON and calls write_metrics_sidecar.py to write the matching sidecar (same stem, same directory)
• All artifact pairs are uploaded and consumed by publish-performance-history
• publish-performance-history only runs on pushes to main (not PRs or forks), avoiding duplicate entries and push-permission errors

Result

One JSONL row per benchmark step per merge to main, covering all backends, models, quantizations, and inference configurations currently in CI.