test: golden-envelope snapshot tests + million-row scale test (#6)#59
Conversation
Two contract/validation guards, both test-only (no src/ change → no release): 1. anomalyx/tests/golden.rs — runs the actual binary and pins its byte-exact stdout for `schema`, `describe`, and a representative `scan` envelope (id col → identifier/skipped, reading col → one point outlier) against committed goldens under tests/golden/. Any accidental contract drift — renamed field, changed dense-row layout, shifted config_version, recalibrated confidence — fails CI as a visible diff. Verified the tripwire fires on a one-field change. Regenerate intentional changes with `BLESS=1 cargo test -p anomalyx --test golden`. 2. ax-validate/tests/stress.rs — million_row_scan_is_deterministic_and_recovers_ outliers: a 1,000,000-row scan is byte-identical across runs AND recovers exactly the five injected outlier rows (precision=recall=1 at scale). Runs in ~3s; a loop-bound regression surfaces as a timeout. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Warning Review limit reached
More reviews will be available in 45 minutes and 4 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (6)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
1 participant
What
The roadmap capstone — two validation guards that lock in everything #1–#5 built. Test-only: no
src/change, so no version bump / release.1. Golden-envelope snapshot tests (
anomalyx/tests/golden.rs)Runs the actual
anomalyxbinary and pins its byte-exact stdout for the three contract surfaces against committed goldens:schema→schema.json(the JSON Schema)describe→describe.json(protocol metadata, detectors, formats,config_version)scanon a fixed CSV →scan_basic.json(full envelope: dict, dense rows, roles, summary, exit)Any accidental contract drift — a renamed field, a changed dense-row layout, a shifted
config_version, a recalibrated confidence — now fails CI as a visible diff instead of slipping into a release. Verified the tripwire fires (a one-field change →golden mismatchFAILED). Intentional changes regenerate withBLESS=1.2. Million-row scale test (
ax-validate/tests/stress.rs)million_row_scan_is_deterministic_and_recovers_outliers: a 1,000,000-row scan must be byte-identical across runs and recover exactly the five injected outlier rows (precision = recall = 1). Determinism and correctness verified at scale, not just on the existing 40k toy input. Runs in ~3s; a loop-bound regression surfaces as a timeout.Gate
fmt/clippy/ full workspacetestgreen (14 result-blocks). No mutation surface (tests only).Roadmap
This completes the "make it full" arc: #1 FDR · #2 output scoping · #3 column roles · #4 robustness harness · #5 confidence calibration · #6 golden + scale ✓.
🤖 Generated with Claude Code