feat: SQ-HNSW scalar-quantized HNSW with online calibration and rerank (nightly 2026-07-07)

[ruvnet]

Summary

• Adds new crate crates/ruvector-sq-hnsw implementing three HNSW index variants all behind a common AnnIndex trait: F32 baseline, SQ8 (int8 graph traversal), and SQ8+Rerank (int8 traversal + f32 exact rerank of top candidates).
• Implements a calibrated ScalarQuantizer that maps each dimension linearly to [-127, 127] using per-dimension min/max statistics collected from the first N vectors.
• Fixes a critical HNSW neighbor-pruning bug (single-arg vs two-arg dist_fn) that silently degraded recall from 0.77 to 0.03.
• All benchmarks are real (cargo run --release); no placeholder numbers.

Benchmark results (n=10k, dims=128, k=10, rustc 1.94.1, x86_64 linux)

┌─────────────────┬──────────┬────────────┬────────────┬─────────────┬──────────────┬──────────────┐
│ Variant         │ Recall@10│ Mean(μs)   │ p95(μs)    │ QPS         │ Mem/vec(B)   │ Build(ms)    │
├─────────────────┼──────────┼────────────┼────────────┼─────────────┼──────────────┼──────────────┤
│ F32 (baseline)  │ 0.7704   │      396.7 │      464.0 │        2521 │          512 │         8333 │
│ SQ8 (no-rerank) │ 0.7682   │      256.6 │      302.3 │        3897 │          128 │         5556 │
│ SQ8 + Rerank    │ 0.7690   │      270.6 │      315.9 │        3696 │          640 │         5595 │
└─────────────────┴──────────┴────────────┴────────────┴─────────────┴──────────────┴──────────────┘

SQ8 vs F32: −35% latency, 4× less memory, −0.3% recall. All acceptance tests PASS.

New files

Path	Description
crates/ruvector-sq-hnsw/Cargo.toml	New crate manifest
crates/ruvector-sq-hnsw/src/lib.rs	AnnIndex trait, HnswConfig, SearchResult, exact_knn
crates/ruvector-sq-hnsw/src/quantizer.rs	ScalarQuantizer: calibrate, encode, decode, sq8_l2_sq
crates/ruvector-sq-hnsw/src/hnsw.rs	HnswGraph: two-arg dist_fn, insert_node, beam search
crates/ruvector-sq-hnsw/src/index.rs	F32Index, Sq8Index, Sq8RerankIndex with unit tests
crates/ruvector-sq-hnsw/src/main.rs	Benchmark binary with acceptance tests
docs/adr/ADR-272-sq-hnsw-calibrated-search.md	Architecture Decision Record
docs/research/nightly/2026-07-07-sq-hnsw-calibrated-search/README.md	Full research document
docs/research/nightly/2026-07-07-sq-hnsw-calibrated-search/gist.md	Public-facing implementation summary

Test plan

• cargo test -p ruvector-sq-hnsw — 8 unit tests + 1 doc test, all pass
• cargo run --release -p ruvector-sq-hnsw — benchmark binary prints RESULT: PASS
• cargo build --workspace — no new warnings or errors
• Integration into ruvector-core behind sq8 feature flag (future milestone M1)

🤖 Generated with claude-flow

https://claude.ai/code/session_01QwG2TymHzK9LapMySKWsRU

---

Generated by Claude Code