Companion to #673 (PQ-guided search), same source read of ruvector-diskann on main (c55e9e8).
ADR-144 says: "mmap load — OS pages in only accessed vectors — working set << total dataset."
What the code does today: load() memory-maps vectors.bin and then copies every float into a heap Vec (index.rs:322-342 — per-chunk f32::from_le_bytes into flat_data). The retained Mmap handle is never read after that loop, so the entire dataset is RAM-resident. The index is currently an in-memory Vamana index with file persistence; the SSD-backed working-set story doesn't hold yet.
Proposal: make FlatVectors storage-agnostic — an owned Vec<f32> variant (current behavior, used during insert/build) and an mmap-slab variant that reads vector slices directly out of the map. The on-disk format already cooperates: little-endian flat f32 data at byte offset 16, so slices are 4-byte aligned and can be exposed zero-copy (bytemuck cast) on little-endian targets. Graph adjacency and PQ codes stay heap-resident — they're small relative to the vectors.
Combined with #673, per-hop traversal cost moves to PQ tables in RAM and only the final re-rank touches mmapped full vectors — which is the actual DiskANN serving architecture from the paper, and what makes 100M+ vector indexes on a laptop realistic.
Happy to contribute this as a PR. It would include a page-cache-cold vs warm latency table so the on-SSD behavior is documented from measurement rather than asserted.
Companion to #673 (PQ-guided search), same source read of
ruvector-diskannon main (c55e9e8).ADR-144 says: "mmap load — OS pages in only accessed vectors — working set << total dataset."
What the code does today:
load()memory-mapsvectors.binand then copies every float into a heapVec(index.rs:322-342— per-chunkf32::from_le_bytesintoflat_data). The retainedMmaphandle is never read after that loop, so the entire dataset is RAM-resident. The index is currently an in-memory Vamana index with file persistence; the SSD-backed working-set story doesn't hold yet.Proposal: make
FlatVectorsstorage-agnostic — an ownedVec<f32>variant (current behavior, used during insert/build) and an mmap-slab variant that reads vector slices directly out of the map. The on-disk format already cooperates: little-endian flat f32 data at byte offset 16, so slices are 4-byte aligned and can be exposed zero-copy (bytemuck cast) on little-endian targets. Graph adjacency and PQ codes stay heap-resident — they're small relative to the vectors.Combined with #673, per-hop traversal cost moves to PQ tables in RAM and only the final re-rank touches mmapped full vectors — which is the actual DiskANN serving architecture from the paper, and what makes 100M+ vector indexes on a laptop realistic.
Happy to contribute this as a PR. It would include a page-cache-cold vs warm latency table so the on-SSD behavior is documented from measurement rather than asserted.