research(nightly): slipstream warm-start streaming HNSW insertions#652
Draft
ruvnet wants to merge 3 commits into
Draft
research(nightly): slipstream warm-start streaming HNSW insertions#652ruvnet wants to merge 3 commits into
ruvnet wants to merge 3 commits into
Conversation
Implements three streaming insertion strategies on a flat proximity graph (HNSW layer-0) exploiting inter-arrival spatial locality in agent memory streams: EntryPoint (baseline), FixedCache (warm-start from previous insert's discovered set), and Adaptive (EMA drift detection with automatic cache reset on distribution shift). Key design: random long-jump edges (m_longjump=8) for small-world navigability across cluster boundaries, enabling ≥0.991 recall@10 on both locality-preserving and shuffled datasets. 16 unit tests; all pass. Co-Authored-By: claude-flow <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01JAJtLh1Bb14jCdkRL8kPyY
Proposes adoption of the Slipstream principle (arXiv:2606.02992) as crates/ruvector-slipstream. Documents three InsertStrategy variants, drift controller pseudocode, benchmark evidence (recall@10=0.991-0.992 on N=4000, D=64), failure modes, security considerations, and four-phase integration roadmap. Co-Authored-By: claude-flow <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01JAJtLh1Bb14jCdkRL8kPyY
Research README covers motivation (agent memory locality), three-variant design, real benchmark numbers, memory math, ecosystem integration points (ruFlo, MCP, RVF), mermaid architecture diagram, and forward-looking roadmap. Gist provides SEO-optimized public article with competitor comparison table and practical/exotic application survey. Co-Authored-By: claude-flow <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01JAJtLh1Bb14jCdkRL8kPyY
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Nightly research branch implementing Slipstream warm-start (arXiv:2606.02992): exploit spatial locality in streaming vector insertion by seeding each beam search from the previous insertion's discovered candidate set rather than a fixed global entry node.
crates/ruvector-slipstream— a standalone flat proximity graph (HNSW layer-0) with threeInsertStrategyvariants:EntryPoint(baseline),FixedCache(warm-start),Adaptive(EMA drift-detection + automatic cache reset)m_longjump=8) added during insertion for cross-cluster small-world navigabilityBenchmark (N=4,000, D=64, K=10, M=16)
Streamed dataset (locality-preserving order):
Shuffled dataset (locality-breaking order — warm-start must not degrade):
All recall checks passed (≥0.80 threshold). Adaptive's 3,997 drift resets on the shuffled dataset confirm the detector correctly fires at every cluster boundary.
Files changed
Cargo.toml— workspace member:crates/ruvector-slipstreamcrates/ruvector-slipstream/— new crate:graph.rs,slipstream.rs,dataset.rs,metrics.rs,lib.rs,src/bin/benchmark.rsdocs/adr/ADR-272-slipstream-warm-start.md— architecture decision recorddocs/research/nightly/2026-07-08-slipstream-warm-start/README.md— research reportdocs/research/nightly/2026-07-08-slipstream-warm-start/gist.md— SEO-optimised public articleEcosystem integration
stream_locality_hintvector_insert_streamtool surface withwarm_start: boolSlipstreamIndexserialises into the cognitive package formatruvector-corebehindfeatures = ["slipstream"]Related
Generated by Claude Code