Optional LatticeEmbedding provider for ruvector-core (feature: lattice-embeddings)#648
Open
ohdearquant wants to merge 5 commits into
Open
Optional LatticeEmbedding provider for ruvector-core (feature: lattice-embeddings)#648ohdearquant wants to merge 5 commits into
ohdearquant wants to merge 5 commits into
Conversation
… (feature: lattice-embeddings) Adds a pure-Rust native EmbeddingProvider backed by lattice-embed 0.5 (CPU-only, default 'native' feature, no metal-gpu). Entirely behind the new lattice-embeddings feature (not in default). - LatticeEmbedding::from_pretrained delegates model-id parsing to lattice_embed::EmbeddingModel's own FromStr impl instead of reimplementing the mapping (bge/e5/minilm/qwen3, display names + HF ids), so it stays in sync with lattice-embed's canonical table. - embed() is passage/document-side (no query instruction) via EmbeddingService::embed_passage; the inherent embed_query() method is query-side (applies E5/Qwen3 query instructions) via EmbeddingService::embed_passage/embed_query, making asymmetric retrieval correct. - Bridges lattice-embed's async EmbeddingService trait onto the sync EmbeddingProvider trait via a dedicated single-threaded tokio Runtime + block_on (documented re-entrancy constraint: do not call from within another running Tokio runtime). - Tests: pure model-id mapping tests (no network) + a #[ignore]'d real end-to-end embed test (needs ~130MB model download). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bumps the optional lattice-embed dependency from 0.5 to the published 0.5.1, which includes the BGE query-instruction prefix for asymmetric retrieval. Verified: cargo check + test -p ruvector-core --features lattice-embeddings pass against lattice-embed 0.5.1 from crates.io.
…models, correct BGE docs Run lattice-embed's runtime and embedding service on a dedicated worker thread instead of calling block_on on the caller's thread. embed/embed_query now send a request over a channel and block on the reply, so they are safe to call from inside an existing Tokio runtime; the previous stored-runtime approach panicked on nested block_on. Reject non-local models (e.g. the remote-only OpenAI variant) at construction in with_model rather than deferring the failure to the first embed() call. Correct the module- and method-level rustdoc: BGE v1.5 applies a query-side instruction prefix for asymmetric retrieval (only MiniLM is symmetric), and disclose that enabling the lattice-embeddings feature raises the effective MSRV to Rust 1.93 while the default build is unchanged. Verified: cargo check + test (lib + doc) + clippy --all-targets pass under the lattice-embeddings feature against lattice-embed 0.5.1 from crates.io. New tests cover the async-context regression and construction-time rejection of remote-only model ids.
…osure Rustfmt reflows the outcome.map_err(...).and_then(...) chain in the worker thread's reply mapping. No behavior change.
…edding
Adds documentation and a dedicated, runnable example demonstrating the
asymmetric-retrieval differentiator of the native LatticeEmbedding provider.
- examples/lattice_embedding_example.rs: constructs the provider via
from_pretrained("bge-small-en-v1.5"), then embeds a passage with embed()
and a query with embed_query(), printing the cosine similarities to show
concretely that the query instruction changes the vector. Matches the
crate's existing example convention (per-crate examples/, //! header with a
"Run with:" block, ===/--- section prints). Gated by a [[example]] block
with required-features = ["lattice-embeddings"] so the default build skips it.
Runnable: cargo run --example lattice_embedding_example --features lattice-embeddings
- embeddings.rs: adds a "# Examples" section to the LatticeEmbedding struct
rustdoc with a no_run doctest of the same passage/query split.
Verified (feature on): cargo fmt --check clean, clippy clean on the new code,
example compiles and runs (bge-small-en-v1.5, 384-dim), doctests compile.
Docs/example only; no provider behavior change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Optional
LatticeEmbeddingprovider forruvector-coreAdds an optional
EmbeddingProviderimplementation,LatticeEmbedding, that wraps the publishedlattice-embedcrate (a pure-Rust BERT-family text-embedding engine). It sits alongside the existingHashEmbedding/CandleEmbedding/OnnxEmbedding/ApiEmbeddingproviders, entirely behind a new default-off featurelattice-embeddings. The default build and the default embedder are unchanged.This is an option, not a default-flip — a correct, measured embedding backend you can opt into. Whether it ever becomes a default
.rvfembedder is entirely your call on the evidence below.Why it might be useful
Correct-by-construction asymmetric retrieval. BGE/E5 are asymmetric retrievers: the query needs an instruction prefix, the document does not.
LatticeEmbeddingexposes an explicitembed()(passage, no prefix) and an inherentembed_query()(applies the BGE query instruction). TheEmbeddingProvidertrait has a single symmetricembed()with no query/passage distinction (the default MiniLM embedder per ADR-210 is symmetric, so this is invisible there), but a BGE/E5 model plugged into that trait embeds queries without the required prefix. Since.rvfpacks shipbge-small-en-v1.5vectors, this asymmetry is exactly the BGE case. This is the strongest, most honest differentiator.Exact model, zero conversion.
.rvfpacks ship 384-dimbge-small-en-v1.5vectors;lattice-embedruns that exact model, so there is no embed-to-a-different-space mismatch.f32 fidelity (real but modest). Against FP32
sentence-transformers/bge-small-en-v1.5gold over a 100-sentence corpus:Real, but small. int8 is a good embedder here, not a broken one.
Native Rust, no Node/WASM/ONNX runtime — an architectural fit for a Rust-first project. Compute is CPU/Accelerate; no GPU required.
What this is not: not a claim that transformers.js is broken, and not a large retrieval-quality jump. Raw single-text speed is roughly a wash — correctness (the asymmetry split) is the lever, not throughput.
Design
lattice-embeddings = ["dep:lattice-embed", "dep:tokio"], not indefault.lattice-embedpulls only itsnative(CPU) feature, nevermetal-gpu. Nothing from the feature touches the default build path.lattice-embed'sEmbeddingServiceis async-only; theEmbeddingProvidertrait is sync. Rather than callRuntime::block_onon the caller's thread (which panics if the caller is already inside a Tokio runtime),LatticeEmbeddingruns the runtime and the embedding service on a dedicated worker thread;embed/embed_querysend a request over a channel and block on the reply. The worker has no ambient async context, so a call from inside an existing runtime is safe. Requests are FIFO-serialized through the single worker (fine for v0; a pool is a later option). A#[tokio::test]embeds from inside a running runtime as the regression guard.from_pretrained(model_id)delegates id parsing tolattice-embed's ownFromStr(case-insensitive; accepts display names, short names, HuggingFace ids), then rejects at construction any id that is unrecognized or resolves to a model the native engine can't run locally (e.g. the remote-only OpenAI variant) — no silent default, no deferred failure on firstembed().lattice-embeddingsraises the effective MSRV to Rust 1.93 (edition 2024), sincelattice-embedrequires it; Cargo cannot express a per-featurerust-version, so this is documented in the module docs and above the dependency line rather than inrust-version.lattice-embed = { version = "0.5.1", optional = true }from crates.io.Testing
cargo check/cargo test -p ruvector-core --features lattice-embeddings(library + doctests) andcargo clippy --all-targetspass against the publishedlattice-embed 0.5.1. Provider-specific tests cover the async-context regression, construction-time rejection of remote-only model ids, and acceptance of a valid native id. The default build (feature off) is untouched.