Skip to content

Add vexdb_lite extension#2012

Open
vibeinging wants to merge 2 commits into
duckdb:mainfrom
vibeinging:add-vexdb_lite
Open

Add vexdb_lite extension#2012
vibeinging wants to merge 2 commits into
duckdb:mainfrom
vibeinging:add-vexdb_lite

Conversation

@vibeinging

Copy link
Copy Markdown

This PR adds vexdb_lite, an HNSW-based vector search extension for DuckDB.

Features

  • Native FLOAT[N] vector type (no custom type required)
  • Distance functions: L2 / Cosine / Inner Product (operator aliases <->, <=>, <#>, <~>)
  • Graph (HNSW) ANN index via CREATE INDEX ... USING GRAPH_INDEX
  • Product Quantization (PQ) for compact indexes
  • Query optimizer rewrite: ORDER BY distance(...) LIMIT k → index scan
  • Runtime SIMD dispatch (SSE / AVX2 / AVX-512 / NEON)

Details

  • Source: VexDB-THU/VexDB-Lite @ v0.0.15 (MIT)
  • Dependencies: only header-only boost/preprocessor (provided via vcpkg boost-preprocessor); no BLAS/Fortran/OpenMP.
  • Platforms: linux/osx amd64 + arm64. Windows/WebAssembly excluded for now (not yet built/validated there).
  • Standard loadable extension (cmake + duckdb_extension_load), no DuckDB core changes.

Quickstart

CREATE TABLE items (id INTEGER, vec FLOAT[3]);
INSERT INTO items VALUES (1, [1.0, 2.0, 3.0]), (2, [4.0, 5.0, 6.0]), (3, [1.0, 2.0, 4.0]);
CREATE INDEX idx ON items USING GRAPH_INDEX (vec) WITH (metric = 'l2');
SELECT id FROM items ORDER BY array_distance(vec, [1.0, 2.0, 3.0]::FLOAT[3]) LIMIT 2;

Vector similarity search for DuckDB: FLOAT[N] vectors, L2/Cosine/InnerProduct
distance functions, a self-developed graph (GRAPH_INDEX) ANN index, product
quantization, and an ORDER BY distance LIMIT k -> index scan optimizer rewrite.

Source: VexDB-THU/VexDB-Lite @ v0.0.15 (MIT). Only header-only boost/preprocessor
as external dependency (vcpkg). Excludes windows/wasm for now (not yet built there).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant