Skip to content

fix(qdrant): return vectors in fetch_data so set_tags works with dense-only configs#3189

Open
happychap189 wants to merge 1 commit into
volcengine:mainfrom
happychap189:fix/qdrant-set-tags-vector
Open

fix(qdrant): return vectors in fetch_data so set_tags works with dense-only configs#3189
happychap189 wants to merge 1 commit into
volcengine:mainfrom
happychap189:fix/qdrant-set-tags-vector

Conversation

@happychap189

Copy link
Copy Markdown

What type of PR is this?

  • Bug fix
  • Feature
  • Refactor
  • Documentation
  • Test

Summary

The set_tags API (POST /api/v1/content/set_tags) fails with Qdrant point requires at least one dense or sparse vector when OpenViking runs against the Qdrant backend with a dense-only embedding configuration (no sparse vector). Since add_resource does not accept a tags field, set_tags is the only way to attach k=v search tags — so this bug completely blocks the search_tags retrieval-filtering feature for Qdrant users.

Root cause: update_search_tags reads the existing record via getfetch_data, but fetch_data sets with_vector: False, so the returned record has no dense_vector. The subsequent full upsert (_make_point) then finds no vector and raises.

This PR makes fetch_data return vectors (with_vector: True) and extracts them into DataItem.fields, so that metadata-only update flows (set_tags, and any future get-modify-upsert caller) retain the vector and can upsert successfully.

Changes

  • openviking/storage/vectordb/collection/qdrant_collection.pyfetch_data:
    • Changed "with_vector": FalseTrue
    • Extract dense/sparse vectors via _extract_named_vectors and store them in the returned DataItem.fields

Single-method change, ~12 lines added.

Why this approach (and not alternatives)

Option Verdict
A. fetch_data returns vectors (this PR) Minimal, fixes the root cause for all get-modify-upsert flows. Safe: _make_point already strips vector fields from the payload (line 348). ✅
B. Pass partial_update=True in update_search_tags Does not work — the partial-update path also reads via getfetch_data (with_vector=False), so it still has no vector. ❌
C. Add dense_vector to FETCH_BY_URI_OUTPUT_FIELDS Used by the filter path, not by get/fetch_data; doesn't fix the actual read path. ❌
D. Dedicated setPayload-based metadata update Cleaner architecturally but a larger change. Better as a follow-up.

Verification

Before (broken)

POST /api/v1/content/set_tags
{"uri":"viking://resources/test/doc.md","tags":["version=v1"],"mode":"replace"}
→ {"status":"error","error":{"code":"INVALID_ARGUMENT",
   "message":"Qdrant point requires at least one dense or sparse vector"}}

After (fixed)

POST /api/v1/content/set_tags
{"uri":"viking://resources/test/doc.md","tags":["version=v1"],"mode":"replace"}
→ {"status":"ok","result":{"success_count":1,"tags_updated":true,
   "tags":["version=v1"]}}

Full end-to-end retrieval matrix (tags written via set_tags, queried via find):

Query Expected Result
tags=["version=v1"] 1 doc ✅ 1
tags=["version=v2"] 1 doc ✅ 1
tags=["version=v3"] (nonexistent) 0 ✅ 0
tags=["product=alpha"] 2 docs ✅ 2
tags=["version=v1","version=v2"] (OR) 2 docs ✅ 2

Root cause trace

update_search_tags (viking_vector_index_backend.py:863)
  → self.get([id])                                    # reads record
    → base._fetch_and_normalize → fetch_data          # with_vector=False → no vector!
  → self.upsert(updated_record)                       # record has no vector
    → _make_point                                      # dense_vector is None
      → raise ValueError("Qdrant point requires...")  # 💥

Test plan

  • Manual: set_tags (replace mode) succeeds on Qdrant + dense-only config
  • Manual: set_tags (append mode) succeeds
  • Manual: find with tags filter returns correct results (6 test cases)
  • CI: existing qdrant-related tests pass
  • Regression: fetch_data consumers that don't need vectors are unaffected

Environment

  • OpenViking 0.4.7
  • Qdrant backend, dense-only embedding (BAAI/bge-m3, dim 1024)
  • Python 3.13

…e-only configs

The set_tags API (POST /api/v1/content/set_tags) fails with
"Qdrant point requires at least one dense or sparse vector" when using
the Qdrant backend with a dense-only embedding configuration (no sparse
vector). Since add_resource does not accept a tags field, set_tags is the
only way to attach k=v search tags, so this bug completely blocks the
search_tags retrieval-filtering feature for Qdrant users.

Root cause: update_search_tags reads the existing record via get ->
fetch_data, but fetch_data sets with_vector=False, so the returned record
has no dense_vector. The subsequent full upsert (_make_point) then finds
no vector and raises.

Fix: make fetch_data return vectors (with_vector=True) and extract them
into DataItem.fields via _extract_named_vectors, so that metadata-only
update flows (set_tags, and any get-modify-upsert caller) retain the
vector and can upsert successfully. This is safe because _make_point
already strips vector fields from the payload.

Verified: set_tags (replace/append) succeeds; find with tags filter
returns correct results across 6 test cases (single-tag, multi-tag OR,
negative tag, etc.).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

1 participant