Skip to content

fix(selfhost): attribute embed/vision/advisory AI usage to a real provider#5259

Merged
JSONbored merged 1 commit into
mainfrom
claude/ai-embed-provider-attribution
Jul 12, 2026
Merged

fix(selfhost): attribute embed/vision/advisory AI usage to a real provider#5259
JSONbored merged 1 commit into
mainfrom
claude/ai-embed-provider-attribution

Conversation

@JSONbored

@JSONbored JSONbored commented Jul 12, 2026

Copy link
Copy Markdown
Owner

Summary

Follow-up to #5247, found during its own live-deploy verification on edge-nl-01.

#5247 fixed createOpenAiCompatibleAi to return real usage.provider attribution, but
src/server.ts's 3 call sites that actually construct the AI_EMBED/AI_VISION/AI_ADVISORY
bindings never passed providerName at all — confirmed live by triggering a real RAG-indexing
pass on edge-nl-01 right after deploying #5247: the new embeddings rows correctly showed the
real model (bge-m3:latest, not the old hardcoded placeholder) but provider was still blank on
every row, since opts.providerName was undefined at the source.

These 3 bindings deliberately accept any openai-compatible endpoint with no explicit provider
selector (unlike the main review chain's AI_PROVIDER), so there's no existing knob to read a
provider name from. Added providerNameFromBaseUrl() to classify the configured base URL into the
same closed "ollama" | "openai" | "openai-compatible" set createOpenAiCompatibleAi already uses
— recognizing the bundled docker-compose ollama service and OpenAI's own endpoint, and falling
back to the honest generic bucket for anything else (vLLM, LM Studio, a custom hostname) rather than
guessing.

Test plan

  • npm run typecheck — clean
  • npm run test:ci — full local gate green
  • New unit test covers all 4 classification branches (ollama by hostname substring, openai's
    exact hostname, a generic other host, and the undefined/malformed-URL fallback)
  • Live-verified on edge-nl-01 after deploying this fix: a manually triggered RAG-index pass
    produced a fresh embeddings row with provider = 'ollama', model = 'bge-m3:latest', and an
    organic screenshot_table_vision row landed with provider = 'ollama', model = 'qwen3-vl:8b-instruct' — both were previously always blank on provider.

…vider

createOpenAiCompatibleAi's usage.provider fix (#5247) never took effect in
production because src/server.ts's 3 call sites that construct the
AI_EMBED/AI_VISION/AI_ADVISORY bindings never passed providerName at all --
confirmed live via a real RAG-indexing pass on edge-nl-01, which recorded
correct model attribution (bge-m3:latest) but every row's provider column was
still blank. These bindings accept any openai-compatible endpoint with no
explicit provider selector, so classify the configured base URL's hostname
into the existing ollama/openai/openai-compatible provider-name set instead of
guessing or hardcoding one.
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.36%. Comparing base (73a0912) to head (976f874).
⚠️ Report is 3 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5259   +/-   ##
=======================================
  Coverage   94.36%   94.36%           
=======================================
  Files         474      474           
  Lines       40081    40088    +7     
  Branches    14614    14617    +3     
=======================================
+ Hits        37821    37828    +7     
  Misses       1585     1585           
  Partials      675      675           
Flag Coverage Δ
shard-1 46.24% <0.00%> (-0.29%) ⬇️
shard-2 34.74% <100.00%> (+0.08%) ⬆️
shard-3 32.07% <0.00%> (+0.03%) ⬆️
shard-4 31.84% <0.00%> (-0.29%) ⬇️
shard-5 33.68% <0.00%> (-0.04%) ⬇️
shard-6 44.98% <0.00%> (+0.18%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/selfhost/ai.ts 98.15% <100.00%> (+0.02%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@gittensory-orb gittensory-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 12, 2026
@gittensory-orb

gittensory-orb Bot commented Jul 12, 2026

Copy link
Copy Markdown

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-12 12:18:44 UTC

3 files · 2 AI reviewers · 2 blockers · readiness 93/100 · CI green · unstable

⏸️ Suggested Action - Manual Review

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.

Review summary
This is a small, well-targeted follow-up to #5247: it adds providerNameFromBaseUrl() to classify AI_EMBED/AI_VISION/AI_ADVISORY base URLs into the existing ollama|openai|openai-compatible set, and wires it into the 3 server.ts call sites that were previously leaving providerName undefined. The classification logic is simple and defensively handles malformed/undefined URLs via try/catch returning the generic fallback, and the new unit test exercises all 4 branches. This directly closes the attribution gap described in the PR summary and matches the existing convention in createOpenAiCompatibleAi.

Nits — 5 non-blocking
  • providerNameFromBaseUrl's `hostname.includes("ollama")` substring match would misclassify a hostname like `myollama-alternative.example.com` as ollama even though it's an unrelated third-party host; consider anchoring to a suffix/exact match if false positives matter in practice.
  • The long comment block above providerNameFromBaseUrl (src/selfhost/ai.ts) is quite dense for a doc comment; could be trimmed to the essential WHY (no explicit provider knob exists) without the full enumeration of every scenario.
  • Consider extracting the ollama-hostname heuristic into a named constant/regex if it's likely to need adjustment as more self-hosted provider names are supported.
  • nit: src/selfhost/ai.ts:612 uses a broad substring match for `ollama`, so a hostname like `not-ollama.example` is attributed as `ollama`; this is acceptable for conservative telemetry only if that looseness is intentional.
  • nit: src/selfhost/ai.ts:612 has a long explanatory block comment that reads more like PR context than durable code documentation; trim it to the invariant the helper enforces.

Concerns raised — review before merging

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.
Signal Result Evidence
Code review ❌ 2 blockers 2 reviewers, synthesized
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 44 registered-repo PR(s), 36 merged, 470 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 44 PR(s), 470 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Improvement ✅ Minor risk: clean · value: minor — Code changes are accompanied by test evidence. LLM value judgment: minor — The change is a narrow, correct bug-fix follow-up that wires an already-existing classification convention into three call sites, fixing real blank-provider data without expanding scope.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 44 PR(s), 470 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @gittensory ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @gittensory chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Development

Successfully merging this pull request may close these issues.

1 participant