feat(inference): gate local Ollama models by memory-layer context-window minimum

[sanil-23]

Summary

• Reject local Ollama models whose native context window is below 8192 — the floor the memory layer requires (bge-m3 embedder is requested at num_ctx=8192; summariser output is capped at 5000 to fit that embed ceiling). Sub-floor models silently truncate chunks/summaries and corrupt recall.
• The floor is derived from the embedder's own EMBED_NUM_CTX (now pub(crate)), so the acceptance gate can never drift from what the pipeline actually requests.
• New Ollama /api/show client resolves each model's real context_length (architecture-agnostic).
• inference.diagnostics now reports per-model context_length + an eligibility verdict, a context_requirement, and rejecting issues for sub-floor active models.
• Settings → Local AI Model shows a per-model context badge (accepted / rejected+flagged / unknown).

Problem

The memory tree's embedder over-requests num_ctx=8192 and the whole summary-budget logic exists to keep embed input ≤ 8192. Nothing validated that a user-selected local model actually supports that context — a short-context model (e.g. a 2048-ctx embedder) was accepted and then silently truncated input at embed time, corrupting recall with no signal to the user. /api/tags (the only Ollama metadata fetched) does not expose context length, so the gate needs /api/show.

Solution

• memory::tree::score::embed::ollama::EMBED_NUM_CTX made pub(crate); inference::local::model_requirements::MIN_CONTEXT_TOKENS re-exports it (single source of truth, drift-guarded by a test).
• evaluate_context(Option<u64>) → Ok / BelowMinimum / Unknown. Unknown (/api/show unreachable or metadata absent) is not a hard rejection — only a conclusive sub-floor report is.
• OllamaAdmin::fetch_model_context issues one bounded (5s) /api/show per installed model, concurrently, only on the diagnostics path.
• Frontend types are additive/optional → back-compatible with older cores (no badge rendered when eligibility is absent).
• No embedding-dimension / re-index work (explicitly out of scope for this PR).

Submission Checklist

If a section does not apply to this change, mark the item as N/A with a one-line reason. Do not delete items.

• Tests added or updated (happy path + at least one failure / edge case) per Testing Strategy
• Diff coverage ≥ 80% — every new/changed line is exercised: 10 Rust tests (eligibility classification incl. below/unknown, /api/show parse incl. missing/float, axum-mock diagnostics gate) + 4 Vitest cases (accept / reject+flag / unknown / older-core). Full pnpm test:coverage/pnpm test:rust merge deferred to CI (see Validation Run).
• Coverage matrix updated — added row 3.1.5 in docs/TEST-COVERAGE-MATRIX.md
• All affected feature IDs from the matrix are listed in the PR description under ## Related
• No new external network dependencies introduced — uses the existing local Ollama /api/show; no new crates/deps; tests use an in-process axum mock
• N/A: manual smoke checklist — diagnostics-panel badge + advisory issues; not a release-cut smoke surface
• N/A: no tracking issue — issue-less feature per maintainer request (no Closes #NNN)

Impact

• Desktop only (Settings → Local AI Model diagnostics). No migration, no schema change, no new deps.
• Performance: adds one /api/show round-trip per installed model, only when diagnostics runs, fetched concurrently with a 5s per-call timeout. No effect on hot paths.
• Behavior change: a sub-8192-context active model now produces a diagnostics issue (ok:false) and a red badge instead of being silently used.

Related

• Closes: N/A (issue-less)
• Affected matrix feature IDs: 3.1.5 Model Context-Window Requirement Gate
• Capability: local_ai.model_context_check
• Follow-up PR(s)/TODOs: embedding-dimension / re-index handling (deferred; out of scope here)

---

AI Authored PR Metadata (required for Codex/Linear PRs)

Linear Issue

• Key: N/A
• URL: N/A

Commit & Branch

• Branch: feat/ollama-context-gate (off upstream/main)
• Commit SHA: b8af174cd18620e5881a2d29a4e4cafcee1bfb37

Validation Run

• pnpm --filter openhuman-app format:check — Prettier run scoped to the 3 changed app files (clean); repo-wide skipped (Windows CRLF churn rewrites ~600 unrelated files — established --no-verify pattern)
• pnpm typecheck — passed
• Focused tests: cargo test --lib -- model_requirements:: context_length_ diagnostics_gates_models_by_context_window (10 passed); Vitest ModelStatusSection.test.tsx (18 passed)
• Rust fmt/check (if changed): cargo fmt + cargo check --manifest-path Cargo.toml — clean, no new warnings on changed files
• N/A: Tauri fmt/check — app/src-tauri not touched (changes are core Rust + app TS); pre-push pnpm rust:check skipped via --no-verify per established Windows pattern

Validation Blocked

• command: N/A
• error: N/A
• impact: N/A

Behavior Changes

• Intended behavior change: reject (don't silently use) local models whose native context window is below the memory-layer minimum of 8192
• User-visible effect: per-model context badge in Local AI Model settings; diagnostics flags a sub-floor active model as an issue

Parity Contract

• Legacy behavior preserved: /api/tags listing unchanged; older cores (no eligibility) render exactly as before (no badge); Unknown does not block
• Guard/fallback/dispatch parity checks: /api/show failure → Unknown (soft), never a hard reject; LM Studio diagnostics path untouched

Duplicate / Superseded PR Handling

• Duplicate PR(s): none
• Canonical PR: this
• Resolution: N/A

Summary by CodeRabbit

• New Features

  • Per-model context-window eligibility badges and visual highlighting for models rejected for insufficient context; diagnostics now reports each model’s resolved context length and the minimum-context requirement.

• Tests

  • Added unit and integration tests for eligibility evaluation, metadata parsing, and diagnostics gating by context-window.

• Documentation

  • Updated test-coverage matrix and diagnostics schema descriptions for the context-window requirement gate.

• Chores

  • Exposed the memory-layer minimum context constant for cross-module use.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: aa9dd1d4-8c7c-4845-b2e8-403ef4b4ca36

📥 Commits

Reviewing files that changed from the base of the PR and between d059990 and 8f03c3b.

📒 Files selected for processing (2)

• src/openhuman/inference/local/mod.rs
• src/openhuman/inference/local/ollama.rs

🚧 Files skipped from review as they are similar to previous changes (1)

• src/openhuman/inference/local/ollama.rs

---

📝 Walkthrough

Walkthrough

Defines a memory-layer minimum (8192), extracts native context windows from Ollama /api/show, evaluates per-model eligibility, includes eligibility and min_context_tokens in diagnostics, and renders eligibility badges and rejection styling in the frontend.

Changes

Local Model Context-Window Eligibility Gating

Layer / File(s)	Summary
Model requirements foundation and constants src/openhuman/memory/tree/score/embed/ollama.rs, src/openhuman/inference/local/mod.rs, src/openhuman/inference/local/model_requirements.rs	Expose EMBED_NUM_CTX (8192) as crate-visible, add ContextEligibility enum (Ok, BelowMinimum, Unknown), evaluate_context(), helpers, and unit tests.
Ollama model metadata extraction src/openhuman/inference/local/ollama.rs	Add /api/show request/response types and context_length() extraction that prefers architecture-scoped keys with fallback scanning and flexible numeric parsing; include extraction tests.
Diagnostics service integration src/openhuman/inference/local/service/ollama_admin.rs	Concurrently fetch per-model context, evaluate eligibility, build installed_models[] with context_length and eligibility, add context_requirement.min_context_tokens, and emit issues for below-minimum expected models.
Diagnostics integration test src/openhuman/inference/local/service/ollama_admin_tests.rs	Tokio integration test mocking Ollama endpoints asserting min_context_tokens, per-model context_length, and eligibility statuses in diagnostics.
Frontend types and UI components app/src/utils/tauriCommands/localAi.ts, app/src/components/settings/panels/local-model/ModelStatusSection.tsx	Add ModelContextEligibility and InstalledModelInfo types, implement ContextEligibilityBadge, and update diagnostics rows to render badges and conditional rejected styling.
Frontend component tests app/src/components/settings/panels/local-model/ModelStatusSection.test.tsx	Tests for accepted models (success badge), below-minimum models (below-min badge and red name styling), unknown eligibility, and legacy models without eligibility metadata.
Documentation and capability catalog src/openhuman/inference/schemas.rs, src/openhuman/about_app/catalog.rs, docs/TEST-COVERAGE-MATRIX.md	Document diagnostics payload fields, add local_ai.model_context_check capability entry, and add test-coverage matrix row for the model context-window gate.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• tinyhumansai/openhuman#1975: Touches the same local-Ollama diagnostics/UI surface and earlier diagnostics refactor relevant to per-model metadata handling.

Suggested labels

working

Suggested reviewers

• graycyrus
• senamakel

Poem

🐰 I hop through models, tallying each span,

Counting tokens with a careful plan.
Eight-thousand-one-hundred-something is the floor,
Badges bloom green or red at the door.
A rabbit's nod: only fit models go on.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately and concisely describes the main feature: gating Ollama models by a memory-layer context-window minimum, which is the primary objective across the entire changeset.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

app/src/components/settings/panels/local-model/ModelStatusSection.tsx (1)

11-48: ⚡ Quick win

Consider internationalizing the badge text.

The badge labels ("ctx ✓", "below X min", "ctx unknown") are hardcoded in English, while the rest of the component uses the useT() i18n hook. For consistency and to support non-English locales, consider extracting these strings to translation keys.

♻️ Example refactor to use i18n

 const ContextEligibilityBadge = ({
   eligibility,
 }: {
   eligibility: ModelContextEligibility | null | undefined;
 }) => {
+  const { t } = useT();
   if (!eligibility) return null;
   const fmt = (n: number) => n.toLocaleString();
   if (eligibility.status === 'ok') {
     return (
       <span
         className="shrink-0 rounded-full bg-green-100 dark:bg-green-500/15 px-2 py-0.5 text-[10px] font-medium text-green-700 dark:text-green-300"
-        title={`Context window ${fmt(eligibility.context_length)} tokens — meets the memory-layer minimum`}>
-        {fmt(eligibility.context_length)} ctx ✓
+        title={t('settings.localModel.contextEligibility.okTooltip', { count: fmt(eligibility.context_length) })}>
+        {t('settings.localModel.contextEligibility.okLabel', { count: fmt(eligibility.context_length) })}
       </span>
     );
   }
   // Similar for below_minimum and unknown cases...

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@app/src/components/settings/panels/local-model/ModelStatusSection.tsx` around
lines 11 - 48, The badge texts in ContextEligibilityBadge are hardcoded in
English; update ContextEligibilityBadge to use the existing i18n hook (call
useT() inside the component) and replace the literal strings ("ctx ✓", "below X
min", "ctx unknown" and the title text snippets) with translation keys (e.g.,
"modelStatus.ctxOk", "modelStatus.belowMin", "modelStatus.unknown", and
corresponding title keys), passing formatted values
(fmt(eligibility.context_length), fmt(eligibility.required)) as interpolation
params to t(); keep existing formatting helpers (fmt) and preserve the
status-based JSX paths in ContextEligibilityBadge so only the displayed text and
titles are swapped to t(...) calls.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/openhuman/inference/local/ollama.rs`:
- Around line 198-200: The current fallback scans info.iter() and picks the
first key ending with ".context_length", which can pick an unrelated low value;
instead, collect all entries whose key ends_with(".context_length"), parse each
via as_u64, and take the maximum numeric value (if any) as the fallback context
length so unrelated low entries won't undercut valid models; update the chain
that currently uses info.iter().find(...).and_then(|(_, value)| as_u64(value))
to a filter_map + max approach and ensure the result remains an Option<u64>
(symbols: info.iter(), as_u64, ".context_length").

---

Nitpick comments:
In `@app/src/components/settings/panels/local-model/ModelStatusSection.tsx`:
- Around line 11-48: The badge texts in ContextEligibilityBadge are hardcoded in
English; update ContextEligibilityBadge to use the existing i18n hook (call
useT() inside the component) and replace the literal strings ("ctx ✓", "below X
min", "ctx unknown" and the title text snippets) with translation keys (e.g.,
"modelStatus.ctxOk", "modelStatus.belowMin", "modelStatus.unknown", and
corresponding title keys), passing formatted values
(fmt(eligibility.context_length), fmt(eligibility.required)) as interpolation
params to t(); keep existing formatting helpers (fmt) and preserve the
status-based JSX paths in ContextEligibilityBadge so only the displayed text and
titles are swapped to t(...) calls.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 261891ef-c701-4f68-976c-e80598fa631e

📥 Commits

Reviewing files that changed from the base of the PR and between 0b053c5 and b8af174.

📒 Files selected for processing (12)

• app/src/components/settings/panels/local-model/ModelStatusSection.test.tsx
• app/src/components/settings/panels/local-model/ModelStatusSection.tsx
• app/src/utils/tauriCommands/localAi.ts
• docs/TEST-COVERAGE-MATRIX.md
• src/openhuman/about_app/catalog.rs
• src/openhuman/inference/local/mod.rs
• src/openhuman/inference/local/model_requirements.rs
• src/openhuman/inference/local/ollama.rs
• src/openhuman/inference/local/service/ollama_admin.rs
• src/openhuman/inference/local/service/ollama_admin_tests.rs
• src/openhuman/inference/schemas.rs
• src/openhuman/memory/tree/score/embed/ollama.rs

[sanil-23]

Nitpick acknowledged (i18n badge text): The labels are hardcoded English. Deferring i18n to a follow-up: the badge text includes dynamic numeric values (, ) that cannot be stored as plain translation keys in the current system (which takes a single key, returns a plain string with no interpolation). Adding these would require either extending with param interpolation or manually doing calls, plus adding keys to all 10+ locale files — meaningful scope for a separate PR. The diagnostic panel itself is an advanced/power-user surface with low localization priority.

[sanil-23]

Nitpick acknowledged (i18n badge text): The ContextEligibilityBadge labels are hardcoded English. Deferring i18n to a follow-up: the badge text includes dynamic numeric values (context_length, required) that cannot be stored as plain translation keys in the current useT() system (which takes a single key and returns a plain string with no interpolation support). Properly internationalizing these would require either extending useT() with param interpolation or manually doing .replace('{count}', ...) calls, plus adding keys to all 10+ locale files - meaningful scope for a separate PR. The diagnostics panel is an advanced power-user surface with low localization priority relative to main chat flows.

[graycyrus]

Walkthrough

Solid feature PR — gates local Ollama models by the memory layer's minimum context window (8192 tokens, derived from the embedder's own EMBED_NUM_CTX). The single-source-of-truth approach is well-designed: the acceptance floor literally re-exports the embedder constant, so it can't drift. Good test coverage across Rust (10 tests) and frontend (4 Vitest cases), including a drift-guard test and the CodeRabbit-suggested .max() regression test.

Change Summary

File	Change type	Description
model_requirements.rs	New	MIN_CONTEXT_TOKENS, ContextEligibility enum, evaluate_context() + 5 unit tests
ollama.rs	Modified	OllamaShowRequest/Response, context_length_from_model_info() + 6 unit tests
ollama_admin.rs	Modified	fetch_model_context(), diagnostics integration with per-model eligibility
ollama_admin_tests.rs	Modified	Integration test with mock /api/show
mod.rs	Modified	Declares + re-exports model_requirements module
embed/ollama.rs	Modified	EMBED_NUM_CTX visibility → pub(crate)
schemas.rs	Modified	Enriched diagnostics output description
catalog.rs	Modified	New capability entry local_ai.model_context_check
ModelStatusSection.tsx	Modified	ContextEligibilityBadge component + model card redesign
ModelStatusSection.test.tsx	Modified	4 new test cases
localAi.ts	Modified	New types ModelContextEligibility, InstalledModelInfo, extended LocalAiDiagnostics
TEST-COVERAGE-MATRIX.md	Modified	Added row 3.1.5

Per-file Analysis

model_requirements.rs — Clean, well-documented module. Drift-guard test is a nice touch. Pure logic with no side effects, so the absence of logging is appropriate.

ollama.rs — context_length_from_model_info resolution order (architecture-prefixed key → fallback max) is sound. The as_u64 helper correctly handles integer/float encodings. Tests are thorough.

ollama_admin.rs — fetch_model_context has proper error handling (.inspect_err + .ok()?) and debug logging with [local_ai:ollama_admin] prefix. The concurrent join_all for /api/show calls is the right call — bounded by installed model count.

Frontend — ContextEligibilityBadge is well-structured with null-safe guard. All new TS types are optional/additive for backward compatibility with older cores. Tests cover all three badge states + the legacy (no eligibility) path.

No critical or major issues found. Two minor notes below.

[sanil-23]

@graycyrus ready for re-review 🙏

Both your review comments are addressed and resolved (commit 8f03c3b1):

• inference/local/mod.rs: pub mod → pub(crate) mod model_requirements (matches sibling module convention)
• inference/local/ollama.rs: added a comment on the capabilities field documenting it exists to mirror the Ollama API wire shape (not yet consumed)

Status: all CI green (22/22, only the always-skipped macOS/Windows e2e remaining), 0 unresolved threads, MERGEABLE, no conflicts. Blocked only on an approving review. Thanks!

[graycyrus]

Looks good, nice work!