feat(tui): wire LLM tier settings and flag config drift on connected tools#34
Merged
Conversation
…tools Fix the Settings screen writing dead LLM_MODEL_* keys nothing reads: the runtime keys now come from services.llm_tiers.LLM_TIER_KEYS (LLM_HIGH/ MEDIUM/LOW) as the single source of truth, with friendly labels and a save-time purge of the legacy names. Connecting bakes the whole env block into each AI tool's own config, so a later Settings edit does not reach an already-connected tool. Record a fingerprint of the block baked into each client at connect time (new client_configs section in ~/.specflow/config.json) and surface drift: - ClientSetupScreen renders a connected-but-stale client in orange with a reconnect affordance; pressing enter re-bakes and clears the flag. - SettingsScreen warns and routes to the connect screen when a runtime change leaves connected tools on stale config. - 'm' on the connect screen opens tier settings before connecting. Probe deliberately never writes the fingerprint (it confirms presence, not env); a dropped connection forgets it. Correct the module docstring that claimed edits propagate to every client.
Drop EDITABLE_LABELS and render the raw env keys (LLM_HIGH/MEDIUM/LOW, WORKSPACE_COUNT, ...) so the Settings screen matches what appears in mcp-config.json verbatim.
awrobel-gd
reviewed
Jul 9, 2026
| # keep them working but out of the footer. The footer carries navigation. | ||
| Binding("d", "show_config", "raw config", show=False), | ||
| Binding("v", "recheck", "re-scan", show=False), | ||
| Binding("m", "edit_tiers", "model tiers", show=False), |
Contributor
There was a problem hiding this comment.
this is a bit small to notice.
I liked your idea of a info box at the bottom explaining
you are on models ..... now, and press "m" to change them.
Notice: changes in MCP servers take effect in the harness sandbox from the next run, not current one (in case its already running)
awrobel-gd
reviewed
Jul 9, 2026
| self._set_status(client.client_id, mcp_clients.ClientStatus.NOT_CONFIGURED) | ||
| self._render_detail() | ||
|
|
||
| def _is_stale(self, client_id: str, status: mcp_clients.ClientStatus) -> bool: |
Contributor
There was a problem hiding this comment.
this app.py is getting extremely long. Maybe its a good idea to start splitting into tui/settings_logic.py and tui/components.py and others
As such _is_stale is pretty specific
awrobel-gd
reviewed
Jul 9, 2026
Add a persistent footer note to ClientSetupScreen listing the model tiers a connect would bake in (blank tier = backend default), the 'm' affordance to change them, and the standing caveat that a tier change reaches the sandbox only from the next run — a generation already in progress keeps its models. Makes the previously easy-to-miss 'm' binding discoverable.
Replace the plain one-line tiers note with a titled Panel on the connect screen: a tier/model/purpose grid (high → planning & KB init, medium → code generation, low → simple steps) plus the 'm' affordance and the next-run caveat. Blank tiers show as the backend default.
Surface per-model validity in the TUI using the existing /api/v1/models/validate endpoint. Add request_model_validation_for(), which validates explicit LLM_* values from the local config block (the TUI edits mcp-config.json, not its own env, so the env-based path does not fit). Connect screen: a background worker validates on open and re-renders the tiers box with a per-tier glyph (✓ available / ✗ unsupported / • unchecked), tints each model, and lists invalid models with the backend's suggestion. Settings: each tier field carries a live ✓/✗ marker updated on mount and on submit. Both are best-effort — a missing backend or unreachable catalog leaves everything neutral, never red.
The per-tier ✓/✗ marker was correctly set but rendered off the right edge: the row's Input did not flex, so label+input already overflowed the viewport and the trailing marker landed past it. Make .settings-row inputs 1fr and give the marker a fixed width so it always stays visible. Add a regression test asserting the marker region falls within the viewport.
awrobel-gd
reviewed
Jul 9, 2026
awrobel-gd
previously approved these changes
Jul 9, 2026
awrobel-gd
left a comment
awrobel-gd
left a comment
Contributor
There was a problem hiding this comment.
one follow-up comment
Address review: apply_llm_tier_overrides mutated its argument in place (a hidden side effect). Add pure llm_tier_overrides_from_env() that returns the overrides, and keep apply_llm_tier_overrides as a thin in-place wrapper for generation_orchestrator (which accumulates several field groups into one form). request_model_validation now passes the returned dict directly — no mutation. Behavior is byte-identical, so the backend request is unchanged.
awrobel-gd
approved these changes
Jul 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes the TUI LLM model-tier settings actually take effect, lets users set tiers before connecting an AI tool, flags connected tools left on stale config after a Settings change, and shows per-model validity against the active provider catalog. Previously the Settings screen wrote tier keys that nothing read, so configured models were silently ignored.
Entrypoint
Start in mcp_server/tui/config.py where EDITABLE_KEYS is now sourced from services.llm_tiers.LLM_TIER_KEYS, then mcp_server/tui/mcp_clients.py for config_fingerprint and stale_connected_ids, then mcp_server/tui/app.py where ClientSetupScreen and SettingsScreen wire the model-tiers box, the orange drift overlay, and the validation workers. Model validity reuses the existing backend endpoint via services/validate_models.request_model_validation_for.
Diagram
Details
The tier-key fix is the core correctness change: the MCP server and backend read LLM_HIGH/MEDIUM/LOW, but the screen wrote LLM_MODEL_*, so save_env now sources the real keys from the shared LLM_TIER_KEYS and purges the dead legacy names (also so they never skew a config fingerprint).
Drift detection exists because connecting bakes the entire env block into each tool, so a later Settings edit does not reach an already-connected tool. A fingerprint of the baked block is stored per client in a new sibling client_configs section of ~/.specflow/config.json (the clients status schema is untouched, no migration). The Claude presence probe deliberately never writes the fingerprint — it confirms the server is registered, not what env it holds, so writing there would silently clear a real drift.
Model validation is best-effort and reuses the existing /api/v1/models/validate endpoint. request_model_validation_for takes explicit tier values because the TUI edits the config file rather than its own process env. Any failure (backend down, catalog unreachable) leaves markers neutral, never a false red. It validates on the connect screen when opened and in Settings on mount and on Enter (not per keystroke), and the did-you-mean suggestion only appears when the backend catalog has a near-match.
One non-obvious layout fix: the Settings marker was correct but rendered off the right edge because the row input did not flex, so .settings-row inputs are now 1fr with a fixed-width marker. No onboarding step was added — onboarding is architecturally .env-only and mcp-config.json does not exist until init runs at the wizard end.
Testing
make unit-tests green (655 passed); new coverage spans the tier-key SSOT and legacy purge, fingerprint/staleness helpers and the orange overlay, connect/probe fingerprint behavior, the Settings drift routing, the validation client and glyph mapping, the validated tiers box and Settings markers, and a regression test that the marker stays within the viewport. ruff clean on all changed files.