Skip to content

feat(providers): structured runtime.provider config for Copilot custom endpoints (closes #136)#225

Open
jrob5756 wants to merge 1 commit into
mainfrom
feature/136-copilot-local-llm
Open

feat(providers): structured runtime.provider config for Copilot custom endpoints (closes #136)#225
jrob5756 wants to merge 1 commit into
mainfrom
feature/136-copilot-local-llm

Conversation

@jrob5756
Copy link
Copy Markdown
Collaborator

Closes #136.

Summary

runtime.provider now accepts either the bare string shorthand
(provider: copilot, current behavior, unchanged) or a structured
object
that forwards a ProviderConfig to the Copilot SDK's
create_session(provider=…) parameter. This lets workflows route the
Copilot SDK at OpenAI-compatible / Azure / Anthropic endpoints (Ollama,
vLLM, LM Studio, Azure OpenAI, etc.) instead of being locked to the
GitHub Copilot service.

runtime:
  provider:
    name: copilot
    type: openai            # openai | azure | anthropic
    wire_api: completions   # completions | responses
    base_url: http://localhost:11434/v1
    api_key: ${OPENAI_API_KEY}
  default_model: llama3.1

Scope agreed with the reporter: Copilot provider only this PR,
YAML + env-var fallbacks (no new CLI flags), and validate type
/ wire_api against the SDK's supported set
.

Design highlights

  • Activation gate – the SDK provider kwarg is only attached when
    YAML explicitly opts in (at least one field beyond name). Ambient
    OPENAI_* env vars never silently divert default Copilot
    traffic.
  • Env-var fallbacks once custom routing is active:
    • base_urlCOPILOT_PROVIDER_BASE_URLOPENAI_BASE_URL
    • api_keyCOPILOT_PROVIDER_API_KEYOPENAI_API_KEY
    • bearer_tokenCOPILOT_PROVIDER_BEARER_TOKEN
    • type defaults to openai when base_url is set
  • Secrets hygieneapi_key and bearer_token are SecretStr,
    so they redact in model_dump / workflow_started payloads /
    dashboard / event logs. A redacted _describe_provider() helper
    feeds verbose logs.
  • Dialog parity_apply_provider_config() is called from both
    _execute_sdk_call (agents) and execute_dialog_turn (dialog mode)
    so all sessions hit the same endpoint.
  • Validationtype / wire_api are Literal[...] (Pydantic
    enforces); azure options require type: azure;
    name != "copilot" rejects any Copilot-only field (structured
    config for other providers is explicit follow-up).
  • Default-model warning – Custom endpoints rarely expose the SDK
    default gpt-4o, so the provider warns when custom routing is
    active without runtime.default_model.
  • CLI override--provider <name> replaces the entire
    ProviderSettings; logs a notice when YAML had structured fields.
    Wired into both run and resume for parity.
  • Round-trip compatibility – a custom model_serializer collapses
    ProviderSettings back to a bare string when only name is set,
    so serialized YAML/JSON stays byte-compatible with the prior schema.

Files

Area Change
src/conductor/config/schema.py New ProviderSettings, AzureProviderOptions; RuntimeConfig rework
src/conductor/providers/copilot.py provider_settings kwarg, _resolve_sdk_provider_config, _apply_provider_config, applied to agent + dialog paths
src/conductor/providers/factory.py Threads provider_settings (Copilot only)
src/conductor/providers/registry.py Forwards runtime.provider to factory only on matching name
src/conductor/cli/run.py Redacted logging helper, override-warning, parity with resume
examples/copilot-local-llm.yaml New: Ollama + Azure OpenAI variants
AGENTS.md Documents the object form + env fallbacks
tests/test_config/test_provider_settings.py New: coercion, validators, serialization, gating
tests/test_providers/test_copilot_provider_routing.py New: resolver + plumbing + registry

Test plan

  • make lint
  • make typecheck ✅ (only pre-existing diagnostic remains)
  • make validate-examples
  • uv run pytest tests/ -m "not performance" -q2817 passed, 16 skipped, 29 deselected

Out of scope (called out for follow-up)

  • Structured provider config for claude / openai-agents.
  • Persisting a redacted provider fingerprint in checkpoints so resume
    can warn on configuration drift.
  • Per-field CLI overrides (e.g. --provider-base-url).

Credit

Inspired by @epowers's proof-of-concept
patch

attached to the issue.

@jrob5756 jrob5756 force-pushed the feature/136-copilot-local-llm branch from 2c6e818 to e1416b4 Compare May 21, 2026 19:40
@jrob5756
Copy link
Copy Markdown
Collaborator Author

Review fixes round 1 (critical + high)

Force-pushed e1416b4 with the following changes after a pre-merge review pass with the pr-review skill (6 agents: code, tests, comments, silent-failures, types, dead-code).

Critical (silent failures that masqueraded as default behavior):

  • _resolve_sdk_provider_config now raises ProviderError when custom routing is activated but every resolved field is falsy (e.g. expected env vars unset). Previously returned None, silently dropping the SDK provider kwarg.
  • azure block is no longer dropped when api_version: null. Schema validator rejects empty azure blocks up front; resolver always emits the block when settings.azure is not None.

High:

  • Validator now rejects anchorless / empty / broken combinations that would have activated custom routing but produced unusable configs: wire_api / type / headers / azure cannot stand alone without base_url / api_key / bearer_token; empty headers, empty SecretStr, empty azure block all fail at config load.
  • _warn_custom_routing_default_model now uses an explicit _default_model_explicit flag instead of comparing _default_model == "gpt-4o". Eliminates the false positive when a user picks gpt-4o deliberately and the false negative if the SDK fallback ever changes.
  • Dual-credential warning moved below the env-var resolution layer so it also fires for YAML × env mixing (e.g. api_key in YAML, COPILOT_PROVIDER_BEARER_TOKEN in shell).
  • Security: ambient OPENAI_API_KEY is no longer an implicit fallback for api_key. Only COPILOT_PROVIDER_API_KEY is consulted from env. Prevents accidentally sending an OpenAI dev credential to whatever base_url points at. Users who want OpenAI env behavior must opt in explicitly via api_key: ${OPENAI_API_KEY} interpolation. OPENAI_BASE_URL remains a fallback (URLs are not secrets).
  • ProviderSettings is now frozen=True (custom routing is set-once at config load; eliminates the Pydantic cross-field validator drift on attribute assignment).
  • Reverted unrelated getattr("model")getattr("default_model") drive-by fix in ProviderFactory.create_provider to keep this PR scoped to Feature: Copilot Local LLM use #136. Will file separately.

Deferred to follow-ups (called out in commit msg):

  • Discriminated-union refactor on ProviderSettings.name (would eliminate the cross-field validator entirely).
  • Documentation-only fixes raised by the comment-reviewer (AGENTS.md wording, example YAML prose).
  • Additional test-coverage gaps (registry isolation negative case, CLI override end-to-end, _describe_provider redaction test, etc.).

Verification: make lint ✅, make typecheck ✅, make validate-examples ✅, uv run pytest tests/ -m "not performance" -q2828 passed, 16 skipped, 29 deselected (11 new test cases over the prior 2817).

@jrob5756 jrob5756 force-pushed the feature/136-copilot-local-llm branch from e1416b4 to 8d62c5f Compare May 21, 2026 19:47
…m endpoints

Closes #136.

`runtime.provider` now accepts either the bare string shorthand
(`provider: copilot`) or a structured `ProviderSettings` object that
forwards a `ProviderConfig` to the Copilot SDK's
`create_session(provider=...)` parameter. This lets workflows route the
Copilot SDK at OpenAI-compatible / Azure / Anthropic endpoints (Ollama,
vLLM, LM Studio, Azure OpenAI, etc.) instead of being locked to the
GitHub Copilot service.

Schema (`src/conductor/config/schema.py`):
- New `ProviderSettings` model (frozen) with `name` / `type` /
  `wire_api` / `base_url` / `api_key` / `bearer_token` / `headers` /
  `azure`. `frozen=True` avoids the Pydantic gotcha where
  cross-field `model_validator` invariants don't re-fire on
  per-attribute assignment.
- `api_key` and `bearer_token` are `SecretStr` (redacted in
  `model_dump`, dashboard payloads, event logs).
- New `AzureProviderOptions` mirrors the SDK's nested
  `azure.api_version` shape.
- `RuntimeConfig.provider` is `ProviderSettings` with a `mode='before'`
  validator that coerces strings, plus `validate_assignment=True` so
  CLI mutation still revalidates.
- Strict validators reject anchorless / empty / broken combinations
  that would silently no-op at the SDK boundary: `wire_api` / `type` /
  `headers` / `azure` cannot stand alone without an endpoint anchor
  (`base_url` / `api_key` / `bearer_token`); empty `headers`, empty
  `SecretStr`, and `azure: {api_version: null}` are rejected. Non-
  copilot `name` plus any structured field is rejected (structured
  config for Claude / openai-agents is out of scope here).
- `model_serializer` collapses `ProviderSettings` back to a bare string
  when only `name` is set so serialized YAML/JSON stays byte-compatible
  with the prior schema.

Copilot provider (`src/conductor/providers/copilot.py`):
- New `provider_settings` kwarg on `__init__`. Tracks
  `_default_model_explicit` so the custom-routing warning fires based
  on whether the caller supplied a model, not on a fragile string
  comparison against the SDK fallback.
- `_resolve_sdk_provider_config()` builds the SDK `ProviderConfig` dict
  with env-var fallbacks: `COPILOT_PROVIDER_BASE_URL` →
  `OPENAI_BASE_URL` for `base_url`; `COPILOT_PROVIDER_API_KEY` (only)
  for `api_key`; `COPILOT_PROVIDER_BEARER_TOKEN` (only) for
  `bearer_token`. Ambient `OPENAI_API_KEY` is intentionally NOT a
  fallback — that would silently leak an OpenAI credential to whatever
  `base_url` points at. Activation is gated on `has_custom_routing()`
  so ambient env vars alone never divert default Copilot traffic.
- `_apply_provider_config()` mutates session kwargs in-place; called
  from both the agent `_execute_sdk_call` path and the dialog
  `execute_dialog_turn` path so all sessions hit the same endpoint.
- Raises `ProviderError` when custom routing activated but every
  resolved field is falsy (silent no-op was previously possible if all
  expected env vars were missing).
- Warns when `api_key` and `bearer_token` BOTH resolve (from any
  YAML × env combination) since the SDK silently prefers
  `bearer_token`; warns when custom routing is active without an
  explicit `runtime.default_model`.

Plumbing:
- `providers/factory.py` `create_provider` accepts `provider_settings`
  and forwards to `CopilotProvider` only.
- `providers/registry.py` passes `runtime.provider` to the factory only
  when the resolved provider type matches the settings' `name`;
  switches all bare-string reads to `.name`.
- `cli/run.py` reads `.name` for logging, adds a redacted
  `_describe_provider()` helper used in verbose output, and emits a
  notice when `--provider` override discards structured YAML fields
  (wired into both `run` and `resume` for parity).

Docs & example:
- `examples/copilot-local-llm.yaml` — Ollama + Azure OpenAI examples.
- `AGENTS.md` — new bullet under "Key Patterns" documenting the object
  form, env-var fallbacks, and CLI override semantics.

Tests:
- `tests/test_config/test_provider_settings.py` (new) — coercion,
  validators (including the new anchorless / empty / broken-combo
  rejection cases), serialization, `has_custom_routing` gate.
- `tests/test_providers/test_copilot_provider_routing.py` (new) —
  resolver behavior (env precedence, secret unwrap, ambient
  `OPENAI_API_KEY` non-fallback regression, YAML×env dual-credential
  warning, raise-on-empty regression, default-model warning suppressed
  when caller supplies a model) and `create_session` plumbing for both
  agent and dialog paths plus registry forwarding.
- Updated assertion sites across the test tree to read
  `runtime.provider.name` now that the field is a model.

Follow-ups (called out for later):
- Structured provider config for `claude` / `openai-agents`.
- Persisting a redacted provider fingerprint in checkpoints so resume
  can warn on configuration drift.
- Per-field CLI overrides (e.g. `--provider-base-url`).
- Converting `ProviderSettings` to a discriminated union on `name` —
  would eliminate the cross-field validator in favor of structural
  per-branch typing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jrob5756 jrob5756 force-pushed the feature/136-copilot-local-llm branch from 8d62c5f to 423bca2 Compare May 21, 2026 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Copilot Local LLM use

1 participant