Skip to content

[codex] Add transport-neutral tool upstreams#1027

Merged
andreasronge merged 3 commits into
mainfrom
codex/transport-neutral-tool-upstreams
May 28, 2026
Merged

[codex] Add transport-neutral tool upstreams#1027
andreasronge merged 3 commits into
mainfrom
codex/transport-neutral-tool-upstreams

Conversation

@andreasronge
Copy link
Copy Markdown
Owner

Reason

The MCP server had outgrown MCP-specific public naming: we now support multiple upstream transports, and the user-facing Lisp surface should describe the action being taken rather than the protocol underneath it. This PR makes the upstream call surface transport-neutral and adds a narrow OpenAPI v1 upstream adapter so HTTP API tools can be exposed through the same catalog/call path as MCP tools.

What Changed

  • Replaced the public Lisp forms mcp/servers and tool/mcp-call with transport-neutral tool/servers and tool/call across evaluator support, prompts, docs, tests, benches, and generated docs.
  • Added OpenAPI upstream config parsing for transport: "openapi", including schema loading from file or URL, operation filters, operation overrides, auth bindings, timeouts, and response caps.
  • Added an OpenAPI compiler/executor for the v1 scope:
    • GET-only operations.
    • JSON-compatible 2xx responses only.
    • path/query parameter support with path escaping.
    • request bodies and header/cookie parameters rejected at compile time.
    • operation name collision checks and provenance metadata via _ptc.
  • Hardened review findings:
    • cap raw OpenAPI schema/API responses before decode.
    • fail schema URL auth materialization/emitter errors instead of silently fetching unauthenticated.
    • preserve inputSchema in catalog rendering.
    • reject missing path parameter declarations and non-JSON 2xx responses at boot/catalog compile time.
    • remove schema-derived String.to_atom/1 usage.
    • make default args consistent between schema metadata and call execution.
    • avoid empty query ? and reject array query values explicitly for v1.
  • Added Stage 6 docs and fixtures:
    • Observatory OpenAPI fixture under mcp_server/test/fixtures/openapi/.
    • sample upstream config under docs/examples/.
    • config docs for transport: "openapi", schema_file, schema_url, filters, overrides, and auth.
  • Updated stale docs/bench examples that still referenced the removed MCP-specific Lisp forms.

Impact

Users should now author upstream tool calls as tool/call regardless of whether the backing upstream is MCP or OpenAPI. MCP upstream behavior remains available through the same internal adapter path, while OpenAPI tools can participate in the same catalog, prompt, and execution flows.

This is intentionally a 0.x breaking rename: old public Lisp names were removed rather than shimmed.

Validation

Local checks on this branch:

  • mix format --check-formatted from repo root: passed.
  • mix test from repo root: 369 doctests, 3 properties, 5130 tests, 0 failures.
  • mix format --check-formatted from mcp_server/: passed.
  • mix compile --warnings-as-errors from mcp_server/: passed.
  • mix test from mcp_server/: 10 doctests, 1207 tests, 0 failures, 1 skipped.
  • Commit hook: root and mcp_server format, compile, credo, scoped tests passed.
  • Push hook: root full tests + dialyzer passed; mcp_server full tests + dialyzer passed; ptc_viewer tests passed.

Real-LLM benchmark runs were also executed with .env credentials. They do not appear related to this implementation, but are recorded here for review context:

  • bench/lisp_eval_real_client_eval.exs: 12/14 passed. Failures were both context_reduce model/harness behavior.
  • bench/lisp_session_real_client_eval.exs: 3/4 passed. Failure returned the expected final value but the harness marked the recovery case failed.
  • bench/agentic_real_eval.exs: 8/9 passed. Failure was lazy catalog discovery choosing /, outside the test filesystem allowlist.
  • bench/mcp_bench.exs --real-llm: passed after rerunning with PTC_RUNNER_MCP_RESPONSE_PROFILE=debug; the first run exposed an older benchmark assumption that MCP responses include structuredContent.

Benchmark reports are in local ignored tmp/ files and are not part of this PR.

Address OpenAPI review findings, complete the Observatory fixture and configuration docs, and update stale transport-neutral tool call examples.

Validated with mcp_server format/compile/test checks, root format/test checks, and real-LLM MCP benchmark runs.
Document how to configure a coding agent to use PtcRunner MCP with an HTTPS OpenAPI upstream, including upstream config, credential binding, agent MCP wiring, and smoke-test forms.

Validated with root and mcp_server format checks.
@andreasronge andreasronge marked this pull request as ready for review May 28, 2026 08:06
@andreasronge andreasronge merged commit ae8f1a5 into main May 28, 2026
7 checks passed
@andreasronge andreasronge deleted the codex/transport-neutral-tool-upstreams branch May 28, 2026 08:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant