feat: add core MCP server, TrainerClient tools and Resources

[abhijeet-dhumal]

fixes #7

Description

Implements the core MCP server framework and all TrainerClient-backed tools per KEP-936.
Claude plugin and skills moved to a separate PR per reviewer feedback.

Core Framework (kubeflow_mcp/core/)

• Server factory with persona-based tool filtering, and progressive/semantic meta-tool modes
• Config — YAML config file with env-var fallback chain
• Security — K8s name validation, training bounds, secret masking, namespace enforcement, heuristic script safety
• Auth — Bearer token + JWT/OIDC for HTTP transport
• Policy — Persona access control (readonly, data-scientist, ml-engineer, platform-admin) with allow/deny overrides
• Resilience — Thread-safe rate limiter and circuit breaker
• Logging — Structured JSON audit logging with correlation IDs

TrainerClient Tools (23 tools) and 3 MCP resource guides

CLI - serve command with stdio / http / sse transport, persona, auth token, instruction tier, progressive/semantic modes

Test Suite (131 tests)

• cli_test.py (25) — CLI flags, config fallback, auth/resilience wiring, SSE transport
• sdk_contracts_test.py (68) — SDK dataclass fields, enum values, API signatures, K8s client methods
• test_architecture.py (38) — Tool metadata, persona gating, instruction tiers, resource registration

Checklist

• Tests pass locally (make test-python) — 131 passing
• Linting passes (make verify) — 0 errors
• Documentation updated
• Commit messages follow conventional format

Related Issues

Implements KEP-936

[abhijeet-dhumal]

Hey @andreyvelich @astefanutti @kramaranya @szaher @Electronic-Waste - the 2nd onboarding phase is ready for review now 🏁
Would appreciate a review when you get a chance 🙏

[kramaranya]

Thanks @abhijeet-dhumal, is it possible to split the changes into a few PRs (for example MCP server, tools, docs/plugin/benchmark)?

[abhijeet-dhumal]

Thanks @abhijeet-dhumal, is it possible to split the changes into a few PRs (for example MCP server, tools, docs/plugin/benchmark)?

Thanks @kramaranya .. I completely understand the preference for smaller PRs, and I normally follow that approach.

The context here is that after discussing with @andreyvelich and @thesuperzapper, we agreed on a phased onboarding strategy with a time constraint: land the project as a cohesive unit first, then refine via focused follow-up PRs. I tried to keep commits in this PR structured as logically separate components to make review easier.

Splitting into parallel PRs now is doable, but these components have tight interdependencies (e.g. tools depend on core server, benchmarks test the tools, security hardening touches multiple layers), so it would add significant overhead to manage the PR dependency chain and likely slow down the merge timeline.

@andreyvelich @astefanutti @thesuperzapper What way do you recommend, Happy to adjust either way.

[andreyvelich]

@abhijeet-dhumal Maybe you can separate Claude plugin and SKILLS into separate PR? So we can review MCP tools first.

[abhijeet-dhumal]

Hi @abhijeet-dhumal Thanks for a ton for this.

I’ve left comments on a couple of issues above—could you please take a look at those? Aside from that, everything looks good to me at this stage of the project. 😊

Disclaimer: Used Claude Opus for PR review.

Prompt Details

Thanks @jaiakash - all 4 items addressed in the latest push.. Appreciate the thorough review 🙌
The CircuitBreaker race and urllib3 internals catches were solid finds

[abhijeet-dhumal]

@abhijeet-dhumal Maybe you can separate Claude plugin and SKILLS into separate PR? So we can review MCP tools first.

@andreyvelich @kramaranya @astefanutti Apologies for the size of this PR - I understand it makes review harder.
IMHO splitting this PR further would require stubs or dead code in each PR just to pass CI, without making review easier.
This PR is the testable baseline for this new repo - reviewers can directly test this mcp project right away thoroughly to test workflows or even make inspector against a real working server.

kubeflow-mcp serve \
  --clients trainer \             # modules: trainer, optimizer (stub), hub (stub)
  --persona ml-engineer \         # readonly | data-scientist | ml-engineer | platform-admin
  --mode full \                   # full | progressive | semantic
  --transport stdio \             # stdio | http | sse
  --auth-token SECRET \           # bearer token for HTTP auth (dev/staging)
  --log-level INFO \              # DEBUG | INFO | WARNING | ERROR
  --log-format console \          # console | json (auto-detected if omitted)
  --no-banner

Re: why MCP Resources are included
The 3 resource guides (training-patterns, platform-fixes, troubleshooting) are not documentation — they're part of the tool runtime. Resources ARE packaged in the MCP server - they get bundled with pip install, and are registered on the server at startup via FastMCP's mcp.resource(uri). Any MCP client that connects gets access to them through the standard resources/read protocol method.

MCP Resources are a protocol-level feature that agents read on-demand when tools reference them:

• pre_flight() and estimate_resources() return trainer://guides/platform-fixes when OpenShift is detected
• fine_tune() returns trainer://guides/training-patterns when the user needs a LoRA script alternative
• get_training_logs() returns trainer://guides/troubleshooting when OOM/CUDA errors are detected
• Server instructions reference all 3 URIs to guide agent behavior

Without these, tool responses would contain issues that agents can't resolve. They're co-located with the tools because they're registered by the same client module and tested together (architecture tests verify all URIs are resolvable).

Happy to clarify anything further — let me know if you'd like me to adjust the scope.

[abhijeet-dhumal]

Thanks for the thorough review, @kramaranya ! I have resolved the comments raised..
Looking forward to the next round whenever you get a chance. 😄

[abhijeet-dhumal]

Hey @kramaranya, thanks again for the thorough pass. Addressed several of your threaded points..
The PR is now ready for the next round 😄

[andreyvelich]

Thanks @abhijeet-dhumal!
/lgtm
/approve

/hold @kramaranya in case you have more suggestions

[google-oss-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andreyvelich

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [andreyvelich]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[kramaranya]

Thank you, @abhijeet-dhumal!
LGTM
/unhold