Skip to content

docs: add comprehensive MCP server documentation#59

Draft
julpayne wants to merge 1 commit into
mainfrom
docs/mcp-integration
Draft

docs: add comprehensive MCP server documentation#59
julpayne wants to merge 1 commit into
mainfrom
docs/mcp-integration

Conversation

@julpayne
Copy link
Copy Markdown
Collaborator

Summary

Adds a new top-level MCP documentation section with 8 pages covering the full EvalHub MCP server integration, per RHOAIENG-60355.

New pages

Page Content
Overview What MCP is, what the EvalHub MCP server provides (tools, resources, prompts), transport modes
Installation Homebrew, GitHub Releases, build from source, Kubernetes operator deployment, EvalHub CLI as MCP server
Quick Start Step-by-step setup for Claude Code (stdio + HTTP) and VS Code / GitHub Copilot (stdio + HTTP)
Configuration CLI flags, YAML config file, environment variables, precedence rules, Kubernetes operator CR fields
Tool Reference submit_evaluation, get_job_status, cancel_job — parameters, responses, examples
Resource Reference All evalhub:// URI patterns — providers, benchmarks, collections, jobs, server version — with filtering, pagination, and autocompletion
Prompt Reference edd_workflow, evaluate_model, compare_runs — arguments, behavior, examples
Troubleshooting Connection issues, auth failures, TLS errors, backend unreachable, common error messages, MCP Inspector usage

Other changes

  • astro.config.mjs: Added MCP section to sidebar navigation (between Deployment and Adapters)
  • reference/mcp.md: Updated to link to the new comprehensive section while preserving the EvalHub CLI MCP workflow for Kubernetes/OpenShift deployments

Sources

Documentation was derived from:

  • eval-hub/eval-hub — MCP server source code (cmd/evalhub_mcp/, internal/evalhub_mcp/)
  • trustyai-explainability/trustyai-service-operator (feat/evalhub-mcp-support branch) — operator MCP support
  • eval-hub/eval-hub/tests/mcp/CLAUDE_CODE_INTEGRATION.md — integration test plan

Test plan

  • npm run build succeeds with all 8 new pages rendering
  • Verify all internal links resolve correctly
  • Review copy-paste commands on a clean environment
  • Verify example outputs match actual server responses

Add a new top-level MCP section covering installation, quick-start
guides for Claude Code and VS Code, configuration reference, tool/
resource/prompt API references, and troubleshooting. Update existing
reference/mcp.md to link to the new section while preserving the
EvalHub CLI MCP workflow for Kubernetes deployments.

Ref: RHOAIENG-60355

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant