|
| 1 | +# MCP SDK Tier Audit: modelcontextprotocol/rust-sdk |
| 2 | + |
| 3 | +**Date**: 2026-02-25 |
| 4 | +**Branch**: alexhancock/conformance |
| 5 | +**Auditor**: mcp-sdk-tier-audit skill (automated + subagent evaluation) |
| 6 | + |
| 7 | +## Tier Assessment: Tier 3 |
| 8 | + |
| 9 | +The Rust SDK (rmcp) is currently at Tier 3. While server and client conformance pass rates exceed the 80% Tier 2 threshold, several critical Tier 2 requirements are not met: issue triage compliance is very low (14.1%), required labels are largely missing (3/12), no stable release ≥1.0.0 exists, and no roadmap is published. |
| 10 | + |
| 11 | +### Requirements Summary |
| 12 | + |
| 13 | +| # | Requirement | Tier 1 Standard | Tier 2 Standard | Current Value | T1? | T2? | Gap | |
| 14 | +|---|-------------|----------------|-----------------|---------------|-----|-----|-----| |
| 15 | +| 1a | Server Conformance | 100% pass rate | >= 80% pass rate | 83.3% (25/30) | FAIL | PASS | 5 failing scenarios (prompts-get-with-args, prompts-get-embedded-resource, elicitation-sep1330-enums, elicitation-sep1034-defaults, dns-rebinding-protection) | |
| 16 | +| 1b | Client Conformance | 100% pass rate | >= 80% pass rate | 85.0% (17/20) | FAIL | PASS | 3 failing date-versioned scenarios (scope-step-up, metadata-var3, 2025-03-26-oauth-endpoint-fallback) | |
| 17 | +| 2 | Issue Triage | >= 90% within 2 biz days | >= 80% within 1 month | 14.1% (9/64) | FAIL | FAIL | 54 issues exceeding SLA; median 4341h | |
| 18 | +| 2b | Labels | 12 required labels | 12 required labels | 3/12 | FAIL | FAIL | Missing: bug, enhancement, needs confirmation, needs repro, ready for work, P0, P1, P2, P3 | |
| 19 | +| 3 | Critical Bug Resolution | All P0s within 7 days | All P0s within 2 weeks | 0 open | PASS | PASS | None | |
| 20 | +| 4 | Stable Release | Required + clear versioning | At least one stable release | rmcp-v0.16.0 | FAIL | FAIL | No release >= 1.0.0 | |
| 21 | +| 4b | Spec Tracking | Timeline agreed per release | Within 6 months | 6d gap (PASS) | PASS | PASS | None | |
| 22 | +| 5 | Documentation | Comprehensive w/ examples | Basic docs for core features | ~8/48 features | FAIL | FAIL | Most features lack prose documentation | |
| 23 | +| 6 | Dependency Policy | Published update policy | Published update policy | dependabot.yml configured | PASS | PASS | None | |
| 24 | +| 7 | Roadmap | Published roadmap | Plan toward Tier 1 | Not found | FAIL | FAIL | No ROADMAP.md or docs/roadmap.md | |
| 25 | +| 8 | Versioning Policy | Documented breaking change policy | N/A | Not found | FAIL | N/A | No VERSIONING.md or BREAKING_CHANGES.md | |
| 26 | + |
| 27 | +### Tier Determination |
| 28 | + |
| 29 | +- Tier 1: FAIL — 3/11 requirements met (failing: server_conformance, client_conformance, triage, labels, stable_release, documentation, roadmap, versioning) |
| 30 | +- Tier 2: FAIL — 4/9 requirements met (failing: triage, labels, stable_release, documentation, roadmap) |
| 31 | +- **Final Tier: 3** |
| 32 | + |
| 33 | +--- |
| 34 | + |
| 35 | +## Server Conformance Details |
| 36 | + |
| 37 | +Pass rate: 83.3% (25/30) |
| 38 | + |
| 39 | +| Scenario | Status | Checks | Spec Versions | |
| 40 | +|----------|--------|--------|---------------| |
| 41 | +| server-tools-list | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 42 | +| server-tools-call-with-progress | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 43 | +| server-tools-call-with-logging | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 44 | +| server-tools-call-simple-text | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 45 | +| server-tools-call-sampling | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 46 | +| server-tools-call-mixed-content | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 47 | +| server-tools-call-image | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 48 | +| server-tools-call-error | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 49 | +| server-tools-call-embedded-resource | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 50 | +| server-tools-call-elicitation | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 51 | +| server-tools-call-audio | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 52 | +| server-server-sse-multiple-streams | PASS | 2/2 | 2025-11-25 | |
| 53 | +| server-server-initialize | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 54 | +| server-resources-unsubscribe | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 55 | +| server-resources-templates-read | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 56 | +| server-resources-subscribe | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 57 | +| server-resources-read-text | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 58 | +| server-resources-read-binary | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 59 | +| server-resources-list | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 60 | +| server-prompts-list | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 61 | +| server-prompts-get-with-image | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 62 | +| server-prompts-get-with-args | FAIL | 0/1 | 2025-06-18, 2025-11-25 | |
| 63 | +| server-prompts-get-simple | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 64 | +| server-prompts-get-embedded-resource | FAIL | 0/1 | 2025-06-18, 2025-11-25 | |
| 65 | +| server-ping | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 66 | +| server-logging-set-level | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 67 | +| server-elicitation-sep1330-enums | FAIL | 4/5 | 2025-11-25 | |
| 68 | +| server-elicitation-sep1034-defaults | FAIL | 2/5 | 2025-11-25 | |
| 69 | +| server-dns-rebinding-protection | FAIL | 1/2 | 2025-11-25 | |
| 70 | +| server-completion-complete | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 71 | + |
| 72 | +--- |
| 73 | + |
| 74 | +## Client Conformance Details |
| 75 | + |
| 76 | +Full suite pass rate: 85.0% (17/20 date-versioned) |
| 77 | + |
| 78 | +> **Suite breakdown**: Core: 4/4 (100%), Auth (date-versioned): 13/16 (81.3%) |
| 79 | +
|
| 80 | +### Core Scenarios |
| 81 | + |
| 82 | +| Scenario | Status | Checks | Spec Versions | |
| 83 | +|----------|--------|--------|---------------| |
| 84 | +| tools_call | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 85 | +| sse-retry | PASS | 3/3 | 2025-11-25 | |
| 86 | +| initialize | PASS | 1/1 | 2025-06-18, 2025-11-25 | |
| 87 | +| elicitation-sep1034-client-defaults | PASS | 5/5 | 2025-11-25 | |
| 88 | + |
| 89 | +### Auth Scenarios (Date-Versioned) |
| 90 | + |
| 91 | +| Scenario | Status | Checks | Spec Versions | Notes | |
| 92 | +|----------|--------|--------|---------------|-------| |
| 93 | +| auth/token-endpoint-auth-post | PASS | 19/19 | 2025-06-18, 2025-11-25 | | |
| 94 | +| auth/token-endpoint-auth-none | PASS | 19/19 | 2025-06-18, 2025-11-25 | | |
| 95 | +| auth/token-endpoint-auth-basic | PASS | 19/19 | 2025-06-18, 2025-11-25 | | |
| 96 | +| auth/scope-step-up | FAIL | 13/14 | 2025-11-25 | | |
| 97 | +| auth/scope-retry-limit | PASS | 11/11 | 2025-11-25 | | |
| 98 | +| auth/scope-omitted-when-undefined | PASS | 15/15 | 2025-11-25 | | |
| 99 | +| auth/scope-from-www-authenticate | PASS | 11/11 | 2025-11-25 | | |
| 100 | +| auth/scope-from-scopes-supported | PASS | 15/15 | 2025-11-25 | | |
| 101 | +| auth/pre-registration | PASS | 14/14 | 2025-11-25 | | |
| 102 | +| auth/metadata-var3 | FAIL | 0/4 | 2025-11-25 | | |
| 103 | +| auth/metadata-var2 | PASS | 14/14 | 2025-11-25 | | |
| 104 | +| auth/metadata-var1 | PASS | 14/14 | 2025-11-25 | | |
| 105 | +| auth/metadata-default | PASS | 14/14 | 2025-11-25 | | |
| 106 | +| auth/basic-cimd | PASS | 14/14 | 2025-11-25 | | |
| 107 | +| auth/2025-03-26-oauth-metadata-backcompat | PASS | 12/12 | 2025-03-26 | | |
| 108 | +| auth/2025-03-26-oauth-endpoint-fallback | FAIL | 0/3 | 2025-03-26 | | |
| 109 | + |
| 110 | +### Auth Scenarios (Informational — not scored) |
| 111 | + |
| 112 | +| Scenario | Status | Checks | Spec Versions | |
| 113 | +|----------|--------|--------|---------------| |
| 114 | +| auth/resource-mismatch | FAIL | 14/15 | draft | |
| 115 | +| auth/cross-app-access-complete-flow | FAIL | 10/12 | extension | |
| 116 | +| auth/client-credentials-jwt | FAIL | 4/5 | extension | |
| 117 | +| auth/client-credentials-basic | PASS | 9/9 | extension | |
| 118 | + |
| 119 | +--- |
| 120 | + |
| 121 | +## Issue Triage Details |
| 122 | + |
| 123 | +Analysis period: Last 64 issues |
| 124 | +Labels present: question, good first issue, help wanted (3/12) |
| 125 | +Uses issue types: No |
| 126 | + |
| 127 | +| Metric | Value | T1 Req | T2 Req | Verdict | |
| 128 | +|--------|-------|--------|--------|---------| |
| 129 | +| Compliance rate | 14.1% | >= 90% | >= 80% | FAIL | |
| 130 | +| Triaged within SLA | 9 | — | — | — | |
| 131 | +| Exceeding SLA | 54 | — | — | — | |
| 132 | +| Median triage time | 4341.3h | — | — | — | |
| 133 | +| P95 triage time | 8095.1h | — | — | — | |
| 134 | +| Open P0s | 0 | 0 | 0 | PASS | |
| 135 | + |
| 136 | +--- |
| 137 | + |
| 138 | +## Documentation Coverage |
| 139 | + |
| 140 | +### Documentation Coverage Assessment |
| 141 | + |
| 142 | +**SDK path**: ~/Development/rust-sdk |
| 143 | +**Documentation locations found**: |
| 144 | + |
| 145 | +- README.md: Top-level overview, basic client/server setup |
| 146 | +- crates/rmcp/README.md: Core library docs with quick start, transport options, feature flags, structured output, tasks |
| 147 | +- examples/README.md: Quick start with Claude Desktop |
| 148 | +- examples/servers/README.md: Server example descriptions |
| 149 | +- examples/clients/README.md: Client example descriptions |
| 150 | +- docs/OAUTH_SUPPORT.md: OAuth 2.1 authorization documentation |
| 151 | +- crates/rmcp-macros/README.md: Macro crate documentation |
| 152 | + |
| 153 | +#### Feature Documentation Table |
| 154 | + |
| 155 | +| # | Feature | Documented? | Where | Has Examples? | Verdict | |
| 156 | +|---|---------|-------------|-------|---------------|---------| |
| 157 | +| 1 | Tools - listing | Yes | crates/rmcp/README.md:21-90 | Yes (1 example) | PASS | |
| 158 | +| 2 | Tools - calling | Yes | crates/rmcp/README.md:21-90, examples/clients/README.md | Yes (2 examples) | PASS | |
| 159 | +| 3 | Tools - text results | Yes | crates/rmcp/README.md:50-60 | Yes (1 example) | PASS | |
| 160 | +| 4 | Tools - image results | No | — | No | FAIL | |
| 161 | +| 5 | Tools - audio results | No | — | No | FAIL | |
| 162 | +| 6 | Tools - embedded resources | No | — | No | FAIL | |
| 163 | +| 7 | Tools - error handling | No | — | No | FAIL | |
| 164 | +| 8 | Tools - change notifications | No | — | No | FAIL | |
| 165 | +| 9 | Resources - listing | No | — | Yes (example in everything_stdio.rs) | PARTIAL | |
| 166 | +| 10 | Resources - reading text | No | — | Yes (example in everything_stdio.rs) | PARTIAL | |
| 167 | +| 11 | Resources - reading binary | No | — | No | FAIL | |
| 168 | +| 12 | Resources - templates | No | — | Yes (example in everything_stdio.rs) | PARTIAL | |
| 169 | +| 13 | Resources - template reading | No | — | No | FAIL | |
| 170 | +| 14 | Resources - subscribing | No | — | No | FAIL | |
| 171 | +| 15 | Resources - unsubscribing | No | — | No | FAIL | |
| 172 | +| 16 | Resources - change notifications | No | — | No | FAIL | |
| 173 | +| 17 | Prompts - listing | No | — | Yes (example in everything_stdio.rs) | PARTIAL | |
| 174 | +| 18 | Prompts - getting simple | No | — | Yes (example in everything_stdio.rs) | PARTIAL | |
| 175 | +| 19 | Prompts - getting with arguments | No | — | Yes (example in everything_stdio.rs) | PARTIAL | |
| 176 | +| 20 | Prompts - embedded resources | No | — | No | FAIL | |
| 177 | +| 21 | Prompts - image content | No | — | No | FAIL | |
| 178 | +| 22 | Prompts - change notifications | No | — | No | FAIL | |
| 179 | +| 23 | Sampling - creating messages | No | — | Yes (servers/sampling_stdio.rs, clients/sampling_stdio.rs) | PARTIAL | |
| 180 | +| 24 | Elicitation - form mode | Yes | examples/servers/README.md:38-53 | Yes (elicitation_stdio.rs) | PASS | |
| 181 | +| 25 | Elicitation - URL mode | No | — | No | FAIL | |
| 182 | +| 26 | Elicitation - schema validation | No | — | No | FAIL | |
| 183 | +| 27 | Elicitation - default values | No | — | No | FAIL | |
| 184 | +| 28 | Elicitation - enum values | No | — | Yes (elicitation_enum_inference.rs) | PARTIAL | |
| 185 | +| 29 | Elicitation - complete notification | No | — | No | FAIL | |
| 186 | +| 30 | Roots - listing | No | — | No | FAIL | |
| 187 | +| 31 | Roots - change notifications | No | — | No | FAIL | |
| 188 | +| 32 | Logging - sending log messages | No | — | No | FAIL | |
| 189 | +| 33 | Logging - setting level | No | — | No | FAIL | |
| 190 | +| 34 | Completions - resource argument | No | — | Yes (completion_stdio.rs) | PARTIAL | |
| 191 | +| 35 | Completions - prompt argument | No | — | Yes (completion_stdio.rs) | PARTIAL | |
| 192 | +| 36 | Ping | No | — | No | FAIL | |
| 193 | +| 37 | Streamable HTTP transport (client) | Yes | crates/rmcp/README.md:175-195 | Yes (clients/streamable_http.rs) | PASS | |
| 194 | +| 38 | Streamable HTTP transport (server) | Yes | crates/rmcp/README.md:175-195 | Yes (servers/counter_streamhttp.rs) | PASS | |
| 195 | +| 39 | SSE transport - legacy (client) | No | — | No | FAIL | |
| 196 | +| 40 | SSE transport - legacy (server) | No | — | No | FAIL | |
| 197 | +| 41 | stdio transport (client) | Yes | crates/rmcp/README.md:140-165 | Yes (clients/git_stdio.rs) | PASS | |
| 198 | +| 42 | stdio transport (server) | Yes | crates/rmcp/README.md:21-90 | Yes (servers/counter_stdio.rs) | PASS | |
| 199 | +| 43 | Progress notifications | No | — | Yes (servers/progress_demo.rs, clients/progress_client.rs) | PARTIAL | |
| 200 | +| 44 | Cancellation | No | — | No | FAIL | |
| 201 | +| 45 | Pagination | No | — | No | FAIL | |
| 202 | +| 46 | Capability negotiation | No | — | No | FAIL | |
| 203 | +| 47 | Protocol version negotiation | No | — | No | FAIL | |
| 204 | +| 48 | JSON Schema 2020-12 support | Yes | README.md:32-33, crates/rmcp/README.md:92-120 | Yes (structured output example) | PASS | |
| 205 | +| — | Tasks - get (experimental) | Yes | crates/rmcp/README.md (Tasks section) | No | INFO | |
| 206 | +| — | Tasks - result (experimental) | Yes | crates/rmcp/README.md (Tasks section) | No | INFO | |
| 207 | +| — | Tasks - cancel (experimental) | Yes | crates/rmcp/README.md (Tasks section) | No | INFO | |
| 208 | +| — | Tasks - list (experimental) | No | — | No | INFO | |
| 209 | +| — | Tasks - status notifications (experimental) | No | — | No | INFO | |
| 210 | + |
| 211 | +#### Summary |
| 212 | + |
| 213 | +**Total non-experimental features**: 48 |
| 214 | +**PASS (documented with examples)**: 9/48 |
| 215 | +**PARTIAL (documented or examples only)**: 11/48 |
| 216 | +**FAIL (not documented)**: 28/48 |
| 217 | + |
| 218 | +**Core features documented**: ~6/36 (16.7%) |
| 219 | +**All features documented with examples**: 9/48 (18.8%) |
| 220 | + |
| 221 | +#### Tier Verdicts |
| 222 | + |
| 223 | +**Tier 1** (all non-experimental features documented with examples): **FAIL** |
| 224 | + |
| 225 | +- 39 features missing full documentation with examples |
| 226 | + |
| 227 | +**Tier 2** (basic docs covering core features): **FAIL** |
| 228 | + |
| 229 | +- Most core features (resources, prompts, sampling, roots, logging, completions, notifications, subscriptions) lack prose documentation |
| 230 | +- Only tools (basic), transports (stdio, streamable HTTP), elicitation (form mode), and JSON Schema have adequate prose docs |
| 231 | + |
| 232 | +--- |
| 233 | + |
| 234 | +## Policy Evaluation |
| 235 | + |
| 236 | +### Policy Evaluation Assessment |
| 237 | + |
| 238 | +**SDK path**: ~/Development/rust-sdk |
| 239 | +**Repository**: modelcontextprotocol/rust-sdk |
| 240 | + |
| 241 | +--- |
| 242 | + |
| 243 | +#### 1. Dependency Update Policy: PASS |
| 244 | + |
| 245 | +| File | Exists (CLI) | Content Verdict | |
| 246 | +|------|-------------|----------------| |
| 247 | +| DEPENDENCY_POLICY.md | No | N/A | |
| 248 | +| docs/dependency-policy.md | No | N/A | |
| 249 | +| .github/dependabot.yml | Yes | Configured — weekly Cargo updates, daily GitHub Actions updates, with PR limits and labeling | |
| 250 | +| .github/renovate.json | No | N/A | |
| 251 | + |
| 252 | +**Verdict**: **PASS** — Dependabot is properly configured with weekly Cargo dependency updates and daily GitHub Actions updates. |
| 253 | + |
| 254 | +--- |
| 255 | + |
| 256 | +#### 2. Roadmap: FAIL |
| 257 | + |
| 258 | +| File | Exists (CLI) | Content Verdict | |
| 259 | +|------|-------------|----------------| |
| 260 | +| ROADMAP.md | No | N/A | |
| 261 | +| docs/roadmap.md | No | N/A | |
| 262 | + |
| 263 | +**Verdict**: |
| 264 | + |
| 265 | +- **Tier 1**: **FAIL** — No roadmap file exists. |
| 266 | +- **Tier 2**: **FAIL** — No roadmap or plan-toward-Tier-1 file exists. |
| 267 | + |
| 268 | +--- |
| 269 | + |
| 270 | +#### 3. Versioning Policy: FAIL |
| 271 | + |
| 272 | +| File | Exists (CLI) | Content Verdict | |
| 273 | +|------|-------------|----------------| |
| 274 | +| VERSIONING.md | No | N/A | |
| 275 | +| docs/versioning.md | No | N/A | |
| 276 | +| BREAKING_CHANGES.md | No | N/A | |
| 277 | +| CONTRIBUTING.md (versioning section) | No | N/A | |
| 278 | + |
| 279 | +**Verdict**: |
| 280 | + |
| 281 | +- **Tier 1**: **FAIL** — No versioning or breaking change documentation exists. |
| 282 | +- **Tier 2**: **N/A** — only requires stable release. |
| 283 | + |
| 284 | +--- |
| 285 | + |
| 286 | +#### Overall Policy Summary |
| 287 | + |
| 288 | +| Policy Area | Tier 1 | Tier 2 | |
| 289 | +|-------------|--------|--------| |
| 290 | +| Dependency Update Policy | PASS | PASS | |
| 291 | +| Roadmap | FAIL | FAIL | |
| 292 | +| Versioning Policy | FAIL | N/A | |
0 commit comments