[FEATURE][RUST]: Native OTEL span emission for Rust MCP runtime fast path

[crivetimihai]

🧭 Type of Feature

• Enhancement to existing functionality
• New feature or capability
• New MCP-compliant server
• New component or integration
• Developer tooling or test improvement
• Packaging, automation and deployment (ex: pypi, docker, quay.io, kubernetes, terraform)
• Other (please describe below)

🧭 Epic

Title: Native OTEL span emission for the Rust MCP runtime fast path
Goal: Keep the Rust public MCP runtime on the direct execution path while preserving full observability parity with the Python runtime.
Why now: In Rust full mode, observability currently works by preserving trace/auth context across the Rust→Python seam and forcing traced tools/call executions back through Python. That keeps Langfuse/OTEL behavior correct, but it adds avoidable latency and prevents the Rust fast path from being fully performance-equivalent when tracing is enabled.

---

🧑🏻‍💻 User Story 1

As a: platform operator using Rust MCP mode with observability enabled
I want: direct Rust tools/call execution to emit OTEL spans natively
So that: I keep Langfuse/OTEL visibility without triggering the Python fallback path

✅ Acceptance Criteria

Scenario: Rust direct tools/call emits native spans
  Given ContextForge is running in Rust full mode with OTEL enabled
  When a client calls a tool through the Rust public MCP transport
  Then the request stays on the Rust direct execution path
  And the runtime emits OTEL spans without requiring the Python observability fallback
  And the spans appear in the configured OTLP backend

Scenario: Observability no longer forces Python fallback
  Given a tool is otherwise eligible for Rust direct execution
  When tracing is active
  Then the gateway does not mark the request with fallbackReason "observability-trace-active"
  And the tool call is still traced end to end

---

🧑🏻‍💻 User Story 2

As a: operator or developer comparing Python and Rust runtime behavior
I want: parity in trace semantics and safe payload handling
So that: switching runtimes does not change the meaning or safety of observability data

✅ Acceptance Criteria

Scenario: Rust trace metadata matches Python semantics
  Given the same authenticated tool call is executed in Python mode and Rust mode
  When spans are exported to OTLP
  Then both runtimes emit equivalent user, session, auth-method, and team context
  And both runtimes emit equivalent trace naming, status, and error semantics

Scenario: Rust payload handling matches Python safety rules
  Given an instrumented tool call includes sensitive inputs, outputs, or error text
  When the Rust runtime exports span attributes or events
  Then payload capture respects the same redaction and size-limit policy as the Python runtime
  And secrets are not leaked in trace attributes, events, or status text

---

📐 Design Sketch (optional)

flowchart TD
    A[MCP Client] --> B[Rust public MCP transport]
    B --> C[Rust direct tools/call]
    C --> D[Native OTEL span emission]
    D --> E[Configured OTLP backend]
    C -. only for non-observability unsupported cases .-> F[Python fallback]

Loading

---

🔗 MCP Standards Check

• Change adheres to current MCP specifications
• No breaking changes to existing MCP-compliant integrations
• If deviations exist, please describe them below:

---

🔄 Alternatives Considered

• Keep the current Python fallback for traced requests.
  • Correct today, but imposes avoidable performance overhead in Rust mode.
• Emit only minimal Rust spans and keep richer trace decoration in Python.
  • Lower implementation effort, but leaves runtime behavior inconsistent and harder to reason about.

---

📓 Additional Context

Current behavior is explicitly documented in mcpgateway/services/tool_service.py: direct Rust tools/call does not emit native OTEL spans yet, so traced executions fall back to Python with fallbackReason: observability-trace-active.

This issue should cover:

• native OTEL span creation in tools_rust/mcp_runtime
• propagation or recreation of the current trace context in Rust
• parity for Langfuse/OTLP-facing attributes and status/error handling
• regression coverage proving that Rust direct execution remains traced without the observability-driven fallback