Skip to content

[FEATURE] Push Claude Code subscription metrics to the gateway OTLP endpoint #741

Description

@edenreich

Summary

Follow-up to inference-gateway/inference-gateway#404 (client side), required by its acceptance criteria. The gateway now exposes an opt-in OTLP/HTTP metrics push endpoint (POST /v1/metrics, gateway PR inference-gateway/inference-gateway#409).

When infer runs with a Claude Code Max subscription, traffic goes infer CLI -> Claude Code -> Anthropic API and bypasses the gateway entirely, so no metrics are recorded. The CLI should parse Claude Code headless-mode logs and push usage/tool-call metrics to the gateway:

  • Push OTLP JSON (or protobuf) to POST {gateway}/v1/metrics with Content-Type: application/json, delta temporality.
  • Supported metrics: gen_ai.client.token.usage (delta sum per operation, with gen_ai.token.type = input|output), gen_ai.client.operation.duration, gen_ai.client.operation.time_to_first_chunk, gen_ai.execute_tool.duration (histograms, seconds; carry error.type on failures so tool success/failure is derivable), inference_gateway.tool_calls (delta monotonic sum).
  • Attributes: gen_ai.provider.name, gen_ai.request.model, gen_ai.tool.name, gen_ai.tool.type, error.type, and set source to claude-code-subscription so subscription clients are labeled distinctly (anything else is dropped by the gateway's allowlist; source="gateway" is rewritten server-side).
  • Forward the user's bearer token when the gateway runs with auth enabled; handle 403 (push disabled) gracefully.

Example payload: see "Pushing metrics (OTLP)" in the gateway's examples/docker-compose/monitoring/README.md.

Acceptance Criteria

  • infer parses Claude Code headless-mode logs for token usage and tool-call outcomes
  • Metrics are pushed to the gateway's POST /v1/metrics endpoint with source=claude-code-subscription
  • Push failures (endpoint disabled, auth, network) degrade gracefully without breaking the session
  • It's documented and tested

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Fields

    No fields configured for Feature.

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions