DefaultClientRequestObservationConvention produces combinatorial URI tag cardinality with optional @RequestParam

[gustavovnicius]

When using HTTP Interface clients (@HttpExchange) with @RequestParam parameters, DefaultClientRequestObservationConvention produces a different uri low-cardinality key value for each combination of present/absent parameters. This defeats URI templating for metrics and causes cardinality explosion in Prometheus-based monitoring.

Example 1: optional parameters (2^n cardinality)

@HttpExchange("/api/v1/items")
public interface ItemClient {

    @GetExchange
    List<Item> search(@RequestParam Optional<String> category,
                      @RequestParam Optional<Integer> limit,
                      @RequestParam Optional<String> sort,
                      @RequestParam Optional<String> status);
}

Depending on which optional parameters are passed, the uri tag on http.client.requests takes different values:

/api/v1/items?{queryParam-category}={queryParam-category[0]}&{queryParam-limit}={queryParam-limit[0]}&{queryParam-sort}={queryParam-sort[0]}&{queryParam-status}={queryParam-status[0]}
/api/v1/items?{queryParam-category}={queryParam-category[0]}&{queryParam-limit}={queryParam-limit[0]}&{queryParam-status}={queryParam-status[0]}
/api/v1/items?{queryParam-limit}={queryParam-limit[0]}
/api/v1/items

4 optional params = up to 16 distinct time series for a single endpoint.

Example 2: collection parameters (unbounded cardinality)

@HttpExchange("/api/v1/products")
public interface ProductClient {

    @GetExchange("/by-ids")
    List<Product> findByIds(@RequestParam List<Long> ids);
}

Each call with a different number of list elements produces a distinct uri tag value:

/api/v1/products/by-ids?{queryParam-ids}={queryParam-ids[0]}
/api/v1/products/by-ids?{queryParam-ids}={queryParam-ids[0]}&{queryParam-ids}={queryParam-ids[1]}
/api/v1/products/by-ids?{queryParam-ids}={queryParam-ids[0]}&{queryParam-ids}={queryParam-ids[1]}&{queryParam-ids}={queryParam-ids[2]}
...

This creates unbounded cardinality — one distinct time series per list size. A single endpoint called with varying batch sizes generates an ever-growing number of Prometheus time series, effectively making the uri tag as high-cardinality as the request itself. This is the exact scenario low-cardinality keys exist to prevent.

Impact

• Aggregation breaks. sum by (uri) (rate(http_client_requests_seconds_count[5m])) splits what should be one endpoint into multiple series. There is no way to get a total request rate for /api/v1/items without enumerating all variants.
• Alerting and dashboards miss variants. Any panel or alert keyed on uri only catches the variants its author knew about at the time.
• Prometheus cardinality bomb. Once Micrometer registers a meter with a given tag set, that time series is reported on every scrape for the lifetime of the process, even if no new samples arrive. With collection parameters, every distinct list size observed during a pod's uptime becomes a permanent time series. A service processing varying batch sizes accumulates an ever-growing number of series for a single endpoint, with no way to reclaim them short of a restart. This increases Prometheus storage, memory, and query cost, and can trigger cardinality limits that drop entire metric families.

Inconsistency with server-side @RequestMapping

Server-side @RequestMapping("/api/v1/items") with @RequestParam parameters produces uri="/api/v1/items" in http.server.requests. Query parameters are never part of the server-side uri tag. This means the same endpoint observed from both sides gets different treatment:

• Server (http.server.requests): uri="/api/v1/items"
• Client (http.client.requests): uri="/api/v1/items?{queryParam-category}={queryParam-category[0]}&..."

You cannot join client and server metrics for the same endpoint because the uri tags don't match.

Inconsistency with UriBuilder-based usage

When using RestClient or WebClient with UriBuilder, query parameters added via .queryParam() do not appear in the uri tag at all:

// UriBuilder approach — uri tag: /api/v1/items
restClient.get()
    .uri(b -> b.path("/api/v1/items").queryParam("category", "foo").build())
    .retrieve()
    .body(String.class);

HTTP Interface clients with @RequestParam are semantically doing the same thing — programmatic query parameter addition — but the implementation leaks the params into the URI template, producing a different (and high-cardinality) uri tag. The behavior should be consistent: programmatically-added query parameters should not affect the uri observation key regardless of whether they are added via UriBuilder.queryParam() or @RequestParam.

Root cause

DefaultClientRequestObservationConvention.extractPath() only strips the scheme and authority from the URI template. The query string portion, including the {queryParam-*} placeholders added by HttpRequestValues.appendQueryParams(), is preserved in the uri low-cardinality key.

The uri key is designed to be low-cardinality for metric aggregation. Including query parameter structure in it undermines that purpose. Path variables ({id}) are different: they represent endpoint structure. Query parameters represent filtering options and their presence/absence should not create distinct metric series.

Expected behavior

The uri low-cardinality key value should contain only the path portion of the URI template. All of the above examples should produce:

/api/v1/items
/api/v1/products/by-ids

Path variables like {id} should continue to be preserved since they identify distinct endpoints.

Suggested fix

DefaultClientRequestObservationConvention.extractPath() could strip the query string (everything from ? onward) before returning:

private static String extractPath(String uriTemplate) {
    // existing scheme+authority stripping...
    int queryStart = result.indexOf('?');
    if (queryStart >= 0) {
        result = result.substring(0, queryStart);
    }
    return result;
}

This is backward-compatible: services that don't use @RequestParam see no change. The high-cardinality http.url key already captures the full URI for tracing, so no information is lost.

Current workaround

We work around this by auto-configuring a MeterFilter that strips everything after ? from the uri tag on http.client.requests metrics.

Relation to prior issues

• Improve query params in uri KeyValue with HTTP interface client #34176 improved query param variable naming (from indexed to named)
• Request param handling in HttpRequestValues overrides existing URI variables with same name #34499 added the queryParam- prefix to avoid path variable collisions
• Improve HTTP Client Metrics URI KeyValue #35264 reported the resulting URI as "ugly" and was closed

This issue is distinct: the problem is not the naming convention. It is that query parameter templates appear in a low-cardinality key at all, creating combinatorial and unbounded cardinality that breaks metric aggregation. Even if the naming were perfect ({category} instead of {queryParam-category}), the cardinality problem would remain.