feat(capman): Add DynamicConcurrentQueries allocation policy

[phacops]

Add an allocation policy that lets every org run queries while the cluster has spare capacity, and progressively sheds load from the heaviest orgs as it approaches its sustainable concurrent-query limit. It is governed by two soft limits — a global one and a per-org one — rather than a single hard per-tenant cap.

Shedding behavior

Concurrent queries are tracked in two Redis buckets: a global bucket (scoped per storage, i.e. per ClickHouse cluster) and a per-org bucket. While the global concurrent count is at or below global_soft_limit, every query for every org runs. Once it crosses the soft limit, the effective per-org ceiling shrinks proportionally to global pressure:

effective_limit = max(1, floor(per_org_soft_limit * global_soft_limit / global_concurrent))

So the orgs furthest above their per-org soft limit are rejected first, with more orgs shed as the cluster gets busier. An org may always run at least one query, keeping the impact focused on the heaviest senders rather than starving small orgs. Per-org soft limits are overridable per organization via organization_soft_limit_override.

CBRS and snuba-admin integration

The policy is wired into the shared EAP routing strategy allocation-policy list so it runs in the CBRS path alongside the existing policies, dormant by default (is_active=0, is_enforced=0) to match the staged-rollout pattern of its neighbors. It composes with regular allocation policies through the existing combine logic (can_run = all, max_threads = min) and is auto-discovered by snuba-admin capacity management for both routing strategies and regular storages — no hardcoded registration. It can also be attached to any storage YAML under allocation_policies.

Agent transcript: https://claudescope.sentry.dev/share/kU8cZ7sz8002CldlP3Ol3TOwLD2U0-I64cIH9-aj68U

[MeredithAnya]

sure why not, lets give it a go