Skip to content

Design: per-tenant scheduling and queue-fairness model for hosted AMS #5223

Description

@JSONbored

Problem

ORB's Phase B has ten granular scaling/capacity issues (#4903-4914), covering db sharding, per-tenant cron scheduling, queue fairness, fleet-scale admission control, load testing, and published capacity numbers. AMS has exactly one generic counterpart in this milestone — #4942, re-evaluating the local concurrency model — and nothing scoped to per-tenant fairness specifically. AMS's existing concurrency logic (iterate-loop.ts admission, submission-gate.ts, portfolio-queue.js) was built assuming one operator's own compute; it has no notion of fairness across multiple tenants sharing the same infra.

Area

AMS / Scaling & capacity

Proposal

Design how AMS's existing single-process concurrency assumptions generalize to fair scheduling across many hosted tenants sharing compute, as a companion to the broader concurrency-model re-evaluation already seeded in this milestone (#4942). The design should define per-tenant admission and fairness rules and how they compose with — rather than replace — the existing single-machine concurrency logic where that logic is still correct.

Deliverables

  • A design doc covering per-tenant admission/fairness rules and how they compose with the existing single-machine concurrency logic in iterate-loop.ts, submission-gate.ts, and portfolio-queue.js
  • An explicit identification of which existing modules need interface changes to support per-tenant fairness vs. which stay untouched

Resources

Boundaries

  • Companion to, not a duplicate of, Re-evaluate the local concurrency model for a shared-service context #4942 — this issue scopes the multi-tenant fairness layer specifically; it should not re-litigate the single-machine concurrency model itself except where per-tenant fairness requires it to change.
  • Design only: no changes to iterate-loop.ts, submission-gate.ts, or portfolio-queue.js in this issue.
  • Must not assume the storage-abstraction or provisioning-API designs elsewhere in this milestone — reference their eventual tenant-identification shape rather than inventing a competing one.
  • Scheduling fairness across tenants sharing compute is a core multi-tenant architecture decision with direct availability implications if under-specified; any gap in the fairness rules should block downstream implementation, not be patched in later ad hoc.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.roadmapOn the Wave-2 agent-layer roadmap board (project 9)

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions