Decouple policy ordering from the mandatory STAGE-declaration contract

[OmarAlJarrah]

Current design

Policy ordering for the staged-builder path is a public IntEnum taxonomy, Stage, with a fixed set of 16 spaced ordinals (packages/dexpace-sdk-core/src/dexpace/sdk/core/pipeline/stage.py):

OPERATION = 50
REDIRECT = 100
POST_REDIRECT = 150
RETRY = 200
...
SEND = 1300

Every concrete Policy (and AsyncPolicy) must declare STAGE: ClassVar[Stage], and this is enforced at class-creation time: __init_subclass__ raises TypeError on import for any concrete subclass that omits it (pipeline/policy.py:43-63, pipeline/async_policy.py:39-50).

There are two construction paths, and they treat STAGE very differently:

• StagedPipelineBuilder reads STAGE to slot policies into stage buckets and flatten them in stage order at build() (pipeline/staged_builder.py).
• The list constructor Pipeline(transport, policies=[...]) orders policies purely by their position in the list and never reads STAGE at all (pipeline/pipeline.py:62-89). The docstring even states this directly: "The list-form Pipeline(client, policies=[...]) constructor ignores STAGE — declaring it is still required for consistency."

So a user who only ever uses the list constructor still pays a mandatory, import-time STAGE-declaration tax for a value that constructor ignores.

Trade-off / concern

Two things are worth reconsidering:

1. The authoring requirement is coupled to a mechanism only one constructor uses. Forcing STAGE on every concrete policy — enforced as a blanket import-time TypeError — punishes list-constructor authors for omitting something their constructor never consults. The requirement and the staged-ordering mechanism are independent concerns that got fused.

2. The ordering vocabulary is a closed numeric enum, and the taxonomy bakes in assumptions. It assumes a single auth pillar (AUTH), redirect-before-retry, and tracing pinned to POST_LOGGING. A third-party policy with a genuinely novel ordering need has no stage to express it and must abuse one of the POST_* buckets. The spaced numbering (50, 100, 150, …, explicitly a renumber-avoidance device per the module docstring) and a reserved-but-unused SERDE value are signals of an ordering model that wants to be open sitting behind a closed enum. There is also an expressiveness mismatch: a policy's real constraint is usually relative ("after auth, before logging"), but STAGE forces it to commit to an absolute ordinal.

This is squarely a third-party-extensibility and authoring-ergonomics concern. The built-in stack is fine; the friction shows up for downstream packages defining their own policies.

Proposed direction

A few options, roughly in increasing ambition:

1. Make STAGE optional and move enforcement to the point of use. StagedPipelineBuilder.append would raise a clear error when handed a policy with no STAGE, instead of a blanket import-time gate. List-constructor authors then declare nothing they do not need. This is the smallest change and removes the dissonance the policy.py docstring already admits.

2. Express ordering as relative constraints. Replace (or supplement) absolute ordinals with before= / after= constraints (e.g. after=AuthPolicy, before=LoggingPolicy) resolved by a topological sort at build time. This captures the real intent and survives third-party policies that do not fit the fixed taxonomy. Trade-off: more builder code, and cyclic specs become possible and need good diagnostics.

3. If keeping ordinals, make Stage an extensible registry that downstream packages can register new stages into, rather than a frozen IntEnum.

Acknowledging the current rationale

The staged design is deliberate and the rationale is sound: per stage.py, ordering by stage rather than caller order "remov[es] a class of bugs where retry runs before redirect or auth runs after logging." That guarantee is worth keeping, and it is fully preservable under option 2 — topological constraints encode exactly that "retry after redirect" relationship without a closed numeric vocabulary. The point here is not to drop staged ordering, but to stop coupling a mechanism only StagedPipelineBuilder uses to a mandatory import-time contract every policy author pays, and to give third-party policies a way to express ordering needs the fixed enum cannot.