Skip to content

Add RetryEvaluator #224

@ptomecek

Description

@ptomecek

Add a generic retry evaluator for wrapping CallableModel invocations that may fail transiently (flaky APIs, databases, object stores, etc.).

This pattern is currently re-implemented ad-hoc by users. A first-class evaluator would standardize the behavior and compose cleanly with the rest of the evaluator stack.

Behavior

  • Configurable maximum number of attempts.
  • Configurable backoff policy (at minimum: constant and exponential, with optional jitter).
  • Configurable allow-list and deny-list of exception types to retry on.
  • Optional hook invoked between attempts (for logging, metrics, etc.).

Open questions

  • Idempotency. Retries assume the wrapped model is safe to re-invoke with the same context. Should CallableModel gain an explicit idempotency flag that this evaluator respects, or should it be the user's responsibility to only stack retry on idempotent models?
  • Interaction with caching. When stacked with MemoryCacheEvaluator, transient failures should not be cached. Need to confirm or adjust the cache evaluator's behavior on exceptions, and document the canonical stacking order.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions