Add an evaluator that enforces exactly-once execution semantics for CallableModels that perform external side effects (sending notifications, posting webhooks, writing to external systems, kicking off downstream jobs, etc.).
This is distinct from durable caching. Durable caching is an optimization — a missing entry simply means recompute. Exactly-once is a correctness guarantee — a missing ledger entry means "the side effect has not happened, run it now," and a present entry means "do not run it again under any circumstances." Clearing the ledger has real-world consequences and should not be casual.
Behavior
- Maintain a durable ledger of completed
(model, context) invocations.
- On invocation, check the ledger; if present, skip execution; if absent, execute and record completion atomically.
- Pluggable ledger backend.
Open questions
- Return value on skip. When a call is skipped because the ledger already records completion, what does the evaluator return? Options include a stored minimal acknowledgment, a typed sentinel, or raising. Each has tradeoffs for downstream composition.
- Atomicity. The side effect and the ledger write are not generally co-transactional. How do we minimize the window where the side effect has happened but the ledger has not yet been updated, and what's the recommended user-side pattern when this matters?
- Relationship to durable caching. Should the two evaluators share a storage abstraction, or stay deliberately separate to keep the semantics from blurring?
Add an evaluator that enforces exactly-once execution semantics for
CallableModels that perform external side effects (sending notifications, posting webhooks, writing to external systems, kicking off downstream jobs, etc.).This is distinct from durable caching. Durable caching is an optimization — a missing entry simply means recompute. Exactly-once is a correctness guarantee — a missing ledger entry means "the side effect has not happened, run it now," and a present entry means "do not run it again under any circumstances." Clearing the ledger has real-world consequences and should not be casual.
Behavior
(model, context)invocations.Open questions