Add an evaluator that emits an OpenTelemetry span for each CallableModel invocation, so that workflows built on ccflow are observable through the same tracing infrastructure used elsewhere in a deployment.
OpenTelemetry already defines the schema, propagation rules, and exporter ecosystem, so this is a focused integration rather than a new observability framework. Once spans are emitted, existing OTel tooling (Jaeger, Tempo, Honeycomb, internal collectors, etc.) handles the rest.
Behavior
- Wrap each call in a span named after the model.
- Parent/child relationships follow the natural call hierarchy via the OTel context.
- Set span status from the call outcome (success or error, with the exception recorded on failure).
- Attach a small set of standard attributes (e.g. model identity, context hash) and allow users to configure additional attributes.
Open questions
- Dependency. Should
opentelemetry-api be a hard dependency of ccflow, an optional extra, or live in a separate package?
- Attribute capture. How much of the context and result should be captured by default? Full serialization is convenient but risks leaking large payloads or sensitive fields into the tracing backend. A conservative default plus user-configurable inclusion/exclusion seems right but worth deciding upfront.
- Cross-process propagation. When a
CallableModel invokes a remote service, trace context needs to flow across the boundary. This is largely the user's responsibility (they configure the relevant OTel instrumentation), but the docs should be explicit about it.
- Interaction with the existing
LoggingEvaluator. Both observe the same lifecycle; should they share any underlying call-identity mechanism, or stay independent?
Add an evaluator that emits an OpenTelemetry span for each
CallableModelinvocation, so that workflows built on ccflow are observable through the same tracing infrastructure used elsewhere in a deployment.OpenTelemetry already defines the schema, propagation rules, and exporter ecosystem, so this is a focused integration rather than a new observability framework. Once spans are emitted, existing OTel tooling (Jaeger, Tempo, Honeycomb, internal collectors, etc.) handles the rest.
Behavior
Open questions
opentelemetry-apibe a hard dependency of ccflow, an optional extra, or live in a separate package?CallableModelinvokes a remote service, trace context needs to flow across the boundary. This is largely the user's responsibility (they configure the relevant OTel instrumentation), but the docs should be explicit about it.LoggingEvaluator. Both observe the same lifecycle; should they share any underlying call-identity mechanism, or stay independent?