Add API endpoint and UI for deadline callback log visibility#66610
Add API endpoint and UI for deadline callback log visibility#66610seanghaeli wants to merge 25 commits into
Conversation
2b1e618 to
9102967
Compare
e303ead to
f6103b3
Compare
7f042b1 to
b300616
Compare
|
I like the interface but I would like to have @bbovenzi opinion on this one |
|
@ashb updated to use |
The rebase conflict resolution dropped the callback logs service function. Re-add it to match the types.gen.ts and backend endpoint.
The manual merge conflict resolution left stale URL/params in services.gen.ts that didn't match the actual API endpoint signature (dag_id, dag_run_id, callback_id vs callback_id, try_number).
Async (TriggererCallback) deadline callbacks now have their logs captured
to a dedicated file path that the UI log endpoint can read, matching the
pattern already established for sync (ExecutorCallback) logs.
Changes:
- TriggerLoggingFactory.ti is now optional (None for callback triggers)
- _create_workload sets up logger_cache for callback triggers without a
task instance, writing to triggerer_callbacks/{dag_id}/{run_id}/{cb_id}
- callback_log_reader checks both executor_callbacks/ and
triggerer_callbacks/ paths when reading logs
- upload_to_remote gracefully handles ti=None by uploading directly
via the remote log handler
Address review feedback from Ash — use the existing log component (TaskLogContent) instead of a custom renderer so callback logs get the same filtering, formatting, grouping, and virtualization as task instance logs.
The (datum as StructuredLogMessage) cast was redundant — datum is already narrowed by the typeof check. CI's lint autofix stripped the cast, which left the StructuredLogMessage import unused and triggered TS6196. Remove both the cast and the now-unused import.
…elated to UI changes)
…p_is_per_index (unrelated to UI changes; passed on prior runs)
…p_is_per_index (2nd flake, unrelated to UI changes, from apache#67684 on main)
…p_is_per_index (3rd flake, apache#67684 on main, unrelated to UI PR)
…p_is_per_index (4th flake; apache#67684 on main; unrelated to UI PR)
…p_is_per_index (5th flake; apache#67684 on main; unrelated to UI PR)
…check typo, both unrelated to UI PR
ddaab67 to
7567838
Compare
…ost-rebase); clean re-run
SameerMesiah97
left a comment
There was a problem hiding this comment.
I have not inspected the diff itself, but I saw your dev list post requesting feedback on potential UI additions.
Overall, I think you've covered most of what users would reasonably expect to see for DeadlineAlerts in the UI. One possible enhancement would be surfacing a few additional pieces of metadata:
- Configured interval value
- Interval source (
timedeltavsVariableInterval) - Callback type (for example,
SlackWebhookNotifier)
Users can always inspect the DAG definition for this information, but exposing it directly in the UI would make deadline investgation more self-service. For e.g., if critical SLA is missed and a DeadlineAlert fires, an operator or stakeholder could immediately understand what deadline was configured and what notification mechanism was invoked without needing to inspect the DAG code.
Summary
Make deadline callback logs visible in the Airflow UI. When a deadline is missed and a callback fires, users can now view the callback's execution logs directly from the web interface.
Changes
Backend:
CallbackLogReaderutility (airflow-core/src/airflow/utils/log/callback_log_reader.py) — reads callback logs from remote or local storage using the sameRemoteLogIOinfrastructure as task logsGET /ui/dags/{dag_id}/dagRuns/{dag_run_id}/callbacks/{callback_id}/logson the deadlines UI routerDeadlineResponsedatamodel — addedcallback_idandcallback_statefieldsFrontend:
CallbackLogViewer.tsxcomponent — fetches and displays callback execution logs in a modal dialogMotivation
Kaxil requested this: "Some way to know if a deadline didn't fire and it failed: something on UI would be great, like Task log. If deadlines are visible, it would be even better experience than previous SLAs."
Dependencies
executor_callbacks/{dag_id}/{run_id}/{callback_id}Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.6) following the guidelines
End-to-End Verification
Verified in breeze with full Airflow stack (scheduler + dag-processor + triggerer + API server + LocalExecutor). A test DAG with a
SyncCallbackdeadline was parsed, unpaused, scheduled, and the deadline missed naturally. The callback executed throughsupervise_callback()and produced real structured logs.Dag Run Page — Deadline Status with Callback
Callback Logs Modal
Verified:
test_callback_logs.py)test_callback_log_reader.py)os.path.commonpathcontainment check