Task SDK: Variable.set/delete swallow API errors — a rejected write does not fail the task

[khmelevskiy]

Under which category would you file this issue?

Airflow Core

Apache Airflow version

3.2.2

What happened and how to reproduce it?

Issue Description

Variable.set() and Variable.delete() in the Task SDK catch AirflowRuntimeError
and only log it — they do not re-raise:

# airflow/sdk/definitions/variable.py
@classmethod
def set(cls, key, value, ...):
    try:
        return _set_variable(...)
    except AirflowRuntimeError as e:
        log.exception(e)        # swallowed, not re-raised

@classmethod
def delete(cls, key):
    try:
        _delete_variable(key=key)
    except AirflowRuntimeError as e:
        log.exception(e)        # swallowed, not re-raised

So when the Execution API rejects a variable write (any non-2xx — e.g. a 403
authorization denial, or an API_SERVER_ERROR), the task logs an error traceback
but still finishes as success, and the variable is left unchanged.

This is inconsistent with Variable.get(), which raises on error — and since
#66575 even refuses secrets-backend fallback on a 401/403 from the Execution API.
Reads fail loudly on denial; writes silently succeed.

Steps to reproduce

1. Make PUT /execution/variables/{key} return a non-2xx for a running task.
   (In our case a custom ASGI middleware enforces per-team access and returns 403;
   any non-2xx that surfaces as AirflowRuntimeError reproduces it.)
2. In a task, call Variable.set("some_key", "v").
3. Observed: the log shows the 403 / AirflowRuntimeError: API_SERVER_ERROR
   traceback, the task state is success, and some_key is not written.

The swallow is unconditional in the code above, so the behaviour does not depend
on the specific error or deployment.

What you think should happen instead?

A write the Execution API rejects should fail the task, consistent with
Variable.get() on the read path. At minimum the error should propagate
(or swallowing be made opt-in), so a task that could not persist a variable
does not report success.

Operating System

Linux (official apache/airflow image; OS-independent)

Deployment

Other 3rd-party Helm chart

Apache Airflow Provider(s)

No response

Versions of Apache Airflow Providers

No response

Official Helm Chart version

Not Applicable

Kubernetes Version

No response

Helm Chart configuration

No response

Docker Image customizations

No response

Anything else?

Occurs every time the write is rejected. Related: #66575 (fixed the analogous
read / secrets-backend path) and #60125 (Execution API access-control umbrella —
the source of the 403 in our setup). Not a security report: the write is
correctly rejected server-side; the bug is only that the task still reports
success.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct