Run each async udf in its own thread

[nkitsaini]

Note

High Risk
Changes core UDF concurrency, cancellation, and embeddings HTTP behavior under load; misconfiguration or regressions could affect timeouts, pooling, and concurrent EMBED_TEXT-style workloads.

Overview
Reworks Python UDF / external-function execution so async calls run on a single dedicated asyncio dispatch thread (fixing loop-bound resource reuse for httpx pools and enabling concurrent async requests), while sync UDFs use persistent per-worker-thread event loops instead of asyncio.run per request. Adds _cancellable_run with threading.Event cancellation, offloads heavy parse/serialize to the executor, and tightens cancel signaling on timeout/disconnect.

Embeddings get a default httpx.AsyncClient with env-tunable timeouts, keep-alive pooling, optional transport IP pin, and TracingAsyncTransport plus http_trace_id for correlated HTTP stage logging. Managed UDF apps set uvicorn timeout_keep_alive=120 to reduce upstream proxy connect timeouts.

Bumps package version to 1.16.11rc1+byasaini and adds test_udf_event_loop.py covering dispatch, cancellation, and Application routing.

^{Reviewed by Cursor Bugbot for commit 018db7a. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 018db7a. Configure here.}

[cursor]

Cancel races successful UDF

High Severity

In _cancellable_run, when cancel_event is set at the same time the UDF coroutine finishes, asyncio.wait can place both the work task and _poll_cancel in done. The handler always treats a finished cancel poll as cancellation and raises CancelledError, even if the UDF task already completed successfully, so timeouts/disconnects can win over a valid result.

 

^{Reviewed by Cursor Bugbot for commit 018db7a. Configure here.}

[cursor]

Sync timeouts ignored async client

Medium Severity

SingleStoreEmbeddingsFactory parses connect_timeout and read_timeout from the optional sync http_client for Bedrock, but the new default httpx.AsyncClient for OpenAI embeddings only uses env defaults (60s read). Callers who configure timeouts via http_client get different limits on async embedding HTTP than they expect.

 

^{Reviewed by Cursor Bugbot for commit 018db7a. Configure here.}