feat(cloudflare): add Cloudflare Workflows support#304
Draft
userlaurin wants to merge 238 commits into
Draft
Conversation
… sebs/cloudflare folder for comprehensive cloudflare doc and next steps
… if all configs are set up manually
…ge is needed and the wrappers have to work
…, at this point the other benchmarks may be translated to nodejs
…and 000 to nodejs. fixed support for nosql in wrapper and sebs. fixed durable objects implementation for sebs
When running the handler as a thread and then awaiting said thread (asyncio.to_thread), we can run asyncio.run() in the subsequent call stack.
… in Azure workflow files
…ironment variables
…er files and package dependencies
…new download method
…d Parallel states
…esult and enhance async_invoke to parse execution result
- Enhance CloudflareContainersDeployment to support additional benchmarks. - Introduce CloudflareWorkflowGenerator for generating TypeScript code from FSM definitions. - Create a new wrangler-workflow.toml template for Cloudflare Workflows. - Implement WorkflowLibraryTrigger for invoking Cloudflare Workflows via HTTP. - Add CloudflareWorkflow class to represent deployed workflows with dispatcher and orchestrator. - Create CloudflareTestSequenceWorkflows for testing workflow benchmarks on Cloudflare.
…te support - Refactor CloudflareWorkflowGenerator to support concurrent execution of Map and Parallel states. - Introduce ItemWorkflow for processing Map chunks and FanInCoordinator for aggregating results. - Update dispatch logic to handle chunked processing and error reporting. - Emit proxy handlers for NoSQL and R2 operations in generated code. - Modify wrangler-workflow.toml template to include new ItemWorkflow and Durable Object bindings.
Add an Azure workflow HTTP trigger that exposes Durable Functions as SeBS library-style workflow invocations by starting the orchestration, polling the status URL, and returning the completed workflow output. Fix Azure workflow packaging/runtime behavior: - preserve benchmark-relative imports by importing activity modules as packages - add wrapper-level User-Agent handling for outbound requests - query the correct workflow HTTP entrypoint after publish - avoid duplicate HTTP trigger handling - quote Azure app settings and restart apps after env updates - support cached workflow trigger deserialization Fix workflow result handling for nested Map states inside Parallel branches so genome workflow outputs preserve the expected branch payload shape. Run high-CPU genome workflows on an EP3 Linux Premium plan with distinct app names, and limit genome activity concurrency in host.json to avoid shared /tmp collisions in benchmark code. Fix Azure blob range reads to use inclusive end-byte semantics, matching the benchmark expectations, and improve regression failure diagnostics so invocation exceptions are written to result JSON instead of being masked by harness errors.
…chmarks-data to the appropriate branch "workflows"
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implement Cloudflare Workflows deployment, generation, invocation, and
regression support for the workflow benchmark suite.
Add a TypeScript workflow generator that translates SeBS FSM definitions into
Cloudflare Workflows, including Map and Parallel fan-out via child workflows,
Durable Object fan-in coordination, dispatcher container calls, retry handling,
and R2-backed storage support.
Add Cloudflare workflow packaging and deployment logic, wrangler workflow
templates, workflow triggers, and workflow resource/config handling for
container-based execution. Extend container readiness handling with
max_instances-aware health checks and configurable chunk_size/max_instances
settings.
Update Cloudflare Python/Node wrappers and regression selection so supported
workflow benchmarks can run on Cloudflare containers.
Regression: Cloudflare Python 3.12 workflow suite passes 13/13.