implement event-sourced architecture #621
Draft
+4,728
−1,641
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🦋 Changeset detectedLatest commit: 4c5fd4c The changes in this PR will be included in the next version bump. This PR includes changesets to release 17 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
🧪 E2E Test Results❌ Some tests failed Summary
❌ Failed Tests▲ Vercel Production (103 failed)astro (8 failed):
example (8 failed):
express (8 failed):
fastify (8 failed):
hono (8 failed):
nextjs-turbopack (8 failed):
nextjs-webpack (8 failed):
nitro (8 failed):
nuxt (8 failed):
sveltekit (23 failed):
vite (8 failed):
🐘 Local Postgres (1 failed)fastify-stable (1 failed):
🌍 Community Worlds (108 failed)mongodb-dev (3 failed):
mongodb (27 failed):
redis (24 failed):
starter (24 failed):
turso-dev (3 failed):
turso (27 failed):
Details by Category❌ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
❌ 🐘 Local Postgres
✅ 🪟 Windows
❌ 🌍 Community Worlds
❌ Some E2E test jobs failed:
Check the workflow run for details. |
This was referenced Dec 16, 2025
Collaborator
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
pranaygp
force-pushed
the
pranaygp/perf-phase-3b-atomic-events
branch
from
6ebd4c5 to
2e46b8a
Compare
pranaygp
force-pushed
the
pranaygp/12-04-perf_parallelize_suspension_handler_for_high-concurrency
branch
from
eece359 to
290e879
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a performance optimization for event creation by adding a createBatch() method to the World interface. The implementation enables atomic batch creation of multiple events, significantly improving the wait completion logic in the runtime from O(n²) to O(n) complexity.
Key Changes
- Added
events.createBatch()method to the World interface for creating multiple events in a single operation - Implemented batch creation across three storage backends (world-vercel, world-postgres, world-local) with backend-specific optimizations
- Optimized runtime wait completion logic using Set-based correlation ID lookup and batch event creation
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
packages/world/src/interfaces.ts |
Added createBatch() method signature with JSDoc documentation to the Storage events interface |
packages/world-vercel/src/storage.ts |
Integrated batch event creation into the storage adapter |
packages/world-vercel/src/events.ts |
Implemented createWorkflowRunEventBatch() using parallel API calls via Promise.all |
packages/world-postgres/src/storage.ts |
Implemented batch creation using a single INSERT query with multiple values for optimal database performance |
packages/world-local/src/storage.ts |
Implemented sequential batch creation to maintain monotonic ULID ordering for filesystem storage |
packages/core/src/runtime.ts |
Refactored wait completion to use Set-based lookup and batch event creation, improving from O(n²) to O(n) complexity |
.changeset/brave-dots-bake.md |
Added changeset documenting the performance improvement across all affected packages |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Technical accuracy improvements: - Clarify that hook states are conceptual (hooks are deleted, not transitioned) - Add note about step cancelled status being reserved for future use - Clarify that waits are conceptual entities not materialized in storage - Fix run_created event description (deployment ID, execution context) - Remove atomicity mention from intro (no longer accurate) - Add hook token reservation explanation and link to hooks docs Cross-linking improvements: - Link "event log" mentions to event sourcing page from: - workflows-and-steps.mdx (3 locations) - understanding-directives.mdx - code-transform.mdx (2 locations) - streaming.mdx (2 locations) - worlds/index.mdx - observability/index.mdx - Add outbound link to errors-and-retries from step retrying section 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Hooks are the general primitive for pausing workflows and receiving external data. Webhooks are a specific HTTP-based abstraction built on top of hooks. Updated documentation to reflect this distinction. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Documents the consistent ID format used across all entities: - 4-character prefix + underscore + ULID - Prefixes: wrun_, step_, hook_, wait_, evnt_, strm_ - Explains benefits: easy introspection and chronological ordering 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pranay:
corresponding workflow-server PR: https://github.com/vercel/workflow-server/pull/154
important: This is a big change to the way workflows work since everything is now event sourced, I introduced new events types, and changed the shape of the step object (
lastKnownError->errorandstartedAt->firstStartedAt). New event logs that use this published version ofworkflowwill be incompatible with previous workflow version event logs. This doesn't affect the runtime of workflows since those are deployment pegged - but this does affect observability since the event shape looks different and the world spec has changed. The web-shared package just needs to be compatible with viewing workflow runs of the old schema for this to work correctly (which I believe it does, but please double check @VaguelySerious if I missed anything).The currently failing e2e tests on vercel world are related to the CLI I believe (slack x-ref). However once we merged the workflow-server PR, we can drop the env var changes on the vercel deployments for PR so that this PR points to the main prod deployment, again and then I'll re-run e2e tests to make sure they work :)
I Also added a new docs page with diagrams to explain the event sourcing and state machine lifecycles (preview link):
small: I also removed the unused run paused/resumed stuff which we've never used to simplify
Summary
Implement event-sourced architecture for runs, steps, and hooks:
run_created,run_started,run_completed,run_failed,run_cancelled)step_retryingevent for non-fatal step failures that will be retriedfatalfield fromstep_failedevent (step_failed now implies terminal failure)lastKnownErrortoerrorfor consistency with serverevents.create()step_createdevent for earlier detectionrun_paused/run_resumedevents andpausedstatusThis makes the system faster, easier to reason about, and resilient to data inconsistencies.
Test plan
🤖 Generated with Claude Code