implement event-sourced architecture

.changeset/event-sourced-entities.md

@@ -0,0 +1,19 @@
+---
+"@workflow/core": patch
+"@workflow/world": patch
+"@workflow/world-local": patch
+"@workflow/world-postgres": patch
+"@workflow/world-vercel": patch
+"@workflow/web": patch
+"@workflow/web-shared": patch
+---
+
+perf: implement event-sourced architecture for runs, steps, and hooks
+
+- Add run lifecycle events (run_created, run_started, run_completed, run_failed, run_cancelled)
+- Add step_retrying event for non-fatal step failures that will be retried
+- Remove `fatal` field from step_failed event (step_failed now implies terminal failure)
+- Rename step's `lastKnownError` to `error` for consistency with server
+- Update world implementations to create/update entities from events via events.create()
+- Entities (runs, steps, hooks) are now materializations of the event log
+- This makes the system faster, easier to reason about, and resilient to data inconsistencies

.changeset/remove-paused-resumed.md

@@ -0,0 +1,15 @@
+---
+"@workflow/world": patch
+"@workflow/world-local": patch
+"@workflow/world-vercel": patch
+"@workflow/cli": patch
+"@workflow/web": patch
+"@workflow/web-shared": patch
+---
+
+**BREAKING CHANGE**: Remove unused paused/resumed run events and states
+
+- Remove `run_paused` and `run_resumed` event types
+- Remove `paused` status from `WorkflowRunStatus`
+- Remove `PauseWorkflowRunParams` and `ResumeWorkflowRunParams` types
+- Remove `pauseWorkflowRun` and `resumeWorkflowRun` functions from world-vercel

docs/README.md

@@ -1,3 +1,66 @@
 # Workflow DevKit Docs
 
 Check out the docs [here](https://useworkflow.dev/)
+
+## Mermaid Diagram Style Guide
+
+When adding diagrams to documentation, follow these conventions for consistency.
+
+### Diagram Type
+
+Use `flowchart TD` (top-down) or `flowchart LR` (left-right) for flow diagrams:
+
+```mermaid
+flowchart TD
+    A["Source Code"] --> B["Transform"]
+    B --> C["Output"]
+```
+
+### Node Syntax
+
+Use square brackets with double quotes for rectangular nodes:
+
+```
+A["Label Text"]           # Correct - rectangular node
+A[Label Text]             # Avoid - can cause parsing issues
+A(Label Text)             # Avoid - rounded node, inconsistent style
+```
+
+### Edge Labels
+
+Use the pipe syntax with double quotes for edge labels:
+
+```
+A -->|"label"| B          # Correct
+A --> B                   # Correct (no label)
+```
+
+### Highlighting Important Nodes
+
+Use the purple color scheme to highlight terminal states or key components:
+
+```
+style NodeId fill:#a78bfa,stroke:#8b5cf6,color:#000
+```
+
+Place all `style` declarations at the end of the diagram.
+
+### Complete Example
+
+```mermaid
+flowchart TD
+    A["(start)"] --> B["pending"]
+    B -->|"started"| C["running"]
+    C -->|"completed"| D["completed"]
+    C -->|"failed"| E["failed"]
+
+    style D fill:#a78bfa,stroke:#8b5cf6,color:#000
+    style E fill:#a78bfa,stroke:#8b5cf6,color:#000
+```
+
+### Guidelines
+
+- Keep diagrams simple and readable
+- Use meaningful node labels
+- Limit complexity - split into multiple diagrams if needed
+- Add a legend or callout explaining highlighted nodes when appropriate

docs/content/docs/foundations/streaming.mdx

@@ -83,7 +83,7 @@ This allows clients to reconnect and continue receiving data from where they lef
 
 [`ReadableStream`](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream) and [`WritableStream`](https://developer.mozilla.org/en-US/docs/Web/API/WritableStream) are standard Web Streams API types that Workflow DevKit makes serializable. These are not custom types - they follow the web standard - but Workflow DevKit adds the ability to pass them between functions while maintaining their streaming capabilities.
 
-Unlike regular values that are fully serialized to the event log, streams maintain their streaming capabilities when passed between functions.
+Unlike regular values that are fully serialized to the [event log](/docs/how-it-works/event-sourcing), streams maintain their streaming capabilities when passed between functions.
 
 **Key properties:**
 - Stream references can be passed between workflow and step functions
@@ -151,7 +151,7 @@ async function processInputStream(input: ReadableStream<Uint8Array>) {
 You cannot read from or write to streams directly within a workflow function. All stream operations must happen in step functions.
 </Callout>
 
-Workflow functions must be deterministic to support replay. Since streams bypass the event log for performance, reading stream data in a workflow would break determinism - each replay could see different data. By requiring all stream operations to happen in steps, the framework ensures consistent behavior.
+Workflow functions must be deterministic to support replay. Since streams bypass the [event log](/docs/how-it-works/event-sourcing) for performance, reading stream data in a workflow would break determinism - each replay could see different data. By requiring all stream operations to happen in steps, the framework ensures consistent behavior.
 
 For more on determinism and replay, see [Workflows and Steps](/docs/foundations/workflows-and-steps).
 

docs/content/docs/foundations/workflows-and-steps.mdx

@@ -36,10 +36,10 @@ export async function processOrderWorkflow(orderId: string) {
 **Key Characteristics:**
 
 - Runs in a sandboxed environment without full Node.js access
-- All step results are persisted to the event log
+- All step results are persisted to the [event log](/docs/how-it-works/event-sourcing)
 - Must be **deterministic** to allow resuming after failures
 
-Determinism in the workflow is required to resume the workflow from a suspension. Essentially, the workflow code gets re-run multiple times during its lifecycle, each time using an event log to resume the workflow to the correct spot.
+Determinism in the workflow is required to resume the workflow from a suspension. Essentially, the workflow code gets re-run multiple times during its lifecycle, each time using the [event log](/docs/how-it-works/event-sourcing) to resume the workflow to the correct spot.
 
 The sandboxed environment that workflows run in already ensures determinism. For instance, `Math.random` and `Date` constructors are fixed in workflow runs, so you are safe to use them, and the framework ensures that the values don't change across replays.
 
@@ -111,7 +111,7 @@ Keep in mind that calling a step function outside of a workflow function will no
 
 ### Suspension and Resumption
 
-Workflow functions have the ability to automatically suspend while they wait on asynchronous work. While suspended, the workflow's state is stored via the event log and no compute resources are used until the workflow resumes execution.
+Workflow functions have the ability to automatically suspend while they wait on asynchronous work. While suspended, the workflow's state is stored via the [event log](/docs/how-it-works/event-sourcing) and no compute resources are used until the workflow resumes execution.
 
 There are multiple ways a workflow can suspend:
 

docs/content/docs/how-it-works/code-transform.mdx

@@ -140,7 +140,7 @@ handleUserSignup.workflowId = "workflow//workflows/user.js//handleUserSignup"; /
 - The workflow function gets a `workflowId` property for runtime identification
 - The `"use workflow"` directive is removed
 
-**Why this transformation?** When a workflow executes, it needs to replay past steps from the event log rather than re-executing them. The `WORKFLOW_USE_STEP` symbol is a special runtime hook that:
+**Why this transformation?** When a workflow executes, it needs to replay past steps from the [event log](/docs/how-it-works/event-sourcing) rather than re-executing them. The `WORKFLOW_USE_STEP` symbol is a special runtime hook that:
 
 1. Checks if the step has already been executed (in the event log)
 2. If yes: Returns the cached result
@@ -283,7 +283,7 @@ Because workflow functions are deterministic and have no side effects, they can
 
 - Can make API calls, database queries, etc.
 - Have full access to Node.js runtime and APIs
-- Results are cached in the event log after first execution
+- Results are cached in the [event log](/docs/how-it-works/event-sourcing) after first execution
 
 Learn more about [Workflows and Steps](/docs/foundations/workflows-and-steps).