Outline the next architectural arc: cloud, distributed, extensions#140
Merged
Conversation
Five companion proposals for where vx goes next, each self-contained and shippable independently, each multiplying the value of the others: - distributed-ci-2026-06: OSS distributed task execution (coordinator + workers over the existing service protocol). Free Nx-Cloud-DTE on any CI, self-hostable. Content-addressing makes work fungible. - vx-cloud-2026-06: hosted observability + cache + execution. Three faces (local insights, team analytics, hosted SaaS) over one event schema. Self-hostable; the OSS binary IS the hosted runtime. No proprietary-only features. - extension-protocol-2026-06: subscribers, inspectors, drivers, and in-process plugins on top of the WireEvent + devframe surface. Thin TS/Python SDKs, three reference plugins, MCP-for-agents. - predictive-execution-2026-06: turn the runs table into scheduling intelligence. History-aware critical-path priority, bandit retry, speculative pre-warming, regression detection at run-end. The only task runner that learns from itself. - architecture-north-star-2026-06: the synthesis. How the four compose, what ships in what order, the perf/DX/openness commitments, the competitive picture vs Turbo / Nx / Nx Cloud, and the carved-in-stone design rules (content addressing as identity, events as the protocol, fail-safe to local).
The five new docs tripped lint.oxfmt; pass them through oxfmt --fix and patch three places where oxfmt's list-detector mis-parsed a continuation line starting with '+ ' as a bullet (rewrote the sentences so they no longer start with a list-marker char).
Two changes:
1. vx-cloud-2026-06.md — pivot from PostgreSQL+S3+container-orchestrator
to a Cloudflare Workers project at apps/cloud/: Workers for HTTP, R2
for cache artifacts (zero egress!), D1 for orgs+runs, Durable Objects
for per-run coordinator + content-addressed in-flight dedup, Queues
for ingest buffering, KV for hot token lookups. Template-spawnable
from the cloned repo (3 commands to a private hosted vx in your own
CF account). Hyperdrive as the escape hatch when D1's 10GB cap is
tight. The OSS-first promise gets stronger: the SaaS at cloud.vx.dev
is one CF account deployment of the same template.
2. architecture-review-2026-06.md (new) — synthesis of two parallel
external-tool research passes + a cross-doc consistency review of
the five proposals. Surfaces:
- 2 structural simplifications (collapse in-process plugin = WS
subscriber to one Plugin contract; lift Digest = (hash, sizeBytes)
as the explicit CAS key type so storage backends are pluggable).
- 4 contract sharpening moves (fold distributed-ci worker:* into
protocol.ts; map extension-protocol channels to JSON-RPC 2.0
methods + notifications; pin run_events to projected WireEvent
form; formalize HistoryProvider as the local-vs-cloud loader).
- 3 bugs/contradictions (worker:pull lacks identity; vx-cloud SQL
was Postgres-flavored, now D1; predictive-execution Phase F
'default-on' downgraded to data-gated).
- 5 duplications across docs with canonical-home assignments.
- The third-party adoption table: ADOPT 3 (MCP SDK, Hono,
OTel CI/CD conventions); INSPIRE from 8 (Bazel CAS digest, Buck2
hybrid-exec racing, JSON-RPC 2.0 envelope, OTel LogRecord shape,
Vite plugin lifecycle, BuildBuddy product UX, Nix narinfo,
A2A); SKIP 12 (REAPI on the wire, NATS, Temporal/Restate,
Cap'n Proto, Postgres in core, WebTransport, WebContainers in
core, Effect-TS, Phoenix LiveView, Sentry/PostHog as backend,
Buildkite/GHA worker protocol); PLAN EXIT 1 (devframe — useful
first surface, not load-bearing; cut by 0.6 in favor of
Hono + raw WS).
- The wire-format consolidation: 7 framings → 2 (in-process RunEvent
for type fidelity, wire JSON-RPC 2.0 with OTel-LogRecord payload).
This single move unblocks MCP, A2A, OTel exporter, devframe cut,
and the three transport variants (WS + SSE + NDJSON) in one PR.
- Revised 5-wave plan reflecting CF pivot + wire-format unlock.
- 5 carved-in-stone rules (revised: content addressing as identity,
one envelope/many transports, event stream as the protocol,
fail-safe to local, shell as the task API).
10 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Five companion design proposals in
docs/design/outlining the next architectural arc for vx. Each is self-contained and individually shippable; together they multiply value via a single substrate (the WireEvent bus + content-addressed cache + RunBackend interface that already shipped).The premise: vx has built out the substrate (event bus, devframe surface, pluggable execution backends, distributed remote cache with HMAC). Now is the time to harvest — turn the substrate into user-visible surfaces that beat Turbo and Nx on openness, on distributed CI, on observability, and on learning-over-time performance.
The five proposals
distributed-ci-2026-06.md— OSS distributed task execution. Coordinator + workers reuse the existing service WS protocol; content addressing makes work fungible across runners. Free Nx-Cloud-DTE on any CI provider, self-hostable. GitHub Actions composite ships as the reference integration. Catalogs failure modes, capability labels, cache-affinity hints, and the trust model for "rent a worker."vx-cloud-2026-06.md— Hosted observability, cache, and execution. Three faces: local insights (the deleted dashboard revived on a substrate that can't freeze the run), team analytics (per-PR diff view, regression cliffs, cost rollups), hosted SaaS. The OSS binary IS the hosted runtime — no proprietary-only features. Data model: append-onlyruns+run_tasks+run_events(the full WireEvent log replayable into a historical timeline).extension-protocol-2026-06.md— Third-party tooling surface. Three roles: subscribers (read events), inspectors (read RPCs), drivers (submit runs). Thin TS/Python SDKs, in-processdefineWorkspace.plugins, MCP-for-agents. Three reference plugins ship in-tree (sentry,slack,influx) as smoke tests for the API.predictive-execution-2026-06.md— Turn therunstable into scheduling intelligence. History-aware critical-path priority (replaces graph-counter heuristic), speculative pre-warming (input WILLNEED, module preload), bandit-driven retry on flaky tasks, regression detection in the run footer. The only task runner that learns from itself.architecture-north-star-2026-06.md— Synthesis. The six-layer spine, the five carved-in-stone design rules (content addressing as identity, events as the protocol, fail-safe to local, shell as the task API, validate at boundaries), the wave-based execution sequence (Wave 2 = next 4 weeks), and the competitive picture vs Turbo / Nx / Nx Cloud.Why this composition matters
Each proposal multiplies the others:
distributed-ciproduces events thatvx-cloudpersists.vx-cloudpowers the HistoryTable thatpredictive-executionconsumes.extension-protocolexposes the live stream + the historical store to third-party tooling.predictive-executioncloses the learning loop — every run improves the next.All four reuse the event substrate already shipped (
event-stream-2026-06.mdPhase 1a/1b).Headline competitive claim
vx becomes the only OSS task runner with an end-to-end story for: local DX → team observability → hosted execution → CI distribution → cache → learning scheduler. Turbo: cache + (paid) analytics. Nx: cache + (paid) DTE + (paid) analytics. vx: all of the above, OSS, self-hostable, with optional hosted convenience.
Test plan
apps/docs/scripts/import-docs.tsalready auto-importsdesign/*.md)architecture-north-star-2026-06.md§ 5 to decide first implementation targetGenerated by Claude Code