Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions apps/docs/astro.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,19 @@ export default defineConfig({
{ label: 'Workspace configuration', link: '/guides/workspace-config/' },
],
},
{
label: 'Platform & extensions',
items: [
{ label: 'vx mcp — AI agents', link: '/guides/mcp/' },
{ label: 'Distributed CI execution', link: '/guides/distributed-ci/' },
{ label: 'Writing a vx plugin', link: '/guides/plugins/' },
{ label: 'Predictive scheduling', link: '/guides/predictive-scheduling/' },
{ label: 'vx insights — local dashboard', link: '/guides/insights/' },
{ label: 'vx Cloud (Cloudflare deploy)', link: '/guides/vx-cloud/' },
{ label: 'OpenTelemetry CI/CD spans', link: '/guides/otel-bridge/' },
{ label: 'vx serve wire protocol', link: '/guides/wire-protocol/' },
],
},
{
label: 'Migrate to vx',
items: [
Expand Down
147 changes: 147 additions & 0 deletions apps/docs/src/content/docs/guides/distributed-ci.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
---
title: Distributed CI execution
description: Run your task graph across multiple machines. vx coordinator dispatches assignments to vx workers over WebSocket. Content-addressed; workers are fungible. OSS, self-hostable, no daemon.
---

vx ships an OSS distributed task execution layer: one coordinator
holds the graph, many workers pull and execute. Tasks are
content-addressed by their v22 cache hash, so any worker producing
artifact `<hash>` satisfies every consumer of `<hash>` — workers
are fungible.

This is the Nx-Cloud-DTE equivalent, OSS and self-hostable.
Phase A-B today; capability labels, cache-affinity, and a hosted
deployment are deferred — see
`docs/design/distributed-ci-2026-06.md` for the full roadmap.

## The two roles

- **Coordinator** (`vx coordinator <tasks…>`) — one per CI build.
Holds the global ready queue, dispatches to workers, exits when
every task ends.
- **Worker** (`vx run --worker <coord-url>`) — N per build.
Stateless and fungible. Pulls assignments, executes via
`runCommand`, reports outcomes, repeats.

## Quick start (two terminals)

```sh
# Terminal 1: start the coordinator
vx coordinator lint test build --port 5180 --workers 2

# Terminal 2: attach a worker
vx run --worker ws://127.0.0.1:5180 --capacity 4
```

The coordinator dispatches every ready task to the worker; the
worker executes them in parallel up to `--capacity`. When every
task terminates, the coordinator sends `coord:drain`, the worker
exits, and the coordinator exits with 0 (or 1 if any task failed).

## GitHub Actions

The canonical pattern: one matrix index hosts the coordinator,
the rest attach as workers.

```yaml
jobs:
build:
strategy:
matrix:
worker: [0, 1, 2, 3] # 4-way parallelism
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: curl -fsSL https://raw.githubusercontent.com/vznjs/vx/main/install.sh | sh

# Worker 0 hosts the coordinator; others wait for it and attach.
- if: matrix.worker == 0
run: vx coordinator lint test build --port 5180 --workers 4

- if: matrix.worker != 0
run: |
until nc -z runner-0 5180; do sleep 1; done
vx run --worker ws://runner-0:5180 --capacity 2
```

The cross-runner networking (`runner-0` resolves to the matrix
index 0 runner) needs either a tunnel (Tailscale free tier
works), self-hosted runners on the same LAN, or one of the
GHA-specific runner-link patterns. A composite action
(`vx/distributed-action@v1`) packaging this is on the roadmap.

## How dispatch works

```
┌────────────────────────────────────────┐
│ vx coordinator │
│ • prepareRun → workspace + task graph │
│ • per-node v22 hash (assignment key) │
│ • ready queue, in-flight per worker │
│ • WS server │
└─────┬──────────────────────────┬───────┘
│ │
▼ task:assign ▼ task:assign
┌────────────┐ ┌────────────┐
│ worker N │ │ worker M │
│ pulls │ │ pulls │
│ spawns │ │ spawns │
│ reports │ │ reports │
└────────────┘ └────────────┘
```

1. Coordinator runs the same `prepareRun` pipeline `vx run` uses
locally — it builds the same graph, with the same v22 cache
hashes per node.
2. Workers register via `worker:hello { workerId, capacity, labels }`.
3. Coordinator dispatches via `task:assign { hash, node }` up to
each worker's capacity.
4. Workers spawn `runCommand`, stream stdout/stderr back over
`worker:stdout` / `worker:stderr` messages.
5. Workers report `worker:done { taskHash, outcome }` on completion.
6. Downstream tasks become ready as their upstream finishes.

The wire format is JSON-RPC 2.0 — same envelope `vx serve` speaks.
Full spec: `docs/design/wire-protocol-2026-06.md`.

## Disconnect recovery

If a worker disconnects mid-task, the coordinator detects the WS
close, pulls every in-flight assignment off that worker, and
puts the hashes back on the ready queue. The next attached worker
picks them up.

## Performance characteristics

| Workload | Local single-host | Distributed (4 workers) | Notes |
| --- | --- | --- | --- |
| Cold run, deep graph | 1× | 0.25–0.4× wall time | Bounded by graph's critical path |
| Warm full-cache | ≤ 200 ms | similar | Cache-hit shortcircuits — coordinator dispatch overhead dominates |
| Mixed cache state | 1× | 0.4–0.7× | Worker mix-and-match wins on long tasks |

These numbers will improve once workers probe the remote cache
before executing (next iteration).

## Known limits today (Phase A-B)

- **Workers don't probe the remote cache yet.** Every assigned task
spawns fresh. Set up `VX_REMOTE_CACHE_URL` for fastest results
via the local prefetch path.
- **No capability labels filter.** Workers report labels; the
coordinator doesn't filter `task:assign` by them.
- **No critical-path priority on the coordinator.** The ready
queue is FIFO. The local scheduler has predictive priorities
(`predictive: true`); the coordinator doesn't read them.
- **Submitter logs aren't aggregated.** Worker stdout reaches the
coordinator but no submitter-side `vx run --coordinator` is wired
yet to fan it back to a user.
- **No TLS.** Hardcoded `ws://`. For cross-host, terminate TLS at a
reverse proxy or use Tailscale/cloudflared.

The protocol extension (`worker:*` + `task:assign` + `coord:drain`)
lives in `src/orchestrator/protocol.ts`; the JSON-RPC 2.0 envelope
adapters live in `src/orchestrator/wire.ts`. Both are stable; the
gaps above are wiring follow-ups.

See also: `docs/design/distributed-ci-2026-06.md` (full design),
`docs/progress/implementation-log-2026-06.md` (Step 4 narrative).
119 changes: 119 additions & 0 deletions apps/docs/src/content/docs/guides/insights.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
---
title: vx insights — local run dashboard
description: Boot a Solid + DuckDB-WASM SPA against your workspace's cache.db. Historical run flamegraphs, per-task trends, no backend, no daemon.
---

`vx insights serve` opens a localhost dashboard backed by your
workspace's `cache.db`. Pure read-only analytics — no backend, no
upload, no daemon. The page reads SQLite in the browser via DuckDB-WASM.

## Quick start

```sh
cd your-workspace
vx insights serve
```

That prints two URLs:

- The SPA on `http://127.0.0.1:5290` (Vite dev server).
- A tiny static HTTP server (kernel-assigned port) exposing
`cache.db` at `/cache.db` with the SQLite MIME so the SPA can
fetch it.

`Ctrl-C` stops both.

Override the SPA port with `--port`:

```sh
vx insights serve --port 7000
```

## What you see

Two pages:

- **Overview** — recent runs list, sorted by start time descending.
Each row shows project, task name, status, duration, cache
source. Click a row → run detail.
- **Run detail** — per-task timeline (flamegraph), one lane per
project, bars colored by status / cache source. The same data
drives both — replayable in the browser.

## How it works

```
Browser
┌─────────────────────────────┐
│ apps/insights SPA (Solid) │
│ • UnoCSS dark theme │
│ • Solid Router (hash) │
│ • DuckDB-WASM (~30 MB lazy)│
└────────┬────────────────────┘
│ fetch /cache.db
┌─────────────────────────────┐
│ Tiny static server (Bun) │
│ • cache.db @ vnd.sqlite3 │
│ • CORS * │
│ • /health │
└────────┬────────────────────┘
│ reads
┌─────────────────────────────┐
│ <workspaceRoot>/.vx/cache/ │
│ cache.db │
└─────────────────────────────┘
```

DuckDB-WASM reads SQLite files directly via the `sqlite_scanner`
extension — no ETL, no conversion. The SPA `ATTACH`-es the fetched
bytes as a database, runs aggregations client-side, renders Solid
components. Queries stay in the browser.

## What's needed on disk

- `<workspaceRoot>/.vx/cache/cache.db` — at least one `vx run`
in the workspace.
- `<vx checkout>/apps/insights/` — the SPA source. Set
`VX_INSIGHTS_DIR` if the installed binary can't find a checkout
alongside its `import.meta.dir`.

If `cache.db` is missing, `vx insights` prints a clean hint and
exits 1. If the SPA scaffold is missing, the binary points you at
`VX_INSIGHTS_DIR`.

## Why client-side analytics

- **Zero backend.** Nothing to provision, nothing to operate. The
static server just serves bytes.
- **Privacy by default.** Data never leaves your laptop.
- **Read-only by construction.** The SPA fetches `cache.db` once
per page load. Mutating queries can't touch your real cache.
- **Open at the data layer.** Anyone can write a DuckDB query
against `cache.db` directly — the SPA is just a UI on top.

For team-wide analytics, see the
[vx Cloud guide](/vx/guides/vx-cloud/).

## Known limits

- **DuckDB-WASM is heavy** (~30 MB). First query is slow because
the WASM bundle and SQLite extension download. Subsequent queries
are fast.
- **No real-time.** The page snapshots `cache.db` on load. Reload
to see new runs.
- **Charts are minimal today.** The Overview and Run detail pages
ship; cache hit-rate trends, per-author breakdowns, and the
"Bottleneck atlas" from the cloud spec are scaffold-pending.

## What's coming

- More pages (per-task trends, cache cliff detection, regression
surfacing).
- Auto-refresh when `cache.db` mtime changes.
- An option to embed the SPA inside `vx serve` for an in-browser
live view of running runs.

See also: `docs/design/vx-cloud-2026-06.md` §2.1 (local face),
`apps/insights/README.md`.
93 changes: 93 additions & 0 deletions apps/docs/src/content/docs/guides/mcp.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
---
title: vx mcp — Model Context Protocol server
description: Expose vx as a typed tool surface to AI coding agents (Claude Code, Cursor, Continue.dev, GitHub Copilot). Read cache stats, run history, and explain rebuild causes through the standard MCP protocol.
---

`vx mcp` boots a Model Context Protocol (MCP) server over stdio so
AI coding agents can query your repo's build state through the
standard agent-tool protocol. No HTTP, no auth — stdio is process-
private.

## What is MCP

The Model Context Protocol is the de-facto standard for AI agents to
discover and call typed tools. Claude Code, Cursor, Continue.dev,
VS Code GitHub Copilot, and many others all speak it. By shipping an
MCP server, vx gives any of them a typed surface for your build
state.

Spec: <https://modelcontextprotocol.io/>

## Quick start

```sh
vx mcp # stdio transport (default)
```

Add to your agent's MCP config (Claude Code example):

```jsonc
// ~/.claude/mcp.json
{
"mcpServers": {
"vx": { "command": "vx", "args": ["mcp"] }
}
}
```

Cursor reads `.cursorrules`-adjacent config; Continue.dev reads
`~/.continue/config.json`. The shape is identical: `command + args`.

## Tools exposed

| Tool | What it answers |
| --- | --- |
| `getCacheStats` | "What's the state of my cache right now?" — entries, total size, runs/hits last 24h, hit rate |
| `getRunHistory` | "Which tasks have I been running, and how fast?" — distinct (project, task) pairs with p50/p99/successRate/hitRate aggregates |
| `explainCacheKey` | "What's the cache identity for `pkg#build`?" — latest entries-row (hash, command, exit code, duration, size, created_at) |
| `whyDidThisRerun` | "Why did this task re-execute instead of using the cache?" — compares the run's cache hash against the previous run for the same task |

All four read your workspace's local `cache.db` on demand. Open
your agent and ask things like:

- "What's my cache hit rate this week?"
- "Why did `pkg-a#test` re-run in the last build?"
- "Which tasks miss the cache most often?"
- "What was the slowest task in the last 50 runs?"

## How it works

The server is the same `@modelcontextprotocol/sdk` package every MCP
implementation uses. `vx mcp` opens cache.db, exposes the four tools
via `setRequestHandler(ListToolsRequestSchema, …)` /
`setRequestHandler(CallToolRequestSchema, …)`, and pipes JSON-RPC
2.0 over stdin/stdout. The agent reads tool results as text content
(stringified JSON).

vx's MCP tools share dispatch with the inspector RPC channel
(`vx:rpc` from `docs/design/wire-protocol-2026-06.md`). When the
WebSocket-side inspector ships, every MCP tool will work over WS
too — one handler, two transports.

## What's coming

- `runTasks` — agents trigger a `vx run` directly (driver surface).
- `getRunState` — live state of an in-flight run (works with
`vx serve --ui`).
- MCP resources for `vx://runs/{runId}` and `vx://history` (browseable).
- HTTP transport (Streamable) so a single `vx mcp` instance can
serve multiple agent clients.

## Troubleshooting

- **Agent says "no MCP tools" after adding the config.** Restart
the agent. Most MCP clients only re-read config on launch.
- **`vx mcp: requires @modelcontextprotocol/sdk`** — the binary
was built without the SDK. Rebuild with `bun install &&
bun src/bin.ts run build`.
- **Empty results from `getCacheStats`** — you haven't run any
`vx run` yet, or you're pointing at the wrong workspace. The
server discovers the workspace via `findWorkspaceRoot(cwd)`; run
the agent from the workspace root.

See also: `docs/design/extension-protocol-2026-06.md`.
Loading
Loading