Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/drop-quickjs-process-host.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@execbox/quickjs": minor
---

Remove `QuickJsExecutor({ host: "process" })` and its process-hosted runner entrypoint. Use `host: "worker"` for local hosted QuickJS execution, or `@execbox/remote` for app-owned process, container, or VM boundaries.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,14 @@ Portable code execution for [Model Context Protocol](https://modelcontextprotoco

</div>

Execbox turns host tool catalogs into callable guest namespaces, supports MCP wrapping on both sides of the boundary, and lets you place guest JavaScript where it fits your deployment: inline, behind a worker or child-process host, or across your own remote transport.
Execbox turns host tool catalogs into callable guest namespaces, supports MCP wrapping on both sides of the boundary, and lets you place guest JavaScript where it fits your deployment: inline, behind a worker host, or across your own remote transport.

## Package Map

| Package | npm | What it is for |
| ----------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
| [`@execbox/core`](./packages/core/) | [![npm](https://img.shields.io/npm/v/%40execbox%2Fcore?style=flat-square)](https://www.npmjs.com/package/@execbox/core) | Core execution contract, provider resolution, MCP adapters, and runtime/protocol subpaths |
| [`@execbox/quickjs`](./packages/quickjs/) | [![npm](https://img.shields.io/npm/v/%40execbox%2Fquickjs?style=flat-square)](https://www.npmjs.com/package/@execbox/quickjs) | QuickJS executor for inline, worker, and process hosts |
| [`@execbox/quickjs`](./packages/quickjs/) | [![npm](https://img.shields.io/npm/v/%40execbox%2Fquickjs?style=flat-square)](https://www.npmjs.com/package/@execbox/quickjs) | QuickJS executor for inline and worker hosts |
| [`@execbox/remote`](./packages/remote/) | [![npm](https://img.shields.io/npm/v/%40execbox%2Fremote?style=flat-square)](https://www.npmjs.com/package/@execbox/remote) | Advanced transport-backed executor for app-owned runtime boundaries |

## Examples
Expand Down
50 changes: 24 additions & 26 deletions benchmarks/benchmark.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
* Comprehensive execbox performance benchmark suite.
*
* Covers:
* 1. Single-execution latency across all in-process & out-of-process executors
* 1. Single-execution latency across inline and worker-hosted executors
* 2. Cold-start vs warm (pooled/prewarmed) first-execution cost
* 3. Tool-call overhead scaling (0, 1, 5, 10 tool calls)
* 4. Schema validation overhead (with vs without input/output schemas)
Expand Down Expand Up @@ -349,7 +349,7 @@ async function suiteToolCalls(): Promise<void> {
).join(";\n");
}

// Use quickjs (fastest) and worker-pooled (representative out-of-process)
// Use quickjs (fastest) and worker-pooled (representative hosted local mode)
const testFactories = factories.filter(
(f) => f.name === "quickjs (in-process)" || f.name === "worker (pooled)",
);
Expand Down Expand Up @@ -507,31 +507,29 @@ async function suiteContention(): Promise<void> {
const poolSizes = [1, 2, 4];
const concurrency = 8;

for (const executor_type of ["worker", "process"] as const) {
console.log(`\n [${executor_type}]`);
console.log("\n [worker]");

for (const poolSize of poolSizes) {
const executor = createContentionExecutor(executor_type, poolSize);
try {
await executor.prewarm?.(poolSize);
const { totalMs, perRequestMs } = await benchConcurrent(
executor,
code,
[simpleProvider],
concurrency,
totalRuns,
);
const throughput = (totalRuns / totalMs) * 1000;
console.log(
` pool_size=${String(poolSize).padEnd(2)} concurrency=${concurrency} ` +
`throughput=${throughput.toFixed(1)} exec/s ` +
`median=${fmt(median(perRequestMs))} ` +
`p95=${fmt(percentile(perRequestMs, 95))} ` +
`max=${fmt(max(perRequestMs))}`,
);
} finally {
await executor.dispose?.();
}
for (const poolSize of poolSizes) {
const executor = createContentionExecutor(poolSize);
try {
await executor.prewarm?.(poolSize);
const { totalMs, perRequestMs } = await benchConcurrent(
executor,
code,
[simpleProvider],
concurrency,
totalRuns,
);
const throughput = (totalRuns / totalMs) * 1000;
console.log(
` pool_size=${String(poolSize).padEnd(2)} concurrency=${concurrency} ` +
`throughput=${throughput.toFixed(1)} exec/s ` +
`median=${fmt(median(perRequestMs))} ` +
`p95=${fmt(percentile(perRequestMs, 95))} ` +
`max=${fmt(max(perRequestMs))}`,
);
} finally {
await executor.dispose?.();
}
}
}
Expand Down
27 changes: 2 additions & 25 deletions benchmarks/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -70,24 +70,6 @@ export function createBenchmarkFactories(): ExecutorFactory[] {
}) as DisposableExecutor,
supportsExplicitPrewarm: true,
},
{
name: "process (ephemeral)",
create: () =>
new QuickJsExecutor({
host: "process",
mode: "ephemeral",
}) as DisposableExecutor,
supportsExplicitPrewarm: false,
},
{
name: "process (pooled)",
create: () =>
new QuickJsExecutor({
host: "process",
...createPooledBenchmarkOptions(),
}) as DisposableExecutor,
supportsExplicitPrewarm: true,
},
];
}

Expand All @@ -98,12 +80,7 @@ export function createMemoryBenchmarkFactories(
return factories.filter((factory) => allowedNames.has(factory.name));
}

export function createContentionExecutor(
executorType: "worker" | "process",
poolSize: number,
): DisposableExecutor {
export function createContentionExecutor(poolSize: number): DisposableExecutor {
const pool = createContentionPoolOptions(poolSize);
return executorType === "worker"
? (new QuickJsExecutor({ host: "worker", pool }) as DisposableExecutor)
: (new QuickJsExecutor({ host: "process", pool }) as DisposableExecutor);
return new QuickJsExecutor({ host: "worker", pool }) as DisposableExecutor;
}
26 changes: 6 additions & 20 deletions benchmarks/results.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,11 @@ This file records the current local benchmark run. Treat the tables as measured
| QuickJS (in-process) | **1.47ms** | **2.71ms** | **3.94ms** |
| Worker (ephemeral) | 62.40ms | 64.16ms | 65.67ms |
| Worker (pooled) | 1.53ms | 2.81ms | 4.10ms |
| Process (ephemeral) | 202.37ms | 204.10ms | 207.70ms |
| Process (pooled) | 1.61ms | 2.92ms | 4.18ms |

### Notes

- On this machine, warmed pooled executors stayed close to QuickJS for trivial scripts.
- Ephemeral executors remained far slower than pooled executors because each execution still pays worker or process startup cost.
- Process pooled stayed low-latency in median terms, but its process startup path remained much more expensive than worker startup.
- Ephemeral executors remained far slower than pooled executors because each execution still pays worker startup cost.

---

Expand All @@ -36,12 +33,10 @@ Only pooled executors expose explicit `prewarm()`. QuickJS and ephemeral executo
| QuickJS (in-process) | 1.46ms | N/A | N/A |
| Worker (ephemeral) | 62.70ms | N/A | N/A |
| Worker (pooled) | 62.30ms | 1.81ms | 97.1% |
| Process (ephemeral) | 202.51ms | N/A | N/A |
| Process (pooled) | 202.31ms | 2.32ms | 98.9% |

### Notes

- True `prewarm()` delivered the intended first-request behavior in this run: pooled worker and pooled process executors dropped from shell-plus-guest startup latency to low-single-digit warm execution latency.
- True `prewarm()` delivered the intended first-request behavior in this run: pooled worker executors dropped from shell-plus-guest startup latency to low-single-digit warm execution latency.
- `prewarm()` pays the host-shell and guest-startup path before live traffic arrives.

---
Expand Down Expand Up @@ -92,13 +87,11 @@ The pooled benchmark factories in this suite use a fixed `pool.maxSize: 2`.
| -------------------- | --------------- | ------ | ------ | ---------- |
| QuickJS (in-process) | **395.3** | 684.6 | 1078.8 | **1635.1** |
| Worker (pooled) | 356.5 | 722.9 | 706.4 | 749.8 |
| Process (pooled) | 352.3 | 653.2 | 634.5 | 675.1 |

### Notes

- QuickJS stayed the strongest path for trusted, in-process workloads at low and high concurrency in this run, while worker pooled edged ahead at concurrency 2.
- Worker and process pooled executors paid visible queueing once demand moved past the benchmark pool size.
- Process pooled stayed competitive, but it still trailed worker pooled at every tested concurrency level.
- Worker pooled executors paid visible queueing once demand moved past the benchmark pool size.

---

Expand All @@ -109,21 +102,16 @@ The pooled benchmark factories in this suite use a fixed `pool.maxSize: 2`.
| Worker | 1 | 362.6 | 22.18ms | 22.51ms | 22.53ms |
| Worker | 2 | 704.2 | 10.97ms | 11.64ms | 12.39ms |
| Worker | 4 | **1327.2** | 5.60ms | 6.95ms | 6.99ms |
| Process | 1 | 351.0 | 22.54ms | 23.66ms | 23.97ms |
| Process | 2 | 693.3 | 11.02ms | 12.95ms | 12.99ms |
| Process | 4 | 1111.3 | 6.50ms | 8.73ms | 8.75ms |

### Notes

- Increasing pool size improved both worker and process executors in this run.
- Worker pooled still kept the better mix of throughput and tail latency at every pool size tested.
- Process pooled also scaled up well here, but its tails remained wider than worker pooled at the same pool size.
- Increasing pool size improved worker throughput and tail latency in this run.

---

## 7. Host-Process Memory Delta

This suite only measures the parent Node process. It does not attempt to attribute child-process RSS back to `QuickJsExecutor({ host: "process" })`.
This suite only measures the parent Node process.

| Executor | Heap Delta | RSS Delta | External Delta |
| -------------------- | ---------- | --------- | -------------- |
Expand All @@ -144,13 +132,11 @@ This suite only measures the parent Node process. It does not attempt to attribu
### High-value takeaways from this snapshot

- QuickJS remained the lowest-latency option for trusted, in-process workloads on this machine.
- True `prewarm()` delivered the intended first-request benefit for pooled worker and pooled process executors.
- True `prewarm()` delivered the intended first-request benefit for pooled worker executors.
- Worker pooled remained the strongest general-purpose local trade-off between isolation, throughput, and tail latency.
- Process pooled stayed viable when process isolation matters, but it still trailed worker pooled on throughput and tail latency.
- Ephemeral modes remained dramatically slower than pooled modes and are best reserved for cases that need a fresh host boundary per execution.

### What this snapshot does not prove

- It does not prove exact throughput rankings for every workload or host. The concurrency and tool-call suites are still sensitive to local scheduler noise.
- It does not prove memory behavior for `QuickJsExecutor({ host: "process" })`, because the memory suite intentionally avoids reporting child-process RSS as if it were host-process memory.
- It does not measure `RemoteExecutor`, because remote performance depends on the caller-owned transport and remote runtime deployment.
10 changes: 5 additions & 5 deletions docs/architecture/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,11 @@ flowchart LR

### Package Roles

| Package | Role |
| ------------------ | ------------------------------------------------------------------------------------------------------------ |
| `@execbox/core` | App-facing core types, provider resolution, MCP adapters, plus runtime and protocol subpaths |
| `@execbox/quickjs` | Default QuickJS executor package with inline, worker-hosted, and process-hosted modes plus a reusable runner |
| `@execbox/remote` | Transport-backed executor that runs against an app-defined runner boundary |
| Package | Role |
| ------------------ | -------------------------------------------------------------------------------------------- |
| `@execbox/core` | App-facing core types, provider resolution, MCP adapters, plus runtime and protocol subpaths |
| `@execbox/quickjs` | Default QuickJS executor package with inline and worker-hosted modes plus a reusable runner |
| `@execbox/remote` | Transport-backed executor that runs against an app-defined runner boundary |

## End-to-End Execution Model

Expand Down
4 changes: 2 additions & 2 deletions docs/architecture/execbox-core.md
Original file line number Diff line number Diff line change
Expand Up @@ -189,12 +189,12 @@ Executors are responsible for their own runtime-specific classification rules, b

## Why the Core Stays Small

The core package does not own QuickJS, worker threads, child processes, or transport mechanics. That separation keeps the core useful for:
The core package does not own QuickJS, worker threads, process boundaries, or transport mechanics. That separation keeps the core useful for:

- direct in-process runtimes
- worker-backed runtimes
- MCP wrapper servers
- process or remote execution models
- remote execution models

The consequence is deliberate separation between:

Expand Down
Loading
Loading