Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
## [Unreleased]

### Changed
- **Filesystem data-plane contract tightened: backend is now root-layer-only and `perform fs` reads are deprecated.** The shell/MCP evaluation wrapper no longer routes data-plane reads through `(perform fs ...)`; filesystem reads now go through WASI path I/O (`load`, `import`, `/ipfs/...`, `/ipns/...`). The legacy `fs` handler remains only to return a migration-grade deprecation error. Backend virtual mount resolution now rejects targeted mounts (`source:/guest/path`) and accepts root layers only; `ww run` enforces this early with a CLI preflight error that lists offending mounts. Docs updated across `doc/shell.md`, `doc/capabilities.md`, and `doc/architecture.md` to reflect the single-path model.
- **CompilationService now uses a dedicated worker pool with in-flight dedupe.** The compiler subsystem moved from single-thread compile handling to a fixed worker pool (`WW_COMPILE_WORKERS`, default derived from CPU count), keys cache entries by `(wasm_blake3, engine identity)`, and coalesces concurrent duplicate compile requests so one cold compile serves all waiters.
- **Shell local-discovery policy aligned to per-user run dir.** `ww shell` local discovery now scans only `~/.ww/run/` for `<peer-id>.sock` entries (no `/var/run/ww` fallback), and the client fails deterministically with a disambiguation error when multiple local daemons are present instead of prompting interactively. Updated `src/discovery.rs`, `src/cli/shell.rs`, and shell/CLI docs to match this behavior and keep the local admin auth boundary consistently user-scoped.
- **CLI module boundary cleanup (no behavior change).** Extracted daemon-management helpers, namespace command handlers, and doctor checks from `src/cli/main.rs` into `src/cli/daemon_cmd.rs`, `src/cli/ns_cmd.rs`, and `src/cli/doctor_cmd.rs`, leaving `Commands` as thin delegators. This reduces `main.rs` surface area and improves maintainability while preserving existing command behavior.
Expand Down
110 changes: 50 additions & 60 deletions crates/cell/src/image.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,11 @@
//!
//! Every positional arg to `ww run` is a mount: `source[:target]`.
//! Root mounts (target `/`) are traditional image layers. Targeted
//! mounts overlay a host file or directory at a specific guest path.
//! mounts are currently rejected in backend virtual mode.
//!
//! Mounts are applied left-to-right via `resolve_mounts_virtual`:
//! root layers are DAG-merged at the IPFS MFS level (file blocks never
//! touched, only directory nodes get new CIDs), and targeted mounts
//! become `LocalOverride` entries that the guest's `CidTree` checks
//! before falling through to the CID walk. No file content is
//! touched, only directory nodes get new CIDs). No file content is
//! materialized to disk by this module.
//!
//! Pre-#416 this file also exposed an `apply_mounts` API that
Expand Down Expand Up @@ -169,10 +167,9 @@ async fn merge_overlay_recursive_inner(

/// Resolve mounts into a root CID and local overrides for the virtual filesystem.
///
/// Performs the DAG merge to produce a merged root CID, and collects
/// targeted mounts as `LocalOverride` entries. No file content is copied
/// or fetched — that happens lazily when the guest opens files via
/// `CidTree` + `fs_intercept`.
/// Performs the DAG merge to produce a merged root CID.
/// Targeted mounts are rejected in backend mode to avoid a second,
/// host-local filesystem path.
///
/// Returns `(root_cid, local_overrides)` suitable for constructing a `CidTree`.
pub async fn resolve_mounts_virtual(
Expand All @@ -182,8 +179,6 @@ pub async fn resolve_mounts_virtual(
String,
std::collections::HashMap<std::path::PathBuf, crate::vfs::LocalOverride>,
)> {
use crate::vfs::LocalOverride;

if mounts.is_empty() {
bail!("No mounts provided");
}
Expand All @@ -195,6 +190,14 @@ pub async fn resolve_mounts_virtual(
bail!("No root mounts provided (at least one required)");
}

if !targeted_mounts.is_empty() {
bail!(
"targeted mounts are not supported in backend virtual mode (received {} targeted mount(s)); \
publish content to IPFS/IPNS and mount as a root layer",
targeted_mounts.len()
);
}

// Resolve all root mounts to CIDs.
let mut cids = Vec::with_capacity(root_mounts.len());
for mount in &root_mounts {
Expand Down Expand Up @@ -226,44 +229,7 @@ pub async fn resolve_mounts_virtual(
let root_cid = dag_merge(&cids, ipfs_client).await?;
tracing::info!(cid = %root_cid, layers = cids.len(), "Virtual DAG merge complete");

// Collect targeted mounts as local overrides.
let mut overrides = std::collections::HashMap::new();
for mount in &targeted_mounts {
let guest_path = mount
.target
.strip_prefix("/")
.unwrap_or(&mount.target)
.to_path_buf();

if guest_path.as_os_str().is_empty() {
continue; // skip empty target
}

if ipfs::is_ipfs_path(&mount.source) {
// IPFS targeted mounts are not supported in virtual mode.
// They would need to be added to the CID tree, which conflicts
// with the design goal of keeping private mounts off IPFS.
bail!(
"IPFS targeted mounts not supported in virtual mode: {} -> {}",
mount.source,
mount.target.display()
);
}

let src = Path::new(&mount.source);
if !src.exists() {
bail!("Mount source does not exist: {}", mount.source);
}

let ovr = if src.is_dir() {
LocalOverride::Dir(src.to_path_buf())
} else {
LocalOverride::File(src.to_path_buf())
};
overrides.insert(guest_path, ovr);
}

Ok((root_cid, overrides))
Ok((root_cid, std::collections::HashMap::new()))
}

/// Split `/ipns/<hash>[/<subpath>]` into `(hash, subpath)`. `subpath`
Expand Down Expand Up @@ -378,18 +344,12 @@ mod tests {
//
// Two pure-validation cases live here (no IPFS roundtrip needed).
//
// Merge correctness (`dag_merge` over multiple layers) and the
// targeted-mount → `LocalOverride` translation are NOT unit-tested
// here: those paths require Kubo to `add_dir` local layers, and
// CI's daemon does not reliably accept ephemeral `tempfile::TempDir`
// paths inside the test runner. The previous `apply_mounts` /
// `merge_layers` tests only worked because the deleted code had an
// all-local `copy_merge` fast path that never hit IPFS — now gone.
//
// End-to-end coverage of the merge + override translation lives
// in `tests/discovery_integration.rs` and `tests/shell_e2e.rs`,
// which boot real kernels through `CidTree` against published
// images (paths Kubo already accepts).
// Merge correctness (`dag_merge` over multiple layers) is NOT unit-tested
// here: those paths require Kubo to `add_dir` local layers, and CI's
// daemon does not reliably accept ephemeral `tempfile::TempDir` paths
// inside the test runner. The previous `apply_mounts` / `merge_layers`
// tests only worked because the deleted code had an all-local
// `copy_merge` fast path that never hit IPFS — now gone.

#[tokio::test]
async fn test_virtual_empty_mounts_errors() {
Expand All @@ -407,6 +367,36 @@ mod tests {
assert!(result.is_err());
}

#[tokio::test]
async fn test_virtual_targeted_mounts_rejected() {
let client = stub_ipfs_client();
let mounts = vec![
Mount {
source: "/ipfs/bafybeigdyrzt".to_string(),
target: PathBuf::from("/"),
},
Mount {
source: "./local-secret".to_string(),
target: PathBuf::from("/etc/identity"),
},
];
let result = resolve_mounts_virtual(&mounts, &client).await;
assert!(result.is_err());
let msg = result.unwrap_err().to_string();
assert!(
msg.contains("targeted mounts are not supported in backend virtual mode"),
"unexpected error: {msg}"
);
assert!(
msg.contains("received 1 targeted mount(s)"),
"error should include targeted mount count: {msg}"
);
assert!(
msg.contains("publish content to IPFS/IPNS and mount as a root layer"),
"error should include migration guidance: {msg}"
);
}

// ── split_ipns_path: pure parsing, IPNS-to-IPFS subpath split ──

#[test]
Expand Down
15 changes: 7 additions & 8 deletions doc/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -395,21 +395,20 @@ preopen.
2. **Root Atom binding** — the `stem::Atom` whose value is the cell's
root CID. Swap the Atom and `CidTree::swap_root` updates the view
atomically.
3. **Glia env bindings** — what's callable inside the cell. Restrict
filesystem access by not binding the `fs` handler in the cell's
env (`std/caps/src/lib.rs:make_fs_handler`).
3. **Glia env bindings** — what's callable inside the cell. Filesystem
data-plane access is via WASI path I/O (`load`, `import`, guest file
reads), not a separate `perform fs` surface.

`LocalOverride` (`src/vfs.rs:59-62`) is the only principled exception:
per-file host-local mounts checked before CID resolution, so private
content (identity keys, per-node secrets) never enters IPFS. Use
sparingly and only for genuinely host-private files.
Backend virtual mode now rejects targeted mounts, so host-local overrides are
not active in the backend mount path. Data-plane content for backend cells must
flow through `/ipfs` / `/ipns` root layers.

Revocation = epoch advance + respawn. Classical ocap: you cannot
un-hand a CID. Advance the epoch (RPC caps fail `staleEpoch`), kill
and respawn the cell under a different root Atom — the new cell sees
a different slice of the universe.

For the agent-facing view of all this (`fs/*`, `(schema cap)`,
For the agent-facing view of all this (WASI path I/O, `(schema cap)`,
structured errors, attenuation strategies), see
[capabilities.md](capabilities.md).

Expand Down
47 changes: 16 additions & 31 deletions doc/capabilities.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,46 +77,31 @@ appearing in the top-level graft list.
Every capability is epoch-guarded: it fails with `staleEpoch` once the
on-chain head advances, forcing a re-graft.

### Content access (no `fs` capability over RPC)
### Content access (WASI path I/O only)

Cells do not receive an explicit filesystem capability over the
membrane. Content access flows through the WASI virtual filesystem,
which the host backs with `CidTree`. Glia code reaches it via the
`fs` capability bound into the cell's env, invoked through `perform`:
which the host backs with `CidTree`.

- `(perform fs :read path)` — bytes
- `(perform fs :read-str path)` — UTF-8 string
- `(perform fs :ls path)` — list of `{:name :size :type}` maps
- `(perform fs :stat path)` — `{:size :type}`
- `(perform fs :exists? path)` — bool, never raises on missing paths
Use regular guest file I/O against filesystem paths:
- `(load "path")` for bytes in Glia
- `(perform import "module")` for module loading
- direct guest reads via WASI-aware code under `/ipfs/<cid>/...` and
`/ipns/<name>/...`

Path resolution (`std/caps/src/lib.rs:resolve_fs_path`): absolute paths
including `/ipfs/<cid>` and `/ipns/<key>` pass through to the WASI VFS;
relative paths resolve against `$WW_ROOT` (defaults to `/`).

The `fs` cap is wired by the kernel after graft (`std/kernel/src/lib.rs`);
the shell and MCP cells wire it the same way (`std/shell/src/lib.rs`,
`std/mcp/src/lib.rs`). Restricting filesystem access for a particular
cell means not binding the `fs` handler in its env — the language layer
is the attenuation point, not a separate cap surface.
`(perform fs ...)` is legacy and returns a migration error. Keeping a separate
`perform` filesystem read surface created dual-path semantics and is being
removed.

## Local overrides

`LocalOverride` (`src/vfs.rs:59-62`) is the only principled exception
to "everything resolves through CIDs." It supports per-file or
per-directory host-local mounts that are checked **before** CID
resolution, so private content (the operator's identity key,
per-node secrets) never enters IPFS.

```
guest opens /etc/identity/key.pem
→ CidTree::resolve_path
→ check LocalOverride map first → matches → return ResolvedNode::LocalFile(host_path)
→ never falls through to the CID walk
```
Backend virtual mode rejects targeted mounts, so host-local overrides are
currently not part of the backend runtime surface. Publish content to IPFS/IPNS
and mount it as a root layer instead.

Use this exception sparingly. It is meant for genuinely host-private
files. Anything that should be reachable across nodes belongs in IPFS.
`LocalOverride` types remain in the codebase as implementation scaffolding for
future shell-local workflows, but they are not used by `ww run` backend mount
resolution in this mode.

## Capability lifecycle

Expand Down
11 changes: 7 additions & 4 deletions doc/shell.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,15 +76,18 @@ The shell cell currently grafts these caps from its membrane (see

### Local effect handlers

The shell cell also wires three glia-only effect handlers that do not
go through a remote capability:
The shell cell wires Glia-only local helpers that do not go through a
remote capability:

- **`fs`** — read paths from the cell's WASI filesystem (reactive to
stem updates per [`architecture.md`](architecture.md))
- **`import`** — load other Glia source files
- **`load`** dispatch form — read bytes from the WASI filesystem
- The kernel's built-in expressions: arithmetic, `let`, `if`, `defn`,
etc.

`(perform fs ...)` is deprecated and returns a migration error. Filesystem
reads should go through normal WASI file I/O (`load`, `import`, and guest
file operations on `/ipfs/...` / `/ipns/...` paths).

### Built-ins

| Form | Description |
Expand Down
Loading