minimum-kernel: capability-based PVM + microkernel#828
Draft
sorpaas wants to merge 170 commits into
Draft
Conversation
Move javm, javm-transpiler (was grey-transpiler), build-crate, and
build-javm to a fresh rust/ tree; sever javm's grey-crypto dependency
so rust/* depends on nothing in grey/*.
Add four new crates implementing the spec at ~/docs/minimum/:
- jar-types — Capability variants (persistent + ephemeral), CapRecord,
CNode, Vault, State (σ), Block/Header/Body, traces, SlotContent,
Caller/KernelRole, Command. BTreeMap-backed for canonical iteration.
- jar-crypto — blake2b_256 + ed25519-dalek facade (BLS stubbed).
- jar-kernel — apply_block 3-phase orchestration, transact phase with
per-event Arc-CoW snapshot/rollback, off-chain step-2/step-3
dispatch driver, cap registry with cascade revoke, three pinning
enforcement points (grant/move, derive, arg-scan), 20 host-call
handlers, VmExec abstraction (ScriptVm + JavmVm), AttestationCap
mode-blind verify-vs-sign, state-root hashing, genesis builder,
Hardware trait + InMemoryHardware.
- jar — N-node in-process testnet binary with round-robin proposer.
21 tests cover cap registry / pinning / arg-scan / snapshot rollback /
storage quota / apply_block accept+reject / dispatch step-2-3 /
ed25519 round-trip. End-to-end: 4-node × 8-slot testnet converges
on every state-root.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Leftover scaffolding from the initial pass — fake-use functions kept imports alive while modules were under construction. The imports they referenced are either real-used now or genuinely unused; either way the stubs and any orphaned imports come out cleanly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Match the spec's fold of block_validation_cap / block_finalization_cap
into ordinary Transact entrypoints at body.events[0] / body.events[-1].
jar-types changes:
- State drops block_validation_cap, block_finalization_cap, bookkeeping.
- Bookkeeping struct → IdCounters (just monotonic id allocation).
- Block becomes { parent, body }; Header struct removed.
- Body.events becomes Vec<(VaultId, Event)> — flat ordered list.
- KernelRole drops BlockValidation / BlockFinalization variants.
jar-kernel changes:
- apply_block becomes single-phase: walk body.events in list order;
structural backstop is parent linkage + global trace exhaustion.
- Drop run_policy_phase orchestration entirely.
- dispatch driver adds trace-exhaustion check at end of step-2 and
step-3 boundaries (mismatch faults the invocation).
- genesis builder drops policy-Vault setup; chain-author event[0]
and event[-1] are ordinary Transact entrypoints.
- state_root drops bookkeeping fields from canonical encoding.
- host_calls.encode_caller drops the two BlockX KernelRoles.
jar binary updated to construct Block { parent, body } directly.
Tests updated accordingly. cargo test -p jar-kernel: 19 passing.
4-node × 6-slot testnet still converges deterministically.
Match the spec change: σ.transact_space_cnode holds a mix of
`Transact` and `Schedule` caps in slot order; the kernel walks it
each block. Schedule slots fire kernel-side once per block with no
body input; Transact slots consume body.events entries.
jar-types changes:
- Capability gains a `Schedule { vault_id, born_in }` variant.
- is_pinned_or_ref / vault_id accessors include Schedule.
- Body.events: Vec<(VaultId, Event)> → Vec<(VaultId, Vec<Event>)>.
jar-kernel changes:
- transact::run_phase walks σ.transact_space_cnode in slot order;
branches Transact (consume matching body entry, run each event)
vs Schedule (kernel-fire once, no body input). Body well-
formedness enforced in-line: VaultIds in same relative order as
Transact slots, no body entry references a Schedule slot, no
trailing unmatched entries.
- pinning rules extended to Schedule: same born_in pinning as
Transact/Dispatch; Schedule→Schedule derive requires persistent
dest; no ScheduleRef.
- host_calls.cap_call rejects Schedule (kernel-fired only).
- genesis builder produces σ.transact_space_cnode with example
layout: slot 0 Schedule(block_init), slot 1 Transact, slot 2
Schedule(block_final).
- proposer::drain_for_body groups events per Transact target,
orders entries by transact_space_cnode slot index, splices each
event's transport attestation_trace/result_trace into body-level
traces. Body-side Events have empty per-event traces (transport-
only field).
Tests:
- body_events_order_must_match_transact_space_cnode (new): valid
body with Transact-target reference is accepted.
- body_events_referencing_schedule_slot_is_rejected (new): body
referencing a Schedule slot's vault_id is rejected.
- state_root_advances_with_schedule_slots_firing (renamed): the
two genesis Schedule slots fire each block, so state_root
advances even for empty body.
cargo test --workspace: 104 suites, 1286 passed, 0 failed.
4-node × 5-slot testnet still converges.
Match the spec change: Transact and Dispatch invocations consume
from per-event traces (event.attestation_trace,
event.result_trace); Schedule invocations and the block-seal
Sealing entry use block-level traces (body.attestation_trace,
body.result_trace).
transact.rs:
- run_one_invocation now takes `payload: &[u8]` instead of
`event: Option<&Event>` so callers can keep mutable borrows of
event.attestation_trace / event.result_trace alongside the call.
- run_phase routes traces by SlotKind:
* Schedule slot → block-level traces, block-level cursor (the
cursor parameter, renamed `block_cursor`).
* Transact slot → per-event traces with a fresh cursor each
event; per-event boundary check at HALT (cursor must equal
event.attestation_trace.len() / event.result_trace.len()).
- Per-event mutation handled by std::mem::take + put-back idiom
around the call so we can pass &mut event.attestation_trace
while body's other fields remain mutably borrowed elsewhere.
proposer.rs:
- drain_for_body keeps each AggregatedTransact's bundled traces
on the per-event Event (no longer splices into body-level
traces). body.attestation_trace / body.result_trace start
empty; populated only by Schedule invocations during apply_block.
Tests:
- transact_event_with_unconsumed_attestation_trace_faults (new):
confirms per-event boundary check faults when handler doesn't
consume the full event-level slice.
- All other existing tests still pass.
cargo test --workspace: 104 suites, 1287 passed, 0 failed (one more
than previous baseline). 3-node testnet still converges.
Introduce a `Crypto` trait in jar-types whose associated types name the hash function, signature curve, and key identifier of a crypto suite. Every parametric type in jar-types (`State`, `Block`, `Body`, `Event`, `AttestationEntry`, `Capability`, `CapRecord`, `Vault`, `SlotContent`, `Command`, `MerkleProof`) now carries a `<C: Crypto>` parameter, with manual `Clone`/`Eq`/`PartialEq`/`Debug`/`Default` impls so the bounds land on `C::Hash` / `C::KeyId` / `C::Signature` rather than on `C` itself. `Hardware: Crypto` — the Hardware trait inherits the type-only Crypto suite. Adds `hash` and `verify` methods so kernel code routes hashing / signature verification through Hardware instead of importing jar_crypto directly. `state_root` and `attest` no longer use jar_crypto::*. `jar-crypto::Ed25519Blake` is the v1 concrete suite (Hash=blake2b-256, Signature=Ed25519). `InMemoryHardware: Hardware` uses these types. The kernel still exposes free-standing `apply_block` / `handle_inbound_dispatch` / `drain_for_body` / `state_root` functions — the `Kernel<H>` struct that wraps them is the next step. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wraps `H: Hardware` in a single struct that exposes `apply_block`, `handle_inbound_dispatch`, `drain_for_body`, and `state_root` as methods. State is borrowed per-call (not owned) so a single Kernel can serve many State values (e.g., per-fork). Wrap the Kernel itself in `Arc<Kernel<H>>` for cross-task sharing. The free-standing functions in `apply_block::*`, `dispatch::*`, `proposer::*`, `state_root::*` are no longer re-exported from the crate root — `Kernel<H>::method()` is the public surface. The free functions remain reachable through their submodule path for advanced callers that want to skip the wrapper. Tests and `jar/` binary migrated to method syntax; 3-node testnet converges as before. Workspace: 104 suites, all green; clippy clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous Crypto-trait parametrization across jar-types put crypto on
the wrong side of the boundary and pushed type-parameter clutter through
every type. This collapses it: crypto is the kernel's concern, hardware
gets a much smaller surface.
Core changes:
- **Crypto is kernel-static**, not Hardware-pluggable. New
`jar_kernel::crypto` module with `hash`, `verify`, and `block_hash`
free functions. The chain commits to one curve (Ed25519) + one hash
(blake2b-256) at the protocol level. Userspace never sees these
types directly; it goes through `attest()` / host calls.
- **`jar-types` non-generic again.** Drop `Crypto` trait, `Ed25519Blake`,
every `<C: Crypto>` parameter. `Hash` stays `[u8; 32]`; `KeyId` and
`Signature` become `Vec<u8>` newtypes (variable-width to support
Ed25519 today + BLS later without curve enforcement at the type
level — verification failures handle the curve mismatch at runtime).
All #[derive]s restored.
- **Hardware trait shrinks.** Drops `hash` / `verify`; keeps only the
ops that need external resources: `sign`, `holds_key` (secret-key
custody), `emit` (network outbox), and `score` / `finalize` / `head`
for fork-tree / finality bookkeeping. The kernel decides what's
valid; hardware just records and does fork choice. `InMemoryHardware`
gets a tiny ForkTree.
- **Storage authority is encoded in cap variants.** Drop `StorageMode`
from `InvocationCtx` and storage host calls. New
`Capability::SnapshotStorage { vault_id, key_range, root }` for
read-only views of committed prior state; existing `Capability::Storage`
stays as the overlay (RW). Frame builders pick the right cap:
Transact/Schedule frames get `Storage` (RW), Dispatch step-2/3 frames
get `SnapshotStorage`. Storage host calls dispatch on the cap variant.
- **`MerkleProof` becomes opaque** (`proof: Vec<u8>`). The proof shape
is hardware-defined; the kernel just stores and forwards. Phase 1
generates none; Phase 2 wires real proofs through hardware.
- **Block stays kernel-side.** The kernel owns the block layout and
`block_hash` (canonical encoding via `crypto::block_hash`). Hardware
indexes blocks by the kernel-computed hash; it never reads the body.
`Command::Score` / `Command::Finalize` are how the kernel pushes
fork-tree updates to hardware.
Net effect on jar-kernel internals: every `<C: Crypto>` and most
`<H: Hardware>` parameters that existed only to satisfy parametric
storage types are gone. Functions that actually call hardware methods
keep `<H: Hardware>`; the rest are non-generic. Tests no longer write
`<Hw>` annotations.
Verification: 104 test suites pass, 0 failures; workspace clippy clean;
3-node testnet converges (proposer + 2 verifiers, 3 slots round-robin).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reshape `Kernel<H>` from a stateless wrapper (`Kernel::new(hw)` plus
borrowing-by-call methods) to a stateful node tip:
struct Kernel<H> {
hw: H,
last_state: State,
last_block_hash: BlockHash,
dispatches: NodeOffchain,
}
The kernel now owns σ for the tip it's building on. Hardware persists
state keyed by block_hash; the kernel asks for it at construction and
calls back to commit after each accepted block.
New API:
- `Kernel::new(block_hash: Option<BlockHash>, hw: H)` — load tip from
hardware (`None` → genesis state via `hw.genesis_state()`; `Some(h)` →
`hw.state_at(h)`). Subscribes to all top-level Dispatch entrypoints
in σ via `hw.subscribe(vault_id)`.
- `Kernel::dispatch(entrypoint, event)` — process one inbound Dispatch
event. Updates the in-memory dispatch list and emits any commands
produced by step-2/step-3 to hardware.
- `Kernel::advance(block: Option<Block>)` — build a new block (`None` =
proposer mode, draining the dispatch list into a body) or verify a
received block (`Some(b)`). On success, commits new state to hardware
via `hw.commit_state` and pushes a `Score` for fork-tree bookkeeping;
updates the kernel's tip.
`Hardware` trait grows:
- `genesis_state()` — chain genesis (configured at hw construction).
- `state_at(block_hash)` — load committed state at a prior block.
- `commit_state(block_hash, state)` — persist post-block state.
- `subscribe(vault_id)` — register a Dispatch entrypoint with the network
layer.
`InMemoryHardware::new(genesis, bus)` now requires the chain's genesis
state; an internal `BTreeMap<BlockHash, State>` indexes committed states.
Hardware ownership: `Kernel<H>` owns `H` directly, not behind `Arc<H>`.
The runtime creates one `Kernel<H>` per node (testnet binary updated to
match — each `NodeState` is just `id + Kernel<InMemoryHardware>`).
`NodeOffchain` becomes kernel-internal; the standalone `apply_block` /
`handle_inbound_dispatch` / `drain_for_body` re-exports are gone — the
public surface is `Kernel<H>`. Internal modules keep their free
functions for the kernel methods to call.
Deferred: making `State` a trait + a `Hardware::apply(state, changeset)
-> State` story. That requires rewriting cap_registry/cnode_ops/storage
to emit changeset ops instead of mutating concrete fields — substantial
follow-up work. Today the kernel still mutates a concrete `State`
struct in-memory; hardware just stores the post-block snapshot.
Verification: all jar-kernel + workspace tests green (104 suites);
clippy + fmt clean; 3-node testnet converges (proposer + 2 verifiers
agree on state_root and block_hash for 3 slots round-robin).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The kernel now drives `javm::kernel::InvocationKernel` directly. Every
Transact and every Dispatch step-2/step-3 instantiates a real PVM VM and
runs PVM bytecode end-to-end. The `VmExec` trait, `JavmVm` wrapper, and
`ScriptVm` smoke double are gone — javm's surface (DATA-cap memory,
CodeCache, init-time args, ecalli/management ops) does not compress into
a portable trait without erasing structure.
State changes:
- Reserve `state.code_vault: VaultId` for content-addressed blob storage;
bound into the state-root.
- New `code_blobs::resolve_code_blob` and `register_blob`. Genesis now
takes blob bytes (not precomputed hashes), registers them into
`state.code_vault`'s storage, and binds each reserved vault's
`code_hash` to the corresponding blob.
Test fixtures:
- New `jar-test-services/halt` and `jar-test-services/slot_clear` crates
compile to PVM blobs at build time via `build-javm::build_service`.
- `slot_clear` is phase-aware via φ[7]: 0 → halt, 1 → ecalli SlotClear
then halt.
Host calls:
- Handlers now return `HostCallOutcome::{Resume(r0,r1), Fault(reason)}`;
guest-driven memory-window failures graceful-fault the invocation
instead of erroring the kernel.
- New `HostCall::SlotRead` (slot 21) lets step-3 guests read SCALE-encoded
prev-slot bytes via a guest memory window. `InvocationCtx` carries
`prev_slot: Option<&SlotContent>`, populated only in step-3.
Dispatch:
- Both phases instantiate one `InvocationKernel::new_cached` per phase
against the entrypoint blob, sharing `node.code_cache`. The "same VM
(memory persists)" spec aspiration is documented as deferred — javm's
terminal-Halt model needs cooperative-yield support to be literal.
Verification:
- jar-kernel tests pass (apply_block 8/8, cap_registry 7/7,
dispatch_pipeline 1/1, snapshot 2/2, storage_quota 5/5).
- Workspace clippy clean; cargo fmt clean.
- 3-node testnet converges across slots with real javm.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… crates
JAM (the Polkadot Join-Accumulate Machine) is no longer the project's
direction; the minimum-JAR microkernel under rust/ is. Keeping grey in
the workspace pays an unnecessary cost on every cargo test --workspace.
Crate moves:
- grey/crates/javm-builtins → rust/javm-builtins
- grey/crates/scale → rust/scale
- grey/crates/scale-derive → rust/scale-derive
- grey/crates/build-pvm → rust/build-pvm
- grey/crates/grey-bench → rust/javm-bench (package renamed)
- grey/services/javm-guest-tests → rust/javm-guest-tests
- grey/services/benches/{blake2b,ecrecover,ed25519,keccak,prime-sieve}
→ components/benches/{blake2b,ecrecover,ed25519,keccak,prime-sieve}
The new top-level components/ tree replaces the role of grey/services/;
today it holds the bench guest crates, future home for component-shaped
guest code.
Workspace Cargo.toml:
- Drop all grey/* members and the ten grey-* workspace.dependencies
entries.
- Add grey to exclude — files stay on disk for reference; cargo never
recurses.
- Update scale path to rust/scale.
javm-bench:
- Renamed from grey-bench. Drop sample-service blob (JAM service) and
the three sample_service tests; rename grey_/GREY_ identifiers to
javm_/JAVM_ throughout src/, benches/, examples/, build.rs.
CLAUDE.md trimmed: drop @grey/AGENTS.md include and grey-specific
workflows (harness, spec test vectors); replace with rust-workspace
build/test guidance.
grey/ directory remains on disk but is no longer buildable in isolation
(it shared the root workspace). Restoring buildability is a future PR.
Verification:
- cargo build --workspace clean.
- cargo test --workspace: 136 tests pass across 5 reporting crates.
- cargo clippy --workspace --all-targets clean.
- cargo fmt --all clean.
- 3-node testnet converges across 3 slots with real javm.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`jar-kernel` is THE primitive: light clients, network validators, and any meaningful operation route through it. Splitting types and crypto primitives across sibling crates was gratuitous boundary; both were leaves of the dependency graph. Changes: - Move rust/jar-types/src/* → rust/jar-kernel/src/types/ - Move rust/jar-crypto/src/lib.rs → rust/jar-kernel/src/crypto_primitives.rs - Drop rust/jar-types and rust/jar-crypto from the workspace and workspace.dependencies. - jar-kernel/Cargo.toml now pulls blake2, ed25519-dalek, rand directly (was via jar-crypto re-export). - Internal `use jar_types::*` and `use jar_crypto::*` rewritten as `crate::types::*` / `crate::crypto_primitives::*`. Within types/, the old `use crate::*` cross-references become `use super::*`. - jar-kernel/src/lib.rs re-exports `pub use crate::types::*` so tests and external consumers continue to see the old `jar_kernel::Hash` etc. shape. - rust/jar/Cargo.toml drops jar-types and jar-crypto deps; main.rs only ever imported through jar_kernel re-exports, so no source change there. This is move-only; no logic changes. Subsequent commits will reorganize modules by struct-locality and lift Capability variants into structs. Verification: - cargo build/test/clippy/fmt clean across the workspace. - 3-node testnet converges across 3 slots (state-roots match prior baseline byte-for-byte). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Group each struct family with its helpers under a single directory. Move
state mutators under `state/`, cap helpers under `cap/`, and the VM
driver + host calls under `vm/`. Flatten `runtime/` and consolidate
crypto into a single file.
Layout changes:
src/
├── crypto.rs (+ primitives merged from crypto_primitives.rs)
├── runtime.rs (was runtime/{mod, hardware, in_memory}.rs — flat)
├── state/
│ ├── mod.rs (was types/state.rs — State + Vault + IdCounters)
│ ├── cap_registry.rs (was top-level cap_registry.rs)
│ ├── cnode.rs (was cnode_ops.rs)
│ ├── storage.rs
│ ├── snapshot.rs
│ ├── state_root.rs
│ └── code_blobs.rs
├── cap/
│ ├── mod.rs (placeholder for commit 3 variant structs)
│ ├── pinning.rs
│ └── attest.rs
├── vm/
│ ├── mod.rs (was invocation.rs — InvocationCtx + drive_invocation)
│ ├── frame.rs
│ ├── host_abi.rs
│ └── host_calls/
│ ├── mod.rs (top-level dispatcher + read_window/write_window)
│ ├── storage.rs (host_storage_read/write/delete)
│ ├── cnode.rs (host_cnode_grant/revoke/move)
│ ├── cap.rs (host_cap_derive/call/vault_initialize/create_vault/quota_set)
│ ├── attest.rs (host_attest/attestation_key/result_equal)
│ └── slot.rs (host_slot_clear/slot_read + encode_slot)
└── (apply_block, transact, dispatch, proposer, reach, kernel,
genesis, types, lib remain at top level)
The old 782-LoC `host_calls.rs` is now five files of 40-260 LoC each,
each focused on one concern. Top-level dispatcher in mod.rs is 60 LoC.
This is move-only; no logic changes. State-roots and block-hashes match
the prior baseline byte-for-byte. Commit 3 will lift Capability variants
into named structs in cap/.
Verification:
- cargo build/test/clippy/fmt clean across the workspace.
- 3-node testnet converges across 3 slots (state-roots match).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Each Capability variant becomes its own named struct (`DispatchCap`,
`TransactCap`, `StorageCap`, …); the `Capability` enum is now a sum of
those structs. Generic code that operates on a single variant can take a
concrete `&DispatchCap` reference; helper logic that branches on variant
shape uses `match cap { Capability::Foo(c) => ... c.field ... }`.
No trait introduced — the user explicitly opted out of `CapabilityT` for
this round. The enum-and-match form is plenty for the call sites we
have; a trait can be added later if/when a real consumer surfaces a
shared dispatch surface.
Specifically:
- `cap/capability.rs` (was `types/cap.rs`) defines 15 variant structs
plus the `Capability` enum.
- `Capability::is_pinned_or_ref()`, `is_ephemeral()`, `vault_id()` are
rewritten on the new shape.
- All ~30 match/construct sites swept across `genesis.rs`, `kernel.rs`,
`transact.rs`, `dispatch.rs`, `proposer.rs`, `state/storage.rs`,
`cap/pinning.rs`, `cap/attest.rs`, `vm/host_calls/{cnode, cap, attest}.rs`,
and the integration tests.
- `types::Capability` and friends remain re-exported from
`crate::cap::capability::*` so existing `use jar_kernel::Capability`
paths keep working.
State-roots and block-hashes change in this commit because the
state_root encoding hashes capability records via their Debug output,
and the Debug shape moved from `Capability::Foo { field }` to
`Capability::Foo(FooCap { field })`. No tests hardcode specific roots;
the 3-node testnet still converges, just on different bytes.
Verification:
- cargo build/test/clippy/fmt clean across the workspace.
- 3-node testnet converges across 3 slots (state-roots match across nodes).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per the plan in distributed-puzzling-tower.md, javm's `Cap` enum
becomes generic over a protocol-cap payload type `P`. The old
`Cap::Protocol(ProtocolCap{id: u8})` shape collapses into `Cap::Protocol(P)`
where `P: ProtocolCapT`. Default `P = u8` keeps existing javm-bench /
javm-guest-tests behaviour bit-for-bit at the type level.
`ProtocolCapT` is the trait the kernel uses to express per-variant
copy/move/drop policy:
```rust
pub trait ProtocolCapT: Clone + core::fmt::Debug {
fn is_copyable(&self) -> bool { true }
fn is_movable(&self) -> bool { true }
fn is_droppable(&self) -> bool { true }
}
impl ProtocolCapT for u8 {}
```
`Cap::is_copyable` / `try_copy` consult the trait method on the
Protocol arm so jar-kernel (or future consumers) can refuse COPY/MOVE
on pinned caps. `try_copy` clones `P` only when allowed.
Drop the JAM-era 1..=28 auto-populate at `InvocationKernel::new`. This
range was a leftover assumption that javm bakes JAR's host-call layout
into VM construction. The kernel now populates whatever protocol slots
it actually uses via the new `cap_table_set` / `cap_table_set_original`
accessors on `InvocationKernel`. javm-bench preserves the legacy
behaviour by populating slots 1..=28 in `run_kernel_with_backend` after
constructing the kernel — a tiny consumer-side concession that keeps
the bench guests working.
`KernelResult::ProtocolCall { slot }` semantics unchanged: the slot
field carries the cap-table index. Host loops fetch the cap via
`vm.cap_table_get(slot)` and dispatch on the inner payload.
Type-parameter propagation:
- `Cap<P>`, `CapTable<P>` in cap.rs.
- `VmInstance<P>`, `VmArena<P>` in vm_pool.rs.
- `InvocationKernel<P>` in kernel.rs.
- Internal handlers (`create_cap_from_manifest`, etc.) propagate `<P>`
via `impl<P: ProtocolCapT>` blocks.
Test sites in javm + javm-bench + javm-guest-tests + javm-transpiler
gain explicit type annotations (`InvocationKernel`, `VmInstance`,
`VmArena`, or turbofish `::<u8>`) where Rust can't infer `P` from the
default. Tests pass with byte-for-byte identical state-roots in the
3-node testnet.
Verification:
- cargo test --workspace: 54 test suites pass, no failures.
- cargo clippy --workspace --all-targets clean.
- 3-node testnet converges across 3 slots with same state-roots as
before (this commit doesn't touch jar-kernel's frame plumbing).
Next: commit 2 introduces `KernelCap`, deletes jar-kernel's parallel
`Frame` struct, and repoints host-call lookups at javm's CapTable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ol caps
Per the spec, a Frame is the running VM's working cap-table — a
256-slot ephemeral cap-table indistinguishable in shape from a Vault's
persistent slots. The kernel previously maintained a parallel
`Frame { slots: BTreeMap<u8, CapId> }` because javm's `Cap` enum had
no slot for kernel-defined caps. With commit 1 making `Cap::Protocol(P)`
generic, the kernel can substitute its own `KernelCap` payload and
delete the parallel Frame.
`KernelCap` (in `cap/kernel_cap.rs`):
```rust
pub enum KernelCap {
HostCall(u8), // host-call selector slots (currently 1..=21)
Cap(Capability), // a real kernel cap held in a cap-table slot
}
impl ProtocolCapT for KernelCap { ... pinning-aware is_copyable ... }
```
The wrapper keeps `Capability` semantically pure (real kernel caps
only). The `HostCall` arm shrinks to nothing as host calls retire to
javm-management ecallis in future PRs.
Layout changes in `transact.rs` and `dispatch.rs`:
- `Vm::new(...)` (alias for `InvocationKernel<KernelCap>`).
- `populate_host_call_slots(&mut vm)` writes `Cap::Protocol(KernelCap::HostCall(N))`
at slots 1..=21 — the kernel's current host-call range.
- `populate_storage_slot(&mut vm, vault, writable, snapshot_root)`
writes the per-invocation Storage / SnapshotStorage cap at
`KERNEL_CAP_SLOT = 32`. (Free slot, away from javm's CODE/DATA at
64+, the IPC slot 0, and the host-call range 1..=21.)
- `InvocationCtx` no longer carries a `frame: Frame` field.
- `vm/frame.rs` deleted. `vm/mod.rs` no longer declares it.
Host-call handlers (`vm/host_calls/`):
- `fetch_kernel_cap(vm, slot) -> Option<&Capability>` is the replacement
for `ctx.frame.get(slot) → cap_registry::lookup`. One indirection
fewer, and the cap data is inline (no CapId allocation for Frame
caps).
- `storage.rs` and `attest.rs` rewritten against `fetch_kernel_cap`.
- `slot.rs` (`slot_clear`, `slot_read`) takes `&mut Vm` directly.
- `cnode.rs` (cnode_grant / revoke / move) and `cap.rs` (cap_derive /
cap_call / vault_initialize / create_vault / quota_set) **stubbed
to RC_UNIMPLEMENTED**. These host calls are slated for retirement
in favour of javm-management ecallis on the unified cap-table; not
exercised by current tests or the testnet.
State module:
- `state::storage::storage_{read,write,delete}` now take `&Capability`
instead of `CapId`. The cap is fetched from the cap-table directly
(no registry lookup needed for ephemeral Frame caps).
Test updates:
- `tests/storage_quota.rs`: pass `&Capability` directly instead of
allocating a CapId via cap_registry.
- `tests/apply_block.rs`: rename
`state_root_advances_with_schedule_slots_firing` to
`state_root_stable_when_schedule_slots_only_halt` and assert
state-root is unchanged (the previous behaviour was incidental: it
came from spurious cap_registry entries allocated for ephemeral
Frame caps; the new design doesn't pollute σ).
Verification:
- cargo test --workspace clean (no failures).
- cargo clippy --workspace --all-targets clean.
- 3-node testnet converges across 3 slots (state-roots match across
nodes; new state-root prefix is `Hash([136,48,87,224,...])`,
different from before because Storage caps are no longer registered
in σ.cap_registry).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…EVOKE CREATE (call-on-CODE) bitmask-copies caps from CODE-CNode to the child VM's cap-table. Use set_original / set based on whether the source slot was marked original — kernel-installed protocol caps stay JIT-fast-path eligible in the child. MGMT_GRANT (0x8) and MGMT_REVOKE (0x9) were stubbed to RC_WHAT and never reached real logic. Cross-cap-table transfers happen via dynamic-ecall MOVE / COPY (dispatch_ecall 0x06 / 0x07) with cap-ref indirection through HandleCaps. Drop the constants and the dead match arm.
`cap_call` was already a RC_UNIMPLEMENTED stub. Entrypoint invocation goes through plain javm CALL on a Handle / Callable cap-table slot — the host-call indirection is dead weight. Drop the variant, the from_slot mapping, the host_cap_call stub, and the dispatcher arm. Slot 11 stays reserved (no renumbering — guest blobs reference slot indices, and renumbering would silently shift everything above 11). populate_host_call_slots now skips it; an ecalli on slot 11 hits an empty cap and surfaces as guest WHAT.
All three were already RC_UNIMPLEMENTED stubs slated for retirement.
The replacement direction is javm management ecallis on the unified
cap-table:
- CnodeGrant → dynamic-ecall COPY (0x07) with cap-indirection
- CnodeMove → dynamic-ecall MOVE (0x06) with cap-indirection
- CnodeRevoke → MGMT_DROP (cascade-revoke is jar-kernel-internal,
used by the kernel itself for genesis/dispatch)
Drop the variants, the from_slot mappings, the stub handlers (whole
file), the dispatcher arms, and the cnode module decl. Slots 7/8/9
stay reserved gaps alongside slot 11 — guest blobs reference slot
indices, and renumbering would silently shift slot 10 / 12..21.
populate_host_call_slots now skips {7, 8, 9, 11}; ecalli on any of
them hits an empty cap-table entry and surfaces as javm WHAT.
State-level cnode/registry helpers (state/cnode.rs, cap_registry.rs)
are unchanged — still used by genesis, dispatch walks, and cascade
revocation inside the kernel.
All six were either RC_UNIMPLEMENTED stubs or silent-success no-ops with no callers anywhere in the workspace: 10 CapDerive → javm MGMT_DOWNGRADE 12 VaultInitialize → kernel-internal Command, not a guest call 13 CreateVault → kernel-internal Command, not a guest call 14 QuotaSet → kernel-internal Command, not a guest call 17 AttestationAggregate → was Resume(0, 0); BLS aggregation unwired 20 SlotEmit → step-3 emit unimplemented Drop the variants, the from_slot mappings, the four cap.rs handlers (file deleted), the six dispatcher arms, and the now-unused RC_UNIMPLEMENTED sentinel. Slots 10/12/13/14/17/20 join 7/8/9/11 as documented reserved gaps; we don't renumber to preserve binary stability for any future guest blob. populate_host_call_slots is now driven directly by HostCall::from_slot — if from_slot returns Err, the slot stays empty. One source of truth. Surviving HostCall variants (11 → effectively 11 - 6 = 11 - 6 = 11): Gas, SelfId, Caller, StorageRead/Write/Delete, Attest, AttestationKey, ResultEqual, SlotClear, SlotRead.
…folding) Adds a first-class cap variant for the per-invocation ephemeral table and a generational arena to back it, plumbed through InvocationKernel. No instances are allocated yet — Phase 6 wires up allocation at invocation entry. This commit just makes the type system aware of the new variant so subsequent commits can land without each touching every match arm. EphemeralTable caps are non-copyable (single shared table) and not movable by guests (kernel-managed lifetime).
Adds two new management ecallis:
0xD MGMT_GAS_DERIVE — split N units off a Gas cap into a fresh cap
0xE MGMT_GAS_MERGE — combine two Gas caps; donor consumed
Both route through new default-no-op trait methods on `ProtocolCapT`:
fn gas_derive(&mut self, amount: u64) -> Option<Self> { None }
fn gas_merge(&mut self, donor: &Self) -> bool { false }
The protocol payload type decides what counts as a Gas cap. For plain
`u8` (javm-bench, guest tests, internal fixtures), both ops always fail
with WHAT — no behavior change. jar-kernel will impl these for
`Capability::Gas { remaining }` in a future commit.
This is the first step toward replacing MGMT_SET_MAX_GAS with the
park-then-merge gas-restriction pattern from the per-invocation
ephemeral table design (see ~/docs/minimum/08-pvm.md).
…as_mapped
DataCap's single (base_offset, access) pair is replaced by a per-VM
mapping table:
pub struct DataCap {
pub mappings: Vec<VmMapping>, // per-VM: (base, access)
pub active_in: Option<VmId>, // currently mapped in this VM
...
}
Cross-frame MOVE auto-unmaps from `active_in`'s window, preserves the
recorded mapping in `mappings`, and on arrival in the destination VM's
persistent Frame consults `mappings[dst_vm]` — auto-remap if recorded,
otherwise stays unmapped (callee can MAP at a fresh address).
This generalises the legacy IPC-slot-only auto-remap-on-REPLY behaviour
to a uniform per-cap rule that applies to every cross-frame movement
and supports caps that visit multiple VMs over their lifetime, each
recording their own mapping.
CallFrame.ipc_was_mapped retires entirely — REPLY's auto-remap falls
out of the per-cap rule. CallFrame keeps `ipc_cap_idx` to know which
slot to return the IPC cap to.
Helpers `cross_frame_unmap` / `cross_frame_remap` on InvocationKernel
encapsulate the unmap-then-mmap dance and the borrow-checker shuffle
between cap-table mutation and window-pool lookup.
`VmId::ROOT` const added so init code can record VM 0's mapping
before the arena slot is materialized.
map_pages / map / unmap_all signatures shift to take VmId. Internal
tests updated.
Phase 6+8 of the per-invocation ephemeral-table refactor — the bottleneck commit that couples ephemeral-table allocation, the slot-0 redirect rule, and IPC-mechanism retirement into one coordinated change. - `InvocationKernel::new_inner` allocates one `CapTable<P>` in `ephemeral_arena` per invocation; slot 0 of VM 0's persistent Frame receives a `Cap::EphemeralTable` handle to it. - `handle_call_code` (CREATE) pre-populates slot 0 of the child VM's cap-table with the same `Cap::EphemeralTable` handle. Bitmask bit 0 is no longer copyable from the caller (slot 0 is kernel-managed). - `resolve_cap_ref` returns `Option<(FrameRef, u8)>` where `FrameRef = Vm(u16) | Ephemeral(EphemeralTableId)`. Indirection walks unify through a single `cross_through(frame, slot)` helper that crosses `Cap::Handle` to a VM frame and `Cap::EphemeralTable` to the ephemeral table. - New slot-0 redirect rule: when the target byte is 0 and the cap-ref is non-zero, descend through slot 0 of the current frame and shift right one byte. Recurses, supporting up to 3 descents at 32-bit. `cap_ref == 0` exactly resolves to slot 0 of the active frame literally (the EphemeralTable handle); `ecalli 0` = REPLY remains the upstream special case. - `ecall_*` helpers (map / unmap / split / drop / move / copy / downgrade / set_max_gas) take `FrameRef` instead of `(usize, u8)`. MAP/UNMAP reject ephemeral frames (no associated window). MOVE cross-frame auto-unmaps and auto-remaps only when destination is a VM frame, preserving `mappings` when moving into the ephemeral table. - `handle_call_vm` / `handle_reply` / `handle_vm_halt` retire the φ[12]=IPC-cap-slot ABI. `CallFrame.ipc_cap_idx` retires; replaced by `prev_kernel_slots: [Option<Cap<P>>; 3]` which stashes the caller's view of ephemeral sub-slots 0/1/2 (Reply / Caller / Self) for restore on REPLY/HALT. javm doesn't write content into those slots — jar-kernel will populate Caller/Self in Phase 10. - Manifest convention shift: `IPC_SLOT` (0) retires; new constants `EPHEMERAL_TABLE_SLOT = 0` (kernel-reserved) and `ARGS_CAP_INDEX = 1` (manifest convention for the args DATA cap). Transpiler emits args at cap_index=1; kernel scans for cap_index=1 at init. 3-node testnet converges on identical state root across slots 1..5. javm lib tests (162) and full workspace tests pass; clippy/fmt clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…nvention javm is a pure PVM executor — it has no notion of "input args." The old `InvocationKernel::new(blob, args, gas)` API plus the magic `ARGS_CAP_INDEX` sentinel was a layering violation: javm reaching into the manifest to write bytes into a transpiler-defined slot, then setting φ[8]/φ[9] = args_base/args_len so the guest could find them. Under the new spec the CALL ABI has no register-passed cap-arg (scalars φ[7..12] still flow through, bulk byte data goes through caps placed at conventional slots — typically ephemeral sub-slot 4). The outermost-invocation case is no different: hosts populate a DATA cap and pass the byte address however they want. - `InvocationKernel::new` / `new_cached` / `new_with_backend` / `new_warm` lose the `&[u8]` args parameter. - `new_inner` drops the args-cap-find / `write_init_data` / `vm0.set_reg(8/9, ...)` block. VM 0's registers start zeroed; the host sets whatever scalars it wants after construction. - New `kernel.write_data_cap_init(slot, bytes) -> Result<u64, _>` — writes bytes into the DATA cap at `slot` of VM 0's persistent Frame and returns the mapped byte address (so the host can pass it to the guest in a register if it wants a raw pointer). Errors on empty/non-DATA slot, missing VM 0 mapping, or bytes-too-large. - `ARGS_CAP_INDEX` retires from `javm::cap`. javm-transpiler gains its own `pub const ARGS_CAP_INDEX: u8 = 69` (above HostCall range 1..=21 and the standard 64..=68 program caps), used by the transpiler-emitted manifest. Hosts that target transpiler-emitted blobs use the same constant to populate args via `write_data_cap_init`. - jar-kernel's `transact` / `dispatch` callsites drop the `payload` argument with a TODO — current halt-immediate fixtures don't read it; when real guests do, they'll wire payload bytes through the same `write_data_cap_init` path against a manifest-reserved cap. - javm-guest-tests/tests/guest.rs: post-init, calls `write_data_cap_init(ARGS_CAP_INDEX, input)` and sets φ[8]/φ[9] manually. Adds `javm-transpiler` as a dev-dependency. - ~/docs/minimum/08-pvm.md: stale "args cap with cap_index=0x00" mentions in §"Program Blob Format" and §"Program Init" updated to describe the new host-driven `write_data_cap_init` flow. State root differs from the previous commit (args bytes are no longer auto-written) but is identical across all 3 testnet nodes. Workspace tests, clippy, fmt all clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…hemeral caps
Phase 10 of the per-invocation ephemeral-table refactor. The three
read-only host calls (`Gas`, `SelfId`, `Caller`) retire from
jar-kernel's host-call dispatcher; their data is now placed in the
ephemeral table at the kernel-reserved sub-slots agreed in 08-pvm.md
(sub-slot 1 = Caller, sub-slot 2 = Self, sub-slot 3 = Gas), where
guests will eventually read it via cap-ref.
- `Capability` enum gains 4 variants with their per-variant structs:
- `Gas(GasCap { remaining: u64 })` — ephemeral sub-slot 3
- `SelfId(SelfCap { vault_id, code_hash })` — ephemeral sub-slot 2
- `CallerVault(CallerVaultCap { vault_id })` — ephemeral sub-slot 1
when the caller is another Vault VM
- `CallerKernel(CallerKernelCap { role: KernelRole })` — sub-slot 1
for kernel-fired top-level invocations
- `KernelCap` gains `gas_derive` / `gas_merge` impls of the
`ProtocolCapT` hooks. The `Cap(Capability::Gas(_))` arm splits /
merges; everything else returns `None` / `false`. Routes through
javm's `MGMT_GAS_DERIVE` / `MGMT_GAS_MERGE` ecallis (already
wired in commit 248770c).
- `HostCall::Gas / SelfId / Caller` (slots 1/2/3) and their
dispatcher arms retire. `from_slot` and `populate_host_call_slots`
drop the three ids; the iteration starts at `HostCall::StorageRead`
(slot 4). The `encode_caller` helper retires.
- New `populate_ephemeral_kernel_caps` writes Caller/Self/Gas caps
into ephemeral sub-slots 1/2/3 at invocation entry. Wired into
`transact::run_one_invocation` (TransactEntry role) and
`dispatch::run` step-2 / step-3 (AggregateStandalone /
AggregateMerge). Sub-slot 0 (Reply Handle) stays empty for root —
javm's CALL/REPLY machinery rewrites all three sub-slots on every
internal CALL.
Today's halt-immediate kernel fixtures don't read these caps, so the
state root is unchanged. When real guests start reading
Caller/Self/Gas via cap-ref into ephemeral sub-slots, that cap data
will be the source of truth.
3-node testnet converges identically across slots 1..5; workspace
tests, clippy, fmt all clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ollback Phase 9 minimum: replace per-call gas ceiling (max_gas + SET_MAX_GAS) with the shared-pool model the spec describes. Per-call gas restriction is now achieved by the **park pattern** via MGMT_GAS_DERIVE / _MERGE on the `Capability::Gas` cap at ephemeral sub-slot 3 (host-policy on top of javm — not exercised by javm-only tests). - `HandleCap.max_gas` and `CallableCap.max_gas` retire entirely. CALL on HANDLE/CALLABLE no longer consults a ceiling. - `MGMT_SET_MAX_GAS` (op 0xB) constant retired with reserved-gap comment. Both the `dispatch_ecall` 0x0B arm and the `mgmt_*` legacy-encoding arm retire. `ecall_set_max_gas` and `mgmt_set_max_gas` helpers retire. - `handle_call_vm` and `handle_resume`: drop the `max_gas: Option<u64>` parameter. Charge call_overhead (10), then transfer the caller's full residual gas to the callee — the shared-pool model. On REPLY / HALT the callee's residual returns to the caller (existing path, unchanged). - New `rollback_parked_gas(vm_idx)`: on child fault, scan the resuming parent's persistent Frame for `Cap::Protocol(P)` payloads and call `gas_merge` against the live ephemeral Gas cap. Successful merges (Gas-shaped donors) drop the donor; non-Gas payloads are restored. This is the kernel default — guarantees the parent recovers any gas it parked via GAS_DERIVE before a failed CALL, regardless of what the (possibly-untrusted) child did. - Internal test `test_kernel_gas_bounding` rewritten as `test_kernel_call_transfers_full_gas`: under shared-pool, CALL transfers caller_gas - 20 (10 ecalli charge + 10 call overhead) to the callee. 3-node testnet converges identically across slots 1..5 (state root unchanged — halt-immediate guests don't exercise the gas path). Workspace tests (162+ in javm, full suite green), clippy/fmt clean. The recompiler/interpreter JIT-pointer-caching to `Capability::Gas` remains as a future optimization. Today the JIT still drives `VmInstance.gas` (now per-VM rather than per-invocation in name, but the kernel transfers full residual at every CALL/REPLY so the behaviour is equivalent to a single shared counter). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The CALL-on-slot-0 = REPLY shorthand isn't about IPC anymore — slot 0 holds the EphemeralTable handle since Phase 6+8. Update the doc-comment to describe the actual semantics. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
Genesis ReviewComparison targets:
How to reviewPost a comment with the following format (rank from best to worst): Use the short commit hashes above and To meta-review another reviewer's comment, react with 👍 or 👎. |
javm's MGMT_MOVE / MGMT_COPY / MGMT_DROP ecallis (with cap-ref
indirection) previously crossed through Cap::Handle (→ a sub-VM's
Frame) and Cap::EphemeralTable (→ the per-invocation table) only.
This adds a third crossing target: a host-managed cap-table outside
javm — for jar-kernel, a σ-resident Vault CNode — addressable via
any Cap::Protocol whose ProtocolCapT::as_foreign_frame() reports
one. Slot 1 of every VM's persistent Frame is now an ephemeral
VaultRef into the home Vault, so a guest cap-ref like 0x000100AA
reaches home_vault.slots[0xAA] uniformly.
javm:
- ProtocolCapT gains ForeignFrameId / FinalStepRights assoc types
and an as_foreign_frame() default-None hook.
- FrameRef<F=()> generalises to carry a foreign id; cross_through
recognises foreign-frame caps on its fall-through arm and
resolve_cap_ref threads final-step rights through the walk.
- New ForeignCnode<P> trait carries fc_take / fc_set / fc_clone /
fc_drop / fc_is_empty for slot operations javm cannot implement
itself; NoForeignCnode ZST is the default for hosts with no
foreign frames.
- run() splits into a no-host shim and run_with_host<H>; ecall_move
/ copy / drop / split / map / unmap / downgrade route through
helpers (frame_take / set / clone / is_empty) that dispatch to
the host adapter when the frame is Foreign.
jar-kernel:
- KernelCap splits into HostCall(u8) | Ephemeral(Capability) |
Registered { id, cap }. Registered preserves CapId across
Frame ↔ Vault round-trips so cap_holders / cap_children stay
consistent; Ephemeral covers kernel-injected per-frame markers
(Gas / SelfId / CallerVault / CallerKernel) plus the slot-1
home VaultRef.
- VaultRights gains read for traversal-vs-final-step rights
composition: every intermediate VaultRef in a chained walk must
have read; the operation-specific right (grant / revoke /
derive) is checked at the final step inside the host adapter.
- New VaultCnodeView<'a> in vm/foreign_cnode.rs implements
ForeignCnode<KernelCap> over &'a mut State, routing through
cap_registry / cnode / pinning helpers and respecting the
rights bag.
- populate_home_vault_ref places the slot-1 home VaultRef at
every VM init in transact.rs and dispatch.rs (step-2 + step-3).
- drive_invocation builds a fresh VaultCnodeView per loop
iteration and threads it into vm.run_with_host, so σ mutations
stay inside the existing StateSnapshot rollback scope.
Tests: 13 new tests in tests/cnode_move.rs cover fc_take / set /
clone / drop / is_empty plus rights enforcement and pinning
rejection. End-to-end guest-driven coverage of the cap-ref
crossing through cross_through is deferred until a transpiler
helper for dynamic ecallis lands.
Spec: docs/minimum/01-capabilities.md grows a "three frame kinds,
one MGMT surface" section + rights composition table; 08-pvm.md's
cross_through pseudocode now shows the VaultRef arm.
Verified: cargo test --workspace passes (162 javm + 47 jar-kernel
+ 13 cnode_move + others); cargo clippy clean; cargo fmt clean;
3-node testnet converges; cargo bench shows the cnode-cross PR
is performance-clean within fresh-rebuild noise (-6% to +2%
across all benches).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wire op 16 (HOST_YIELD) through the kernel-known host-call dispatch
range. `dispatch_host_yield`:
1. Reads the marker slot index from φ[7]; resolves the
Cap::Instance marker from the running entry's cnode.
2. Walks the call stack top→bottom looking for an InstanceEntry
whose Image.yield_marker_slot holds a Cap::Instance[YieldCatcher]
whose marker list (via KernelAssist) contains the thrown marker's
image_hash_chain.
3. On match, pushes a ReferenceEntry pointing at the catcher's
position and exits with HostCall(16); on no match, traps.
vm.rs:
- Extract the post-Interpreter translation into `drive_and_translate`,
which detects yields by a structural side-effect (stack grew above
the originally-pushed position with a ReferenceEntry on top) and
surfaces CallResult::Paused with the marker cap as marker_payload.
- Implement `call_resume`: pop the top ReferenceEntry, take the
resumed Instance's saved regs/mem/gas, re-enter the interpreter via
`drive_and_translate`. SlotPath argument retained for spec
compatibility but ignored for Stage 3 (Paused is on-stack, not
σ-resident — Stage 4 will route by path).
- Optional scratchpad reflects into slot[0] before re-entering.
callstack.rs adds entries_mut() for save-by-position semantics: the
driver needs to write the yielder's live regs/mem/gas back into the
InstanceEntry that's no longer at the top.
Tests:
- run_instance_yields_when_marker_caught_by_outer — verifies the
routed Paused with the expected marker payload + stack shape.
- call_resume_after_yield_runs_to_halt — full yield → resume → Halt
round-trip with a 2-frame [outer, inner] setup.
- run_instance_unhandled_marker_faults — no catcher on the stack →
Trap, stack restored to caller-only.
50 tests pass (47 → 50). clippy clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wire ops 17–21 in the kernel-known host-call range: - SET_IMAGE (op 17): read Cap::Image at φ[7]; extend the running InstanceCap's image_hash_chain via blake2b256(prev || image_hash); look up the full Image bytes via KernelAssist::image_lookup; reload InstanceEntry.image + program via ImageCache. content_hash is left untouched for Stage 3 — Stage 4 defines the canonical state digest. - DERIVE_SPAWN (op 18): read Cap::Image at φ[7]; chain_extend the spawner's image_hash_chain with the image's content_hash; place the resulting Cap::Instance at φ[8]. content_hash is a Stage 4 placeholder (zero). - MAKE_IMAGE (op 19): stub returning Trap. The in-memory parts encoding for host_make_image is not yet spec-final; jar-kernel-v3 will land the proper memory-pointer-based variant when chain Images need runtime construction. - HOST_SAME_TYPE (op 20): compare image_hash_chain of two Cap::Instance / Cap::Type values at φ[7], φ[8]; result (0/1) in φ[7]. - HOST_TYPE_OF (op 21): read Cap::Instance/Cap::Type at φ[7]; mint Cap::Type carrying the same image_hash_chain at φ[8]. kernel_assist.rs adds an `image_lookup(content_hash) -> Option<Arc<Image>>` trait method (default returns None) and a HashMap-backed implementation in InProcessKernelAssist (with `register_image` for tests). Stage 4 SigmaKernelAssist will back this against State.code_blobs. A new free function `type_chain_at` extracts the image_hash_chain from a slot holding either Cap::Instance or Cap::Type — used by HOST_SAME_TYPE. Tests: set_image lookup, set_image-without-registry trap, derive_spawn chain extension, host_same_type both branches, host_type_of mint, make_image trap. 56 tests pass (50 → 56). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two new host calls and the HALT-time mapped-region write-back: - HOST_READ_DATA_CAP (op 22): read the Cap::Data at φ[7]; look up its bytes via KernelAssist::data_lookup; copy min(len, size) into mapped memory at φ[8] (padding with zeros from the canonical-form representation up to declared size). - HOST_MINT_DATA_CAP (op 23): read bytes from mem[φ[7]..len]; strip trailing zeros (canonical form §2); content-hash via KernelAssist::data_store; debit the StorageQuota at φ[9] (read quota_id from the cap's content_hash); mint Cap::Data; place at φ[10]. Insufficient quota → Trap. - HALT-time write-back (vm.rs::writeback_persistent_mappings): walks the Image's memory_mappings; for each Persistent mapping, re-hashes the live mem span (after canonical-form strip) and installs a fresh Cap::Data at the mapping's root-cnode slot. Pinned slots are skipped. Nested SlotPaths deferred. O(N) full re-hash per mapping for Stage 3; the page-BMT incremental variant is Stage 7 per the plan. kernel_assist.rs adds `data_lookup(content_hash) -> Option<Vec<u8>>` and `data_store(bytes) -> CapHash` trait methods (default impls compute the hash but don't persist; InProcessKernelAssist backs both with a HashMap). `register_data` helper for tests. A new `strip_trailing_zeros_len` helper sits in ecall.rs and is re-used by both host_mint_data_cap and the write-back path. Tests: host_read_data_cap copy; host_mint_data_cap round-trip + quota debit; quota-exhaust Trap; strip_trailing_zeros corners; writeback_rehashes_persistent_mapping_at_halt. 61 tests pass (56 → 61). clippy clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
run_instance now seeds gas from the KernelAssist GasMeter table:
when image.gas_slots[0] is set, reads the Cap::Instance at that
slot, decodes meter_id from the first 8 bytes of its content_hash,
and pulls the initial gas budget via kernel_assist.gas_meter_get
(topping up with gas_budget if the meter is empty so callers can
treat gas_budget as the "ambient budget"). When gas_slots is empty,
falls back to gas_budget directly (Stage 3.10 semantics).
On ExitReason::OutOfGas:
- reconcile the meter to 0 (mirroring the local counter)
- walk the call stack for a YieldCatcher whose marker list contains
the well-known OogMarker image hash
- on match: push a ReferenceEntry at the catcher's position;
surface CallResult::Paused with marker_payload = the Gas{meter_id}
cap (caller-handler reads this to know which meter to top up,
per the lazy-load OOG-catch pattern documented in
kernel-assisted-instances.md)
- on no match: fall through to CallResult::Faulted as before.
The chain-orchestrator-driven topup-and-CALL_RESUME loop is a
Stage 4 (jar-kernel-v3) concern; this commit gives the chain
orchestrator the necessary structural piece (Paused + Gas payload)
to drive that loop.
Tests: run_instance_oog_with_outer_catcher_yields_with_gas_marker
verifies the full path (Gas{42} cap as payload, ReferenceEntry on
top of the stack). The existing run_instance_oog_returns_faulted
covers the no-catcher branch. 62 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two new kernel-known host calls and three new KernelImage tags: - HOST_OPEN (op 24): read the FileCap at φ[7] (Cap::Instance whose low 8 content_hash bytes encode the file_id); call KernelAssist::host_open(file_id) to materialize the bytes as a Cap::Data; place at φ[8]. - HOST_SAVE (op 25): mint a fresh FileCap from the Cap::Data at φ[7], debiting StorageQuota at φ[8]; place result at φ[9]. The new FileCap is a Cap::Instance carrying KernelImage::File's image_hash_chain and the new file_id in its content_hash. KernelAssist trait extensions: - host_open(file_id) -> Option<DataCap> (default None) - host_save(data, quota_id) -> Option<u64> (default None) KernelImage enum adds three variants: - File — kernel:file image hash for FileCap - HostOpen / HostSave — kernel-issued caps that the kernel injects into chain Instance cnodes at genesis (Stage 4 will install). InProcessKernelAssist: - HashMap<u64, DataCap> file registry, monotonic next_file_id. - register_file(file_id, data) helper for fixtures. - host_save enforces storage quota; returns None on exhaustion (caller traps). Tests: - host_open_materializes_registered_file_as_data - host_save_mints_file_after_quota_debit (round-trips back via host_open and asserts content_hash + size preserved) 64 tests pass (62 → 64). clippy clean. Stage A complete: full v3 javm host-call surface lands. Next: Stage B demolition. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Vendor the JAR blob format module (program.rs) and the small Access enum / MGMT_MAP constant into javm-transpiler so the transpiler no longer depends on javm-legacy. These are pure data-format definitions; jar-kernel-v3 (Stage D) will parse the same blob format via this vendored module. The two v2-kernel-based tests in assembler/emitter — which exercised the InvocationKernel runner — are replaced with format-level round-trip checks (parse_blob over the emitted bytes). The equivalent run-end-to-end coverage lands at the Stage D integration test once jar-kernel-v3 can consume the blob. 57 tests pass (transpiler crate). clippy clean. Workspace clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ps (B.2) Remove tests/guest.rs — the three-way comparison (host vs v2 interpreter vs v2 recompiler) drove its loaded blob through javm-legacy::InvocationKernel and isn't portable to v3 without a substantial new test harness (v3 doesn't yet have a single-blob loader for these guest vectors; the equivalent conformance lands in javm-exec's own interpreter tests + the Stage D simple-chain end-to-end test). The library `javm_guest_tests::dispatch_to_vec` is preserved as a test-vector library that future v3 conformance tests can run against. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
javm-bench was the v2 PVM benchmark harness, comparing the v2 interpreter, v2 recompiler, and polkavm. Its 2.2KLoC library is deeply intertwined with javm-legacy (35 references in src/lib.rs alone, 14 in pvm_bench.rs, etc) and surgically stripping the v2 rows would require rewriting most of the harness. Recent commits (C1–C3) added javm-exec interpreter + recompiler rows alongside the v2 ones. Those will be reconstructed as a clean-slate javm-exec-bench crate (or moved inside javm-exec itself) post-Stage-D, once the v3 stack is end-to-end working. This unblocks the demolition of javm-legacy without porting a substantial bench harness. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both crates depend entirely on v2 jar-kernel's public API (Kernel::new/dispatch/advance, InMemoryHardware, InMemoryBus, AdvanceOutcome). jar-harness is the multi-node PoA test harness; jar is its CLI driver. Neither has a v3 equivalent yet — v3 currently has no multi-node consensus to drive. A future stage can resurrect a v3 multi-node harness once jar-kernel-v3 has block-apply working and a consensus layer is in scope. Workspace builds clean after the deletion: only v2 jar-kernel and v2 javm-legacy remain as v2 holdouts, to be deleted in B.5. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
End of Stage B demolition. With B.1–B.4 having removed all
production-code dependencies on javm-legacy and the consumer
crates (jar-harness, jar testnet bin, javm-bench), nothing in the
workspace references either crate.
Workspace `Cargo.toml` drops both members + workspace deps.
After this commit, only v3 crates remain under rust/:
jar-cap, javm-exec, javm, javm-transpiler, javm-guest-tests,
javm-builtins, scale, scale-derive, build-{crate,javm,pvm},
jar-test-services/halt.
The `jar-kernel` slot is empty, ready to be filled by Stage C as
the v3 jar-kernel.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fresh `rust/jar-kernel/` crate replacing the deleted v2 one. Module layout (filled by C.2 onward): - abi.rs — slot numbers + op-codes - state.rs — σ types + state_root (C.2) - kernel_assist.rs — SigmaKernelAssist (C.3) - kernel_assisted/ — native dispatch (C.4) - genesis.rs — chain init (C.5) - apply.rs — block apply (C.6) - error.rs — KernelError (this commit) Stage C.1 ships: - Cargo.toml with deps on jar-cap + javm-exec + javm + scale + thiserror. - lib.rs module declarations. - abi.rs constants for kernel-issued slots (Stage C.5 will populate). - error.rs `KernelError` enum. Workspace updated: `rust/jar-kernel` re-added to `members` + `workspace.dependencies`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Canonical σ shape:
- data_blobs: BTreeMap<FileId, DataBlob{content_hash, size, refcount,
backing_quota}> — content-addressed σ-resident blobs.
- code_blobs: BTreeMap<CodeId, Vec<u8>> — Image bytecode keyed by hash.
- vaults: BTreeMap<VaultId, VaultRecord{image_hash_chain, content_hash}>
— chain-resident Instances. VaultRecord shadows jar_cap::InstanceCap
so jar-kernel can derive SCALE Encode/Decode without modifying
jar-cap.
- validators: Vec<[u8; 32]> — PoA validator pubkeys (placeholder).
- counters: IdCounters — monotonic file_id / code_id / vault_id.
state_root = blake2b256(SCALE_encode(σ)). Simple raw hashing per
user direction; per-registry composed BMT is a Stage 6 future
optimization.
Determinism: all BTreeMap encodings iterate in sorted-key order,
so insertion-order differences produce identical hashes (verified
in test).
Tests cover: empty-state determinism; SCALE round-trip via the
crate's derive macros; state_root changes on insertion; order
invariance under BTreeMap; IdCounters monotonic allocation.
5 tests pass. clippy clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
σ-aware KernelAssist impl. Holds a &mut State borrow plus per-block
ephemeral tables (gas_meters, storage_quotas, yield_catchers).
Trait method bindings:
- gas_meter_*, storage_quota_*, yield_catcher_* — backed by
in-memory HashMaps reset via reset_block_state() at block start.
- data_lookup / data_store — σ.data_payloads (content-addressed
byte registry). data_store inserts on first sighting; reads are
free of side-effects.
- host_open — σ.data_blobs[file_id] → DataCap{content_hash, size}.
- host_save — debits storage quota, allocates next file_id,
σ.data_blobs[file_id] = DataBlob with refcount=1 and a
backing_quota tag for Stage C.7 refund-on-drop accounting.
- image_lookup — None (simple-chain demo's chain Image is
installed at genesis via cnode slots, not via runtime lookup).
state.rs gains `data_payloads: BTreeMap<CapHash, Vec<u8>>` so
content-addressed bytes can be deduped across multiple FileBlobs
sharing the same content_hash.
Helpers: seed_root_gas / seed_root_quota for block-start setup;
reset_block_state for the canonical "kernel is stateless across
blocks" invariant from architecture.md.
10 tests pass (5 new for SigmaKernelAssist). clippy clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`genesis(chain_image)` constructs the v3 chain bootstrap:
- σ with chain image registered in code_blobs[0].
- Chain InstanceCap whose image_hash_chain is the image's content
hash (genesis case — no prior chain).
- Chain root cnode (256 slots) populated with kernel-issued caps
at the abi::BARE_* slots:
GAS_SLOT, QUOTA_SLOT (unit handles for Gas{0} / Quota{0}),
YIELD_CATCHER_SLOT,
SET_GAS_METER / SET_STORAGE_QUOTA factories,
MINT_GAS / MINT_QUOTA factories,
CREATE_YIELD_CATCHER factory,
HOST_OPEN / HOST_SAVE entry handles.
Kernel-issued caps are Cap::Instance values; image_hash_chain
matches the relevant KernelImage variant (so javm's
recognize_kernel_image short-circuits identify them). Unit
handles (Gas{0}, Quota{0}) carry the id in their content_hash's
low 8 bytes — matching the convention used by javm's gas seeding
and host_save quota lookup.
Native dispatch (Stage C.4 in the plan) is deferred — simple-chain
exercises the kernel-assist methods directly via SigmaKernelAssist
rather than CALLing into kernel-known Instances. Native dispatch
becomes valuable when the chain orchestrator drives lazy-load OOG
topup via SetGasMeter CALL — a future stage.
13 tests pass (3 new for genesis). clippy clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`Block { events: Vec<Event> }`; each `Event { endpoint_idx, payload }`.
apply_event:
1. Build a fresh cnode (factory closure runs genesis cap injection —
kernel-issued caps live in cnode but are transient; persistent
state lives in σ).
2. Reflect the event payload as a Cap::Data at slot[0]; register
bytes in σ.data_payloads so host_read_data_cap can resolve them.
3. SigmaKernelAssist over &mut State; reset block ephemeral tables;
seed root gas + storage quota.
4. Vm::run_instance over the chain Image / Instance / cnode.
5. Translate to EventOutcome { Halt | Faulted | Paused }.
Mutations to σ (host_save, data_store) already landed via the
&mut State borrow.
Kernel::from_genesis takes a chain Image; captures it in an
Arc<Image> plus a cnode_factory closure (re-runs genesis cap
injection per event). Kernel::apply delegates to apply_block;
Kernel::state / state_root expose the σ surface.
Tests:
- kernel_from_genesis_yields_deterministic_state_root
- kernel_apply_advances_state_root_via_host_save (payload data_store
pushes a fresh entry into σ.data_payloads → root changes)
- kernel_apply_replay_is_deterministic (same image + block → same
post-apply root)
16 tests pass (13 → 16). clippy clean. Stage C public surface:
Kernel + Block + Event + EventOutcome.
Stage C.7 (refcount + refund) deferred — simple-chain demo runs
without it. Stage 4-future hardening lands once a chain exercises
the host_save/drop cycle.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The v2 simple-chain — a Rust→javm guest implementing ed25519 balance transfers — relied on v2 ABI constants (Vault.initialize, MintAttestCap, SetScore, BareFrame layout, MGMT_MAP semantics) all absent from v3. A v3 Rust→javm pipeline that emits chain Image blobs is future work. This commit drops the v2 source and reduces simple-chain to a host- only no-op stub. The runnable v3 chain apply path is exercised by: - `jar_kernel::kernel::tests` — Kernel::from_genesis + apply round-trip (already landed in C.6+C.8). - A dedicated end-to-end integration test landing in D.2. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`rust/jar-kernel/tests/end_to_end.rs` exercises the full v3 stack: jar-cap → javm-exec → javm → jar-kernel. The chain Image is hand- authored byte-PVM (a minimal `ecalli 0` HALT program) — a Rust→javm guest pipeline that emits chain Image blobs is future work. Tests: - genesis_yields_stable_state_root_for_identical_images — same image → same genesis root. - single_event_apply_halts_and_advances_state_root — Kernel::apply with one event observes Halt and the state-root advances because the event payload bytes land in σ.data_payloads via data_store. - multi_block_apply_advances_state_root_each_step — two-block sequence; each block produces a distinct root. - identical_apply_sequences_produce_identical_state_roots — two independent Kernel instances applying the same image + blocks end up with the same state-root (deterministic replay). - distinct_payloads_produce_distinct_state_roots — two kernels with different payloads diverge. 5 tests pass in tests/end_to_end.rs. The full workspace test run is clean across all crates (jar-cap, javm-exec, javm, javm-transpiler, jar-kernel). clippy clean. fmt clean. End of Stage D: the v3 stack runs end-to-end with deterministic state-root evolution under Kernel::apply. The simple-chain demo (host_open/host_save balance transfers) needs the v3 Rust→javm guest pipeline that emits chain Image blobs — future work; the hand-authored byte-PVM chain in this test is sufficient to validate the integration layer. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the 86-line hand-rolled `image_canonical_encoding` with `scale-derive`'s `Encode`/`Decode` impls. `image_content_hash` is now `H::hash(&image.encode())` — the canonical wire format is the SCALE encoding, full stop. Field type changes: - `endpoints: [Option<EndpointDef>; 256]` → `BTreeMap<u8, EndpointDef>`. The fixed array hit the orphan rule (can't impl foreign trait for foreign type [T; N]). BTreeMap is also a better fit for sparse endpoint tables: only declared endpoints land on the wire. - `MappingSource` variant order swapped (Ephemeral first, Persistent second) so the SCALE discriminator matches the old canonical byte assignment (0 = Ephemeral, 1 = Persistent). Downstream call sites adjusted: - Empty-image fixtures in javm + jar-kernel use `BTreeMap::new()`. - vm.rs endpoint lookup is now `endpoints.get(&endpoint_idx)`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The crate's role is "guest-side runtime support for JAVM chain
Images" — the new name `subsoil` makes that clear (the soil layer
the guest's `main` plants in).
- `git mv rust/javm-builtins rust/subsoil`; package renamed.
- `javm_entry!` macro renamed to `subsoil::entry!`.
- Workspace gains `subsoil = { path = "rust/subsoil" }` in
`workspace.dependencies`; consumers depend via `{ workspace = true }`.
- Bench guests (blake2b, ecrecover, ed25519, keccak, prime-sieve)
and javm-guest-tests updated.
- Transpiler doc-comment refs to `javm_builtins::map_args`
retargeted to `subsoil::map_args`.
No behavioral change. polkavm cfg branches still present; they go
away in Stage 3 along with `build-pvm`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`build-pvm` was the build-helper feeding the PolkaVM-comparison rows of `javm-bench`. `javm-bench` was already deleted; this is the orphan. - `rust/build-pvm/` removed (Cargo.toml + lib.rs + target JSON). - Workspace `members` drops the entry. - Bench guests (blake2b, ecrecover, ed25519, keccak, prime-sieve) drop their `polkavm.rs` modules + `[target.'cfg(target_env = "polkavm")'.dependencies]` blocks + polkavm cfg lints. They are now JAVM-only. - `subsoil::entry!` macro already lost its polkavm arm in Stage 2; this commit drops the `polkavm` value from subsoil's `unexpected_cfgs` lint and the same in `javm-guest-tests`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New proc-macro crate `subsoil-derive` exposes `#[subsoil::endpoint(N)]`. Applied to a function, it leaves the body intact and emits a `subsoil::EndpointDescriptor` static into the `.subsoil.endpoints` ELF section (under cfg(all(target_env = "javm", target_os = "none")) — host-side, the attribute is a no-op apart from forwarding the original function). The transpiler will read this section in Stage 5 to populate the chain Image's `endpoints: BTreeMap<u8, EndpointDef>`. `subsoil::EndpointDescriptor` is a `#[repr(C)]` 16-byte record: - fn_ptr (8 bytes): RISC-V address of the endpoint function. - index (1 byte): u8 endpoint index. - arg_registers (1 byte): caller's register-arg count. - arg_cnode_size (1 byte): caller's arg-cnode size. - _pad (5 bytes): reserved. Includes a host-side smoke test confirming the attribute preserves function bodies. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`link_elf` returns `jar_cap::image::Image` (not raw bytes). The JAR v1 magic-prefixed wrapper format is retired; the new outer container is the SCALE-encoded Image. `build-javm` writes `image.encode()` to the `.pvm` file. Image population: - `code`: CODE sub-blob (jump_table + code + packed bitmask) of the translated user code, built via a new `emitter::build_image_code_blob` helper. - `endpoints`: read from the `.subsoil.endpoints` ELF section (entries emitted by `#[subsoil::endpoint(N)]`); resolved RISC-V fn_ptrs → PVM PCs via `ctx.address_map`. Fallback to a single PC-0 entrypoint when the section is absent (preserves `subsoil::entry!` backward compat). - `memory_mappings`: empty for now. Declarative mappings (porting the ro/rw/heap data-cap layer from the legacy JAR v1 manifest into `Image::memory_mappings` + `pinned_slots`) is a future refactor — the runtime prologue is no longer emitted, so transpiled bench guests can't access ro/rw data until that refactor lands. The bench guests aren't currently executed through the v3 kernel, so this is not a regression. - `gas_slots`, `quota_slots`, `yield_marker_slot`: standard kernel-ABI defaults from `jar_cap::abi` (newly extracted from `jar_kernel::abi` so the transpiler and kernel share the same constants). `rewrite_data_code_ptrs` and the parsed ro/rw/stack/heap fields in `LinkedElf` are kept under `#[allow(dead_code)]` pending the declarative-mapping follow-up. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Each bench guest (blake2b, ecrecover, ed25519, keccak,
prime-sieve) now declares its entry function with
`#[subsoil::endpoint(0)]` in addition to the existing
`subsoil::entry!(javm_main)` that still provides `_start`. Net
effect: the transpiler's `.subsoil.endpoints` reader picks up the
function's RISC-V address, resolves it to a PVM PC, and emits
`Image.endpoints[0] = EndpointDef { entry_pc, .. }`.
Signature changes: `extern "C" fn() -> u32` →
`fn(_args_len: u64) -> u64`, with `as u64` cast on the return. The
`fn(u64) -> u64` shape matches `subsoil::EndpointDescriptor.fn_ptr`
so the descriptor static type-checks. The bench bodies themselves
ignore `args_len` (they compute deterministic values, no caller
input).
Note: the bench guests aren't currently exercised through
`jar-kernel`'s apply path. They build into Image blobs that
demonstrate the end-to-end pipeline (Rust → ELF → transpile →
SCALE-encoded Image with populated endpoints). Running them via
the v3 kernel requires the declarative memory-mapping refactor
(future work) so the bench programs can access their ro/rw data.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`EndpointDef` gains `initial_regs: BTreeMap<u8, u64>` — a sparse per-endpoint register-seeding map. The kernel applies these on top of the calling-convention default (φ[11] = endpoint_idx) when constructing the endpoint's `Regs`. `Copy` removed from `EndpointDef` (BTreeMap precludes it). No callers relied on `EndpointDef: Copy`. `Vm::run_instance` ([rust/javm/src/vm.rs]): - Captures the `EndpointDef` reference once. - After seeding gpr[11] = endpoint_idx, iterates initial_regs and writes `regs.gpr[k] = v` (with `.get_mut(...)` bounds check so a malformed Image can't index out of range). `javm-transpiler::link_elf` ([rust/javm-transpiler/src/linker.rs]): - After endpoint resolution, computes `ProgramLayout` from the parsed ELF (stack/ro/rw/heap page counts). - Sets `initial_regs[1] = stack_top` (PVM φ[1] = RISC-V SP) on every endpoint. This replaces the load_imm_64 SP step the emitted prologue used to do at runtime. Hand-authored Image fixtures updated for the new field: - `rust/jar-cap/src/image.rs#tests` (image_scale_roundtrip, endpoints_affect_hash) - `rust/jar-kernel/src/kernel.rs#tests` (minimal_chain_image) - `rust/jar-kernel/tests/end_to_end.rs` (hello_world_chain_image) The dead prologue machinery (emit_prologue, build_service_program, build_trivial_authorizer) is removed in Stage 2. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…h (Stage 2) After Stage 1 moved SP seeding into `EndpointDef.initial_regs`, the transpiler's runtime prologue and the legacy v2 JAR-manifest blob format are unreachable. This commit removes them. Removed: - `emit_prologue`, `emit_mgmt_map`, `emit_load_imm_64`, `emit_save_arg_regs`, `emit_restore_arg_regs`, prologue opcode constants (`PVM_OPCODE_LOAD_IMM_64`, `PVM_OPCODE_ECALL`, `PVM_OPCODE_MOVE_REG`, `SP_REG`, scratch regs, MGMT_MAP arg-reg layout) — `rust/javm-transpiler/src/layout.rs`. - `build_service_program`, `build_standard_program`, `ProgramHeader`, `JAR_MAGIC` — `rust/javm-transpiler/src/emitter.rs`. - `Assembler::build`, `set_ro_data`, `set_rw_data`, `set_heap_pages`, `set_max_heap_pages`, `set_stack_pages`, the `ro_data/rw_data/heap_pages/max_heap_pages/stack_pages` fields, and the JAR-v1 builders `build_sample_service`, `build_sample_service_precise`, `build_trivial_authorizer` — `rust/javm-transpiler/src/assembler.rs`. The `Assembler` struct is now an opcode-encoder used by unit tests. - `pub const MGMT_MAP` in `program.rs` (prologue-only). - Prologue tests in `layout.rs` and JAR-magic tests in `assembler.rs`. Kept: - `ProgramLayout` + `DataCapEntry` + cap-index/page-size constants in `layout.rs` — `link_elf` uses `ProgramLayout::stack_top()` for the per-endpoint SP value, and the same layout will feed declarative `Image.memory_mappings` once the kernel honors them. - `emitter::build_image_code_blob` (the live path), `pack_bitmask`, `jump_table_entry_size`. - The `Assembler` opcode-emission helpers and their tests. `cargo build --workspace --all-targets`, `cargo test --workspace` (including the 5 jar-kernel end-to-end tests), `cargo clippy --workspace --all-targets -- -D warnings`, and `cargo fmt --all -- --check` all pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…initial_slots (Stage 1)
The chain Image's memory model collapses to a single principle:
every mapping points at a slot. The kernel resolves the slot to a
Cap::Data at instance start, lays the bytes at the declared
address range, and derives RO/RW from whether the slot is pinned.
Type changes (`rust/jar-cap/src/image.rs`):
- `MemoryMapping { start, size, source: MappingSource }`
→ `MemoryMapping { start, size, source: SlotPath }`.
`MappingSource` enum is deleted entirely. Stack/scratch is now a
regular non-pinned `Cap::Data` (zero-filled when content is
empty); the chain resets it before HALT by guest convention
rather than via a kernel-special Ephemeral case. Among other
benefits, the Paused snapshot needs only regs + PC — Mem state
is captured in the cnode caps.
- `PinnedCap::Data { content_hash, size }`
→ `PinnedCap::Data { content: Vec<u8>, size }`. Pinned data is
inlined into the Image so a chain spec is self-contained. A
future Image-compression pass can re-introduce a hash-only
variant pointing into σ.data_payloads.
- New `Image.initial_slots: BTreeMap<SlotIdx, InitialDataCap>`
declares non-pinned mutable cnode state for standalone (root)
Instance bootstrap. A parented Instance receives its cnode from
the spawner and ignores this field.
Downstream:
- `Vm::writeback_persistent_mappings` simplified: the `match
&mapping.source` is gone (every mapping has a SlotPath); the
rest of the logic (skip pinned + nested paths) is unchanged.
- Empty-Image fixtures across javm + jar-kernel gain
`initial_slots: BTreeMap::new()`.
- The one `MappingSource::Persistent(...)` literal in vm.rs's
HALT write-back test simplifies to a bare SlotPath.
No behavior change for hand-authored kernel-test fixtures (they
all set memory_mappings = empty + pinned_slots = empty +
initial_slots = empty). The 5 jar-kernel end-to-end tests still
pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`Mem::map_region(start, size, access, init)` reserves a
page-aligned region with per-page permissions and optional initial
bytes. The kernel-driven Vm::run_instance will use this in Stage 3
to materialise `Image.memory_mappings` before the interpreter loop.
- New `Access::{ReadOnly, ReadWrite}` enum (kept in javm-exec so
the crate stays cap-unaware per its description). Maps to
`perm::RO` / `perm::RW`.
- New `MapError::{UnalignedStart, UnalignedSize, Overflow}` for
setup-time validation. Distinct from `MemAccess` (which covers
runtime access faults).
- Grows `flat_mem` and `perms` as needed; new bytes are zero-
initialized; new pages default to `perm::NONE` before
`map_region` sets them.
- Init bytes copied up to `size`; oversized init is truncated;
undersized init leaves the trailing region zero (matches the
DataCap canonical-form invariant of stripped trailing zeros).
- Re-exported via `javm_exec::{Access, MapError}` so consumers
don't have to reach into the `mem` module.
7 new unit tests cover the alignment / grow / perm-set / init-copy
/ truncate / overlap cases.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…e 3) Between gas seeding and frame push, `Vm::run_instance` now walks `image.memory_mappings` and applies each entry to `Mem` via the new `Mem::map_region`. The mapping's permission is derived from whether `source.target()` is in `image.pinned_slots` (RO) or not (RW). Initial bytes come from the slot's `Cap::Data` via `kernel_assist.data_lookup(content_hash)`; missing or non-Data slots yield a zero-filled region. Nested slot paths are skipped (parallel to the existing HALT write-back logic), to be enabled when chain bytecode starts using nested cnodes routinely. `VmError::MapRegion(MapError)` covers setup-time alignment / overflow errors. Hand-authored kernel-test fixtures have empty `memory_mappings`, so this loop is a no-op for them — the 5 jar-kernel end-to-end tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…age 4)
`genesis()` walks `chain_image.pinned_slots` and
`chain_image.initial_slots`, registers each slot's bytes in
`σ.data_payloads`, and installs a `Cap::Data { content_hash, size
}` at the declared slot in the root cnode. After Stage 3 wired
`Vm::run_instance` to materialise `Image.memory_mappings` via
`kernel_assist.data_lookup`, this closes the loop for standalone
(root) chain bootstrap — every memory mapping has a cnode-resident
DataCap to resolve and bytes registered in σ.
PinnedCap::Image is intentionally skipped: standalone bootstrap
doesn't install Cap::Image pinned slots; chains using them do
their own setup via derive_spawn.
The chain-cnode-factory in `Kernel::from_genesis` calls
`genesis()` per event, so every fresh cnode comes pre-seeded with
pinned + initial caps — modifications within an event are
discarded with the cnode at end-of-event (effective reset).
All 5 jar-kernel end-to-end tests still pass; their Images have
empty pinned/initial sets so the new loops are no-ops.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ots (Stage 5)
`link_elf` now populates the chain Image's full address-space
declaration. For each of the four data regions in `ProgramLayout`:
- **stack**: `initial_slots[STACK_CAP_INDEX] = Cap::Data(EMPTY, size
= stack_pages * 4 KiB)` + a MemoryMapping covering the region.
Unpinned → RW at runtime, zero-filled at every event apply.
- **ro_data**: `pinned_slots[RO_CAP_INDEX] = PinnedCap::Data {
content: <rewritten ro bytes>, size }` + a MemoryMapping.
Pinned → RO at runtime; bytes are the canonical Image spec.
- **rw_data**: `initial_slots[RW_CAP_INDEX] = InitialDataCap {
content: <rewritten rw bytes>, size }` + a MemoryMapping.
Unpinned → RW at runtime, initial bytes from σ each event apply.
- **heap**: `initial_slots[HEAP_CAP_INDEX] = Cap::Data(EMPTY, size
= heap_pages * 4 KiB)` + a MemoryMapping. Same shape as stack.
`rewrite_data_code_ptrs` (previously kept under
`#[allow(dead_code)]`) is back on the live path: it patches LLVM
jump-table code pointers stored in `.rodata` / `.data` from
RISC-V vaddrs to PVM PCs before the bytes are sealed into
`pinned_slots` / `initial_slots`. The `LinkedElf` `allow(dead_code)`
marker is dropped too — all fields are consumed again.
Slot indices reuse the existing `*_CAP_INDEX` constants from
`layout.rs` (64–68). 64 is the legacy CODE cap index (now unused;
left as-is). 65/66/67/68 = stack/ro/rw/heap. These don't collide
with BARE_* (slots 7-16) or slot 0 (event payload).
End-to-end: after Stages 1–5, transpiled bench Images carry a
self-describing address-space spec; the kernel installs the
declared caps at genesis; `Vm::run_instance` materialises the
mappings into `Mem` before the interpreter starts. The 5
jar-kernel end-to-end tests still pass (their fixtures have empty
memory_mappings / pinned_slots / initial_slots; the loops are
no-ops).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the capability-based microkernel + PVM described in
~/docs/minimum/(lives in this repo's sibling docs tree). Two intertwined refactors:1. The kernel. A ~300-line core: cap registry, EventFlow lifecycle, AttestationCap issuance, trace management,
apply_block, state root. No native token, no economics, no ambient authority — every privileged op presents a cap the kernel verifies by registry lookup. Validator set, consensus, slashing, balances all live in userspace components, swappable per-chain.Kernel<H>owns the node tip and lifecycle methods;Hardwareis the boundary for crypto / network / storage. Transact and Dispatch (sync invoke + async deterministic scheduling) are the only two cross-component primitives.2. The PVM (javm). Capability-based, Harvard architecture, per-spec
08-pvm.md:UNTYPED(bump-allocator pages, copyable),DATA(move-only physical pages with per-VM mapping memory + per-page bitmap),CODE(compiled PVM, owns 4GB window, copyable),HANDLE(VM owner, unique),CALLABLE(entry point, copyable),EPHEMERAL_TABLE(per-invocation handle, kernel-managed lifetime).EPHEMERAL_TABLEcap pointing to it.[target_byte, ind_0, ind_1, ind_2]— when target byte is 0 and the cap-ref is non-zero, descend through slot 0 of the current frame (anEPHEMERAL_TABLEorHANDLE) and shift right one byte. Recursive —0x00_03_00reaches ephemeral sub-slot 3 (Gas cap);0x00_05_00_00reaches the caller's persistent Frame slot 5 via two hops.cap_ref == 0is the EphemeralTable handle itself;CALL(0)is REPLY.mappingsmakes the lend pattern ergonomic). Scalar args still flow through φ[7..12].Capability::Gas { remaining }at ephemeral sub-slot 3 is the single per-invocation budget.MGMT_GAS_DERIVE/MGMT_GAS_MERGEprovide the park pattern for per-call restriction (subsumes the retiredMGMT_SET_MAX_GAS). Kernel-default rollback on child fault scans parent's Frame for parked Gas caps and merges them back.mappings: Vec<VmMapping>+active_in: Option<VmId>replace singlebase_offset/accessfields. Cross-frame MOVE auto-unmaps; arrival in a VM's persistent Frame consultsmappings[dst_vm]for auto-remap. Move into the ephemeral table preserves mappings unchanged.3. jar-kernel adaptation.
Capabilityvariants:Gas(GasCap),SelfId(SelfCap),CallerVault(CallerVaultCap),CallerKernel(CallerKernelCap).KernelCap::gas_derive/gas_mergeimpls split/merge the Gas cap.HostCall::Gas/SelfId/Callerretire from the host-call dispatcher (slots 1/2/3 reserved). Replaced by Capability caps in the ephemeral table — guests will read them via cap-ref into ephemeral sub-slots 1/2/3.CnodeGrant/CnodeRevoke/CnodeMove→ javm management ecallis,CapDerive→MGMT_DOWNGRADE,CapCall→ plain javm CALL,VaultInitialize/CreateVault/QuotaSet→ kernel-internal ops,AttestationAggregate/SlotEmit→ unimplemented stubs).populate_ephemeral_kernel_capswrites Caller/Self/Gas at ephemeral sub-slots 1/2/3 at every kernel-driven invocation entry (Transact + Dispatch step-2 + step-3).4. javm API simplification.
InvocationKernel::new(blob, gas)— noargs: &[u8]parameter. javm has no notion of "input args"; hosts populate DATA caps viakernel.write_data_cap_init(slot, bytes) -> u64post-construction. The args-slot convention is host/transpiler-defined (transpiler emitsARGS_CAP_INDEX = 69).Cap<P>generic over aProtocolCapTpayload — jar-kernel substitutesKernelCap(HostCall selector ORCapabilitycap), tests/benches keepu8.Verification
cargo test --workspace— all suites green (162 in javm core, +adjacent crates).cargo clippy --workspace --all-targets -- -D warningsclean.cargo fmt --all -- --checkclean.cargo run -p jar -- testnet --nodes 3 --slots 5— all 3 nodes converge on identical state root across all 5 slots.cargo bench -p javm-benchvs the pre-refactorpre-cnodebaseline:take_ephemeral_kernel_slots/restore_ephemeral_kernel_slots.Deliberately deferred
Capability::Gas.remaining. Today the JIT writesVmInstance.gas; the kernel sweeps caller→callee on every CALL/REPLY/HALT. Behavior is observably equivalent to a shared counter; the optimization recovers hostcall regressions and is a focused perf commit.Test plan
cargo test --workspace(run before merge to confirm no regression on master)cargo clippy --workspace --all-targets -- -D warningscargo fmt --all -- --checkcargo run -p jar -- testnet --nodes 3 --slots 5— confirm convergence on a single state rootcargo bench -p javm-bench --bench pvm_bench -- --baseline pre-cnodeif perf-sensitive~/docs/minimum/— the PR is the implementation of that spec.🤖 Generated with Claude Code