feat(subos): subos-as-xpkg system (M1-M5) by Sunrisepeak · Pull Request #296 · openxlings/xlings

Sunrisepeak · 2026-05-15T21:51:11Z

Summary

Implements the subos-as-xpkg system per design doc .agents/docs/subos-as-xpkg-design-2026-05-16.md (rev4). Subos environments are now first-class xpkg packages with type = "subos", forked in near-zero seconds via --from, executed non-interactively via --cmd, and accelerated for high-frequency sandbox exec via the auto-keeper primitives.

Five user-facing capabilities land here (M1–M5):

M	Capability	Example
M1	`type = "subos"` package format + default install/config/uninstall hooks	`xlings install subos:py-ds@1.0.0`
M2	`subos new --from <local	pkg-spec>` fork with auto-install
M3	`subos use --cmd "<string>"` non-interactive exec	`xlings subos use exp --cmd "python script.py"`
M4	Auto-keeper primitives (Linux) + `subos stop` lifecycle	`xlings subos stop exp`
M5	Keeper policy overrides (`--no-keep`, `--ttl <sec>`, `--keep`)	`xlings subos use exp --sandbox --ttl 600 --cmd "..."`

Key design decisions

type = "subos" is a normal xpkg type, not a special schema. Packages land at the standard xpkgs/<namespace>-x-<name>/<ver>/ path with no special prefix; xvm registration is normal. Authors can override any default hook by defining install()/config()/uninstall() in their .lua.
Fork is just file copy with reflink/clonefile where supported (Linux btrfs/xfs, macOS APFS). xpkg dependencies stay shared in xpkgs/; the fork only carries the small .xlings.json + static files, so it's near-instant.
Storage choice belongs to the fork, not the base. Base packages are platform-agnostic recipes; users pick --storage shared|tmpfs|image per fork.
Keeper is opt-in for now: this PR ships the keeper module primitives + subos stop CLI + --keep/--no-keep/--ttl flag plumbing. The auto-default behavior (storage=image|tmpfs + sandbox + Linux → spawn keeper with TTL=5min) is a deliberate one-line follow-up once we validate the bwrap-keeper fork point against the CI matrix.

Dependency

Requires the upstream mcpplibs/libxpkg PR adding PackageType::Subos:
👉 openxlings/libxpkg#23

Until that merges, this PR builds via xmake f --local_libxpkg=/path/to/libxpkg against the feature branch.

File map

Created

src/core/xim/libxpkg/types/subos.cppm — default install/config/uninstall hooks (mirrors script.cppm pattern)
src/core/subos/keeper.cppm — keeper primitives: is_alive (with stale PID cleanup), register_pid, touch_activity, nsenter_and_exec, stop_keeper, should_auto_keeper predicate
tests/e2e/fixtures/subos_xpkg/py-demo.lua — fixture: minimal type = "subos" package
tests/e2e/subos_xpkg_install_test.sh — M1 e2e
tests/e2e/subos_xpkg_fork_test.sh — M2 e2e (local + pkg-spec fork, auto-install, isolation)
tests/e2e/subos_xpkg_use_cmd_test.sh — M3 e2e (--cmd, exit-code, incompatible flags)
tests/e2e/subos_xpkg_keeper_test.sh — M4+M5 e2e (subos stop, flag mutual-exclusion, stale PID cleanup)
.agents/docs/subos-as-xpkg-design-2026-05-16.md — design doc (rev1→rev4)
docs/superpowers/plans/2026-05-16-subos-as-xpkg.md — implementation plan

Modified

src/core/xim/index.cppm — type_to_int/int_to_type map Subos ↔ 4
src/core/xim/libxpkg/types/type.cppm — comment updated for new pkgType value
src/core/xim/installer.cppm — install/config/uninstall dispatch for pkgType == 4
src/core/subos.cppm — new_from() export, use_spawn_shell cmd param, bwrap/proot argv builders thread cmd, subos new --from/subos stop/keeper-flag argparse, mutual-exclusion validation

Test plan

Build clean on Linux (gcc 16.1.0, non-musl)
All 4 e2e tests pass: install / fork / use-cmd / keeper
CI matrix passes (Linux + macOS + Windows)
Real bwrap+nsenter sandbox keeper integration (deferred follow-up — primitives in place)

What's verified locally

$ for t in tests/e2e/subos_xpkg_*.sh; do bash "$t" | tail -1; done
[project-e2e] PASS: subos-as-xpkg install path works end-to-end (M1)
[project-e2e] PASS: subos new --from works end-to-end (M2)
[project-e2e] PASS: subos use --cmd works end-to-end (M3)
[project-e2e] PASS: keeper CLI surface works end-to-end (M4 + M5)

Out of scope (deferred, see design §11 — Future)

One-line throwaway subos use subos:xxx --sandbox --cmd ... (cleanup/GC complexity; explicit two-step suffices for now)
Binary cache for fast first-fork
subos = {...} descriptor-only form (no tarball)
Overlayfs / COW layered subos
Subos pkg with embedded user-data
Real auto-keeper spawn integration in use_sandbox_mode_ (keeper module + CLI plumbed; one-line flip when ready)

Commit walk

feat(xim): map PackageType::Subos to pkgType=4 — schema/dispatch foundation (requires upstream PR)
feat(xim): dispatch type='subos' through default install/config/uninstall — M1
feat(subos): subos use --cmd <string> for non-interactive exec — M3
feat(subos): subos new --from <spec> for fork from local subos or pkg — M2
feat(subos): auto-keeper primitives + --keep/--no-keep/--ttl + subos stop — M4+M5
docs: subos-as-xpkg design (rev4) + implementation plan — design + plan

Mirrors upstream mcpplibs/libxpkg enum addition. Foundation for type=subos package dispatch — see .agents/docs/subos-as-xpkg-design-2026-05-16.md (Phase 0 / Task 1). Requires upstream commit "feat(xpkg): add PackageType::Subos for subos-as-xpkg" on mcpplibs/libxpkg feat/pkgtype-subos branch.

…tall Adds xim::subos namespace mirroring the script.cppm pattern. Default hooks: - install: ensure install_dir + bin/ skeleton; synthesize .xlings.json workspace from xpm.deps when tarball doesn't carry one - config: register via xvm.add_version so the package is queryable and uninstallable like any normal xpkg - uninstall: remove xvm entry; on-disk payload removal is handled by xim's standard uninstall path Authors can override any of the three hooks by defining install()/ config()/uninstall() in the package .lua. The existing executor's has_hook() check still routes to user code first. Package path convention is unchanged — type='subos' packages land at xpkgs/<namespace>-x-<name>/<ver>/, identical to xim-x-foo / scode-x-foo. E2E test covers: install path, xpkgs layout, .xlings.json synthesis, xvm registration. Fixture in tests/e2e/fixtures/subos_xpkg/py-demo.lua. Refs: .agents/docs/subos-as-xpkg-design-2026-05-16.md (M1)

Extends 'xlings subos use' with --cmd <string> (and --cmd=<value>) to run a single command in the subos and exit with the command's exit code. POSIX: shell -c <cmd>; Windows: pwsh -Command / cmd /c. Works in both shell-level and sandbox modes; threaded through build_bwrap_argv_ and build_proot_argv_ to append -c <cmd>. Rejected combinations: --cmd + --global : --global persists active subos, doesn't spawn --cmd + --shell : --shell emits eval-able env code, no shell to exec E2E coverage: stdout capture, exit-code propagation, incompatible flag combinations, --cmd=<value> equals form. Refs: .agents/docs/subos-as-xpkg-design-2026-05-16.md (M3)

Adds `xlings subos new <name> --from <spec>` which forks the new subos from either: - a local subos (bare name, e.g. --from base-env): copies content from subos/<base>/ to subos/<new>/ - a pkg-spec (contains ':' or '@', e.g. --from subos:py-ds@1.0.0): locates xpkgs/<ns>-x-<name>/<ver>/; if not yet installed, auto-invokes `xlings install <spec>` (E5: agent always 1 command) Cross-platform copy uses reflink-where-possible (Linux btrfs/xfs, macOS APFS clonefile); falls back to full byte copy via std::filesystem::copy on Windows / non-COW filesystems. xpkg deps stay shared in xpkgs/ so the fork itself is near-instant on shared storage; the new subos's workspace inherits the base's .xlings.json. Storage choice belongs to the fork (per E2 design): base is a recipe that doesn't pin storage mode; user picks --storage at fork time. copy_tree_ overlays base content, then storage/imageSize fields in .xlings.json are re-applied so the new subos's storage wins. E2E coverage: local fork content inheritance, fork independence (modification isolation), pkg-spec fork with auto-install, --from= equals form, error path for missing source. Refs: .agents/docs/subos-as-xpkg-design-2026-05-16.md (M2, E1-E5)

…stop Adds the keeper module (Linux-focused, cross-platform stubs) that holds bwrap's mount namespace alive between sandboxed --cmd execs so high-frequency agent workloads avoid per-call mount overhead. This commit lands: - keeper.cppm: register_pid / touch_activity / is_alive (with stale PID cleanup) / nsenter_and_exec / stop_keeper / should_auto_keeper predicate. POSIX headers in global module fragment to avoid `import std;` redeclaration conflicts. - subos.cppm: argparse for --keep / --no-keep / --ttl <sec> on `subos use`; mutual-exclusion validation; integer-parse error handling for --ttl. - `xlings subos stop <name>` CLI: SIGTERM then SIGKILL fallback, cleans .keeper.pid + .keeper.lastused. Idempotent — safe to call when no keeper is running. - E2E coverage: stop-no-op, --keep/--no-keep mutual exclusion, --ttl non-integer rejection, --ttl + --no-keep parses, stale PID file cleanup via subos stop. The auto-spawn integration with use_sandbox_mode_ (full bwrap-fork + nsenter dispatch on first --sandbox --cmd, with should_auto_keeper gating per D9) is a deliberate follow-up: the primitives are wired, the CLI surface is complete, and the auto-trigger flip is a one-line change once bwrap-keeper fork point is validated against the matrix. Refs: .agents/docs/subos-as-xpkg-design-2026-05-16.md (M4 + M5, D9)

Design doc records the converged subos-as-xpkg architecture across revisions: rev1: initial brainstorming convergence rev2: simplification (no Lua API needed) rev3: bring back type='subos' + default hooks; sandbox/storage decisions; xvm-as-normal-package registration rev4: auto-keeper with TTL=5min idle (M4); explicit overrides (M5) Plan decomposes into 11 tasks across 5 phases (Phase 0 + M1-M5), with parallel/sequential dependencies noted so a downstream subagent run can fan out where the file map allows. Each task has TDD-style checkpoints + exact file paths. This PR's commits realize Phase 0 + M1-M5 (CLI surface complete; auto-keeper runtime spawn deferred to follow-up). Refs design: .agents/docs/subos-as-xpkg-design-2026-05-16.md Refs plan: docs/superpowers/plans/2026-05-16-subos-as-xpkg.md

Pulls in the upstream Subos enum addition required by the subos-as-xpkg dispatch. Replaces the local_libxpkg dev override that was needed before openxlings/libxpkg#23 merged. Refs: mcpplibs/mcpplibs-index#13, openxlings/libxpkg#23

Includes: - feat(subos): subos-as-xpkg system (M1-M5) - type='subos' xpkg dispatch + default install/config/uninstall hooks - subos new --from <local|pkg-spec> fork with auto-install - subos use --cmd <string> non-interactive exec (POSIX + Windows) - keeper primitives + --keep/--no-keep/--ttl flags + subos stop - chore: bump mcpplibs-xpkg dep to 0.0.41 (brings PackageType::Subos)

Sunrisepeak added 8 commits May 16, 2026 05:27

Sunrisepeak marked this pull request as ready for review May 15, 2026 22:29

Sunrisepeak merged commit 8e72e62 into main May 15, 2026
3 checks passed

Sunrisepeak deleted the feat/subos-as-xpkg branch May 15, 2026 22:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(subos): subos-as-xpkg system (M1-M5)#296

feat(subos): subos-as-xpkg system (M1-M5)#296
Sunrisepeak merged 8 commits into
mainfrom
feat/subos-as-xpkg

Sunrisepeak commented May 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Sunrisepeak commented May 15, 2026

Summary

Key design decisions

Dependency

File map

Test plan

What's verified locally

Out of scope (deferred, see design §11 — Future)

Commit walk

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant