forked from vercel/pkg
-
-
Notifications
You must be signed in to change notification settings - Fork 61
feat: enhanced SEA mode with walker integration and VFS #229
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
robertsLando
wants to merge
41
commits into
main
Choose a base branch
from
feat/sea-enhanced-vfs
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
41 commits
Select commit
Hold shift + click to select a range
f7dd471
feat: enhanced SEA mode with walker integration and VFS (#204)
robertsLando 7b55262
fix: move @platformatic/vfs to optionalDependencies for Node 20 CI co…
robertsLando 26ddcb0
feat: drop Node.js 20 support, require Node.js >= 22
robertsLando 69c0010
fix: restore test-00-sea Node version guard to < 20
robertsLando 13541e1
fix: restore assertSeaNodeVersion to check Node >= 20
robertsLando 49ad5a9
test: enhance SEA mode tests for pkg compatibility and error handling
robertsLando 0842c82
fix: improve error handling and conditionally remove Mach-O signature…
robertsLando 862df2a
refactor: extract shared runtime code into bootstrap-shared.js
robertsLando 8a42a62
fix: address Copilot review feedback on SEA pipeline
robertsLando f539f92
fix: throw when JSON config targets Node < 22 in SEA mode
robertsLando 6f580b6
fix: normalize paths for Windows compatibility in SEA bootstrap
robertsLando 151f25c
docs: add technical architecture documentation
robertsLando e49e17e
fix: normalize all SEA provider paths to POSIX for Windows compatibility
robertsLando 4d0ef4a
docs: mention --no-bytecode flag in architecture comparison
robertsLando 3399b66
fix: address Copilot review feedback for SEA enhanced mode
robertsLando f3d47fe
feat: add DEBUG_PKG diagnostics to SEA mode, deduplicate diagnostic code
robertsLando e6f5753
fix: address remaining Copilot review feedback
robertsLando 10a84db
feat: add worker thread support and fix SEA walker warnings
robertsLando 8e612e3
docs: update architecture documentation to include worker thread supp…
robertsLando 0808a05
fix: mount VFS at POSIX /snapshot on all platforms for Windows compat
robertsLando 5e4a53a
fix: use Module._compile for worker threads instead of raw eval
robertsLando a4dcb29
fix: use postject JS API instead of npx to inject SEA blobs
robertsLando c2d3b91
fix: avoid eager file loading and unnecessary temp files in SEA walker
robertsLando 2cdf100
fix: pass defaultEntrypoint in SEA bootstrap for API consistency
robertsLando 8293cf8
feat: add snapshotify function to convert file paths to snapshot paths
robertsLando d7229f7
docs: fix version comments and Node 22 requirement notes
robertsLando 1e80fed
refactor: reuse replaceSlashes from common.ts instead of duplicate to…
robertsLando 2801ce4
fix: handle VFS-incompatible fs calls in SEA bootstrap
robertsLando ebc1162
feat: add SKILL.md for packaging performance benchmarks comparison
robertsLando 3d0b118
perf: optimize SEA provider startup and add DEBUG_PKG_PERF instrument…
robertsLando 219d978
docs: split project instructions into modular .claude/rules files
robertsLando 8255a4a
feat: enhance SEA asset generation with single archive blob and offse…
robertsLando 58ccfa5
docs: update README and ARCHITECTURE.md for single archive blob and r…
robertsLando 3fb1387
fix: enable model invocation for performances-compare skill
robertsLando 051c73b
refactor: replace @platformatic/vfs with @roberts_lando/vfs@0.3.0
robertsLando f64bf63
refactor: update Node.js version requirements and testing commands in…
robertsLando 9ed8ce5
feat: track body modifications in stepStrip for enhanced record handling
robertsLando a536430
chore: update @roberts_lando/vfs to version 0.3.1 in package.json, pa…
robertsLando 394deb1
chore: update @roberts_lando/vfs to version 0.3.2
robertsLando af5b851
fix: address Copilot review — patchDlopen regression, SEA blob fallba…
robertsLando f2f1d9f
refactor: use assertSeaOutput helper in test-00-sea
robertsLando File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| --- | ||
| description: Build, lint, formatting, and TypeScript rules | ||
| --- | ||
|
|
||
| # Development | ||
|
|
||
| ## Build | ||
|
|
||
| ```bash | ||
| npm run build # Required before testing — compiles lib/ to lib-es5/ | ||
| npm run start # Watch mode with auto-rebuild | ||
| ``` | ||
|
|
||
| ## Lint & Format | ||
|
|
||
| ```bash | ||
| npm run lint # Check both ESLint + Prettier | ||
| npm run fix # Auto-fix all issues | ||
| npm run lint:code # ESLint only | ||
| npm run lint:style # Prettier only | ||
| ``` | ||
|
|
||
| - Always run `npm run lint` before committing. Fix all issues — never push dirty code. | ||
| - Console statements are disallowed in production code but allowed in test files. | ||
|
|
||
| ## TypeScript | ||
|
|
||
| - Strict mode enabled. Target: ES2022. Module system: CommonJS. | ||
| - Edit `lib/*.ts` only — never edit `lib-es5/*.js` directly. | ||
| - Requires Node.js >= 22.0.0. | ||
|
|
||
| ## Dependencies | ||
|
|
||
| - Keep runtime dependencies minimal — they affect all packaged apps. | ||
| - Use exact or caret ranges. | ||
| - `pkg-fetch` provides pre-compiled Node.js binaries. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| --- | ||
| description: Git workflow, commit conventions, and PR rules | ||
| --- | ||
|
|
||
| # Git & PR Workflow | ||
|
|
||
| ## CRITICAL: PR Target | ||
|
|
||
| - ALWAYS create PRs against `yao-pkg/pkg` (this fork). | ||
| - NEVER target `vercel/pkg` — it is archived and read-only. | ||
|
|
||
| ## Commits | ||
|
|
||
| - Use conventional commits: `feat:`, `fix:`, `refactor:`, `test:`, `chore:`, `docs:` | ||
| - Default branch: `main`. Tag format: `v${version}`. | ||
| - Branch protection requires passing CI checks. | ||
|
|
||
| ## Before Committing | ||
|
|
||
| 1. Clean test artifacts from test directories (`*.exe`, `*-linux`, `*-macos`, `*-win.exe`). | ||
| 2. Run `npm run lint` and fix all issues. | ||
| 3. Show `git status --short` and get user approval before committing. | ||
|
|
||
| ## Release | ||
|
|
||
| Uses `release-it` with conventional commits (`npm run release`). This runs linting, generates changelog, creates a git tag, pushes to GitHub, and publishes to npm as `@yao-pkg/pkg`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| --- | ||
| description: VFS, native addons, cross-compilation, ESM, and platform-specific notes | ||
| paths: | ||
| - 'prelude/**' | ||
| - 'lib/**' | ||
| --- | ||
|
|
||
| # Platform & Virtual Filesystem | ||
|
|
||
| ## VFS Path Handling | ||
|
|
||
| - Packaged apps use `/snapshot/` prefix (or `C:\snapshot\` on Windows). | ||
| - Use `__dirname`/`__filename` for snapshot files, `process.cwd()` for runtime fs. | ||
| - Path handling differs between packaged and non-packaged execution. | ||
|
|
||
| ## Native Addons | ||
|
|
||
| - Extracted to `$HOME/.cache/pkg/` at runtime. Must match target Node.js version. | ||
| - `linuxstatic` target cannot load native bindings. | ||
| - Add `.node` files to `assets` if not detected automatically. | ||
|
|
||
| ## Cross-Compilation | ||
|
|
||
| - Bytecode generation requires the target architecture binary. | ||
| - Use `--no-bytecode` for cross-arch builds. | ||
| - Linux: QEMU for emulation. macOS: Rosetta 2 for arm64 building x64. | ||
|
|
||
| ## ESM | ||
|
|
||
| - Requires `--options experimental-require-module` on Node.js < 22.12.0. | ||
| - Check existing dictionary files for package-specific ESM handling. | ||
|
|
||
| ## Platform Notes | ||
|
|
||
| - **Linux**: `linuxstatic` target for max portability. | ||
| - **macOS**: arm64 requires code signing (`codesign` or `ldid`). | ||
| - **Windows**: `.exe` extension required; native modules must match target arch. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| --- | ||
| description: Project overview, structure, and key files for yao-pkg/pkg | ||
| --- | ||
|
|
||
| # Project Overview | ||
|
|
||
| `pkg` packages Node.js projects into standalone executables for Linux, macOS, and Windows. It supports Node.js 22 and newer, virtual filesystem bundling, V8 bytecode compilation, native addons, and compression (Brotli, GZip). | ||
|
|
||
robertsLando marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| This is `yao-pkg/pkg` — a maintained fork of the archived `vercel/pkg`. | ||
|
|
||
| ## Repository Structure | ||
|
|
||
| - `lib/` — TypeScript source (compiled to `lib-es5/` via `npm run build`) | ||
| - `prelude/` — Bootstrap code injected into packaged executables | ||
| - `dictionary/` — Package-specific configs for known npm packages | ||
| - `test/` — Numbered test directories (`test-XX-name/`) | ||
| - `.github/workflows/` — CI/CD (GitHub Actions) | ||
|
|
||
| ## Key Entry Points | ||
|
|
||
| - `lib/index.js` — API entry point | ||
| - `lib/bin.js` — CLI entry point | ||
| - `prelude/bootstrap.js` — Injected into every packaged executable | ||
| - `dictionary/*.js` — Special handling for specific npm packages | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| --- | ||
| description: Test commands, organization, and patterns | ||
| paths: | ||
| - 'test/**' | ||
| --- | ||
|
|
||
| # Testing | ||
|
|
||
| ## Commands | ||
|
|
||
| ```bash | ||
| npm run build # Always build first | ||
| npm run test:22 # Test with Node.js 22 | ||
| npm run test:host # Test with host Node.js version | ||
| node test/test.js node22 no-npm test-50-* # Run specific test pattern | ||
| ``` | ||
|
|
||
robertsLando marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ## Organization | ||
|
|
||
| - Tests live in `test/test-XX-descriptive-name/` directories (XX = execution order). | ||
| - Each test has a `main.js` entry point using utilities from `test/utils.js`. | ||
| - `test-79-npm/` — npm package integration tests (only-npm). | ||
| - `test-42-fetch-all/` — verifies patches exist for all Node.js versions. | ||
|
|
||
| ## Writing Tests | ||
|
|
||
| 1. Create `test/test-XX-descriptive-name/` with a `main.js` | ||
| 2. Use `utils.pkg.sync()` to invoke pkg | ||
| 3. Verify outputs, clean up with `utils.filesAfter()` | ||
|
|
||
| Test artifacts (`*.exe`, `*-linux`, `*-macos`, `*-win.exe`) must be cleaned from test directories before committing. | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.