fix(build): auto-filter non-Linux platform-specific native binaries#1117
Conversation
…rom Lambda bundles Native packages like @swc/core-darwin-arm64 (~22MB) are included by Next.js output file tracing when building on macOS but are never needed on Lambda (Linux). This adds automatic detection and exclusion of darwin/win32/freebsd/android platform binaries during the traced files copy step, reducing bundle size significantly.
🦋 Changeset detectedLatest commit: 558568e The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
@clichedmoog Thanks for the PR. Could you please expand about the rationale for this PR? Would I still be able to dev on my local Mac Book if we merge this? |
|
@vicb Thanks for the review! Rationale: When building on macOS, npm/pnpm installs platform-matching native binaries (e.g. I ran into this in a production project using In that project, I measured ~65MB of darwin-only binaries in a 209MB standalone output - This PR tries to automate that filtering by matching the I should note that this mainly affects local macOS builds — CI environments on Linux wouldn't install darwin binaries in the first place (same reason Vercel doesn't hit this issue). Local Mac development: Yes — this filter only runs inside I tested on macOS and confirmed |
I was referring the launching a local Open Next server, i.e. what we do for local e2e tests. Maybe you can try if this still works. Edit: also I think we have a feature to install external packages and specify what arch to install - I don't have time to lookup for the details but you should be able to find something in the code or looking at the Open Next config. |
Add empty loader for .node files in both esbuildSync and esbuildAsync,
preventing build crashes when packages like @swc/core include
platform-specific native binaries that esbuild cannot bundle.
Uses loader: { ".node": "empty" } approach which works for both sync
and async builds, replacing .node imports with empty modules.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
@vicb Thanks for the clarification! I haven't tested the full local OpenNext e2e flow yet — I'll set up the examples and try That said, I think it should be fine: the packages typically matched by this filter (SWC, esbuild, Rollup, etc.) are build-time native binaries that aren't needed by the server at runtime — Next.js pre-compiles everything in standalone mode. The Good point about the I'll follow up with local e2e test results. |
|
Update: Tested the local OpenNext e2e flow on macOS (M1):
Local OpenNext development on macOS is not affected by this change. |
Which as you touch base in your last message tend to confirm that they are not used. |
conico974
left a comment
There was a problem hiding this comment.
I don't think the esbuild thing is doing anything, or if it is we have a bigger problem that we cannot just dismiss like that.
The esbuild issue is more likely coming from here
I'd prefer this esbuild fix to be in another PR as well, these are 2 different issue
| ); | ||
| } | ||
|
|
||
| const NON_LINUX_PLATFORMS = ["darwin", "win32", "freebsd", "android"]; |
There was a problem hiding this comment.
Pretty sure you can drop android 😄
There was a problem hiding this comment.
I've dropped the android
| sourcesContent: false, | ||
| ...esbuildOptions, | ||
| // Native .node binaries cannot be bundled by esbuild | ||
| loader: { ".node": "empty", ...esbuildOptions.loader }, |
There was a problem hiding this comment.
This is supposed to be called only for OpenNext code, if not then we have a way bigger issue.
There was a problem hiding this comment.
That makes sense — moved to #1120 so we can discuss it separately there.
| } | ||
| // Skip non-Linux platform-specific native binaries (e.g. @swc/core-darwin-arm64) | ||
| if (isNonLinuxPlatformPackage(from)) { | ||
| const match = from.match( |
There was a problem hiding this comment.
This is always a bit dangerous to skip things like that. Could you add a bypass for that (like an env variable to bypass this check)
There was a problem hiding this comment.
Good idea. Added OPEN_NEXT_SKIP_PLATFORM_FILTER env variable for that.
… split esbuild change - Remove android from platform filter list - Add OPEN_NEXT_SKIP_PLATFORM_FILTER env variable to bypass the filter - Revert esbuild .node loader change (to be addressed in a separate PR)
conico974
left a comment
There was a problem hiding this comment.
LGTM, thanks for the PR
commit: |
* Add SKILL.md for porting PRs and AGENTS.md for coding guidelines * update skill * Port opennextjs/opennextjs-aws#1118 as a test * Port opennextjs/opennextjs-aws#1117 * update skill * Port opennextjs/opennextjs-aws#1114 * Port PR opennextjs/opennextjs-aws#1107 * update skills * Port PR opennextjs/opennextjs-aws#1108 * Port PR opennextjs/opennextjs-aws#1104 * Port PR opennextjs/opennextjs-aws#1101 * Port PR opennextjs/opennextjs-aws#1098 * review
* Add SKILL.md for porting PRs and AGENTS.md for coding guidelines * update skill * Port opennextjs/opennextjs-aws#1118 as a test * Port opennextjs/opennextjs-aws#1117 * update skill * Port opennextjs/opennextjs-aws#1114 * Port PR opennextjs/opennextjs-aws#1107 * update skills * Port PR opennextjs/opennextjs-aws#1108 * Port PR opennextjs/opennextjs-aws#1104 * Port PR opennextjs/opennextjs-aws#1101 * Port PR opennextjs/opennextjs-aws#1098 * chore: port PR #1083 from source repository opennextjs/opennextjs-cloudflare#1083 Changeset: .changeset/port-pr-1083.md * chore: port PR #1105 from source repository opennextjs/opennextjs-cloudflare#1105 Changeset: .changeset/port-pr-1105.md * chore: port PR #1097 from source repository opennextjs/opennextjs-cloudflare#1097 Changeset: .changeset/port-pr-1097.md * chore: port PR #1122 from source repository opennextjs/opennextjs-cloudflare#1122 Applied bugfixes and improvements to the 'migrate' command: - Fixed extra newlines when appending to files (updated conditionalAppendFileSync signature) - Fixed error when 'public' directory is missing (creates parent directories automatically) - Fixed Next.js config file update to check if the file exists first - Updated checkRunningInsideNextjsApp to accept { appPath: string } instead of full BuildOptions Changesets: - .changeset/port-pr-1122-cloudflare.md - .changeset/port-pr-1122-aws.md * chore: update port PR skill instructions for staging and committing changes * chore: port PR #1126 from source repository opennextjs/opennextjs-cloudflare#1126 Fix: prevent Worker hang on HEAD requests to static assets When run_worker_first is enabled, HEAD requests to static assets hang the Worker because response.body is null (per HTTP spec) and the fallback new ReadableStream() creates a stream that never closes. Changes: - Return null body for HEAD requests instead of falling through to the hanging ReadableStream fallback - Add tests for maybeGetAssetResult covering GET, HEAD, 404, POST, and run_worker_first=false cases Changeset: .changeset/port-pr-1126.md * linting * chore: port PR #1127 from source repository opennextjs/opennextjs-cloudflare#1127 Changeset: .changeset/port-pr-1127.md * chore: port PR #1138 from source repository opennextjs/opennextjs-cloudflare#1138 Changeset: .changeset/port-pr-1138.md * chore: port PR #1133 from source repository opennextjs/opennextjs-cloudflare#1133 Changeset: .changeset/port-pr-1133.md Update the migrate command to attempt to create an R2 bucket for caching as part of the migration process, if that is not possible an application without caching enabled will be generated instead. * chore: port PR #1142 from source repository opennextjs/opennextjs-cloudflare#1142 Changeset: .changeset/port-pr-1142.md * chore: port PR #1147 from source repository opennextjs/opennextjs-cloudflare#1147 make dev /cdn-cgi/image behaves like prod for consistency Changeset: .changeset/port-pr-1147.md * chore: port PR #1150 from source repository opennextjs/opennextjs-cloudflare#1150 Changeset: .changeset/port-pr-1150.md * fix lockfile * fix test
Motivation
When building on macOS, npm/pnpm installs platform-matching native binaries (e.g.
@swc/core-darwin-arm64,@esbuild/darwin-arm64) viaoptionalDependencies. Next.js output file tracing picks these up because they exist on disk, causing two problems:.nodefiles and fails with "No loader is configured for .node files" (next-intl#2255)copyTracedFilescopies darwin/win32 binaries into the Lambda bundle where they can never work (~65MB in a production app)The current user-side workaround is
outputFileTracingExcludesinnext.config.ts, but this requires manually listing every platform-specific package.This mainly affects local macOS/Windows builds — CI environments on Linux wouldn't install these binaries in the first place.
Local development is not affected — these changes only run during the OpenNext build step.
next dev,next build, and the local OpenNext server (openbuild:local:start) all work normally on macOS.Changes
1.
helper.ts— Prevent esbuild crash on.nodefilesAdded
loader: { ".node": "empty" }to bothesbuildSyncandesbuildAsync, so esbuild skips native binaries instead of crashing.2.
copyTracedFiles.ts— Filter non-Linux platform binariesAdded
isNonLinuxPlatformPackage()that auto-detects and excludesdarwin/win32/freebsd/androidpackages matching the{pkg}-{platform}-{arch}naming convention. Operates independently from existingEXCLUDED_PACKAGES.Both
linux-arm64andlinux-x64packages are kept since the target architecture is unknown at build time.This filter complements the existing
installfeature (InstallOptionswith--os/--arch):installprovides the correct platform binaries for runtime dependencies, while this filter removes the wrong ones that came from the host machine.Related issues
Verification
Tested on a production Next.js app (next-intl + Prisma + esbuild) by completely removing
outputFileTracingExcludes— both Next.js and OpenNext builds succeed, and no darwin binaries end up in the Lambda bundle.