Skip to content

feat: ship prebuilt better-sqlite3 binaries for CentOS/Alpine compatibility#81

Closed
Lets7512 wants to merge 3 commits intomksglu:mainfrom
Lets7512:feat/prebuild-better-sqlite3
Closed

feat: ship prebuilt better-sqlite3 binaries for CentOS/Alpine compatibility#81
Lets7512 wants to merge 3 commits intomksglu:mainfrom
Lets7512:feat/prebuild-better-sqlite3

Conversation

@Lets7512
Copy link
Copy Markdown
Contributor

@Lets7512 Lets7512 commented Mar 6, 2026

Summary

Ships pre-compiled better_sqlite3.node binaries so npm install context-mode works on CentOS 7/8, Alpine, and systems where the default prebuild-install binaries fail due to glibc mismatch.

This is an alternative to the sql.js WASM fallback approach in #77. Instead of replacing the database engine, we ship platform-native prebuilds compiled against old glibc (2.17) that work everywhere.

How it works

  1. CI builds better-sqlite3 from source in Docker containers targeting old glibc (manylinux2014) and musl (Alpine)
  2. The .node files are shipped in prebuilds/{platform}-{arch}/ inside the npm package
  3. At runtime, SQLiteBase tries the default native binding first
  4. If that fails (glibc mismatch, missing build), it falls back to our prebuild via better-sqlite3's own nativeBinding constructor option

No extra dependencies. No postinstall scripts. No binding.gyp. Just better-sqlite3's built-in nativeBinding API.

Platforms

Target Build Environment Compatibility
linux-x64 manylinux2014 (glibc 2.17) CentOS 7+, Ubuntu 14.04+, all modern distros
linux-x64-musl Alpine 3.x Alpine, Docker Alpine images
darwin-x64 macOS 13 Intel Macs
darwin-arm64 macOS 14 Apple Silicon Macs
win32-x64 Windows Server 2022 Windows 10+

Changes

  • src/db-base.ts: Added findPrebuildPath() + fallback in SQLiteBase constructor using better-sqlite3's nativeBinding option
  • package.json: Added prebuilds to files array
  • .github/workflows/prebuild.yml: CI workflow to build + verify all 5 targets
  • tests/prebuild-fallback.test.ts: Tests for loading, FTS5, nativeBinding, findPrebuildPath

Test plan

  • Build succeeds (tsc)
  • All existing tests pass
  • New prebuild-fallback tests pass (5/5)
  • CI prebuild workflow produces binaries for all 5 targets
  • npm pack includes prebuilds/
  • Verified on CentOS 7 Docker (no compilation needed)

…bility

Add runtime fallback to prebuilt better_sqlite3.node binaries when the
default native binding fails to load (e.g. glibc mismatch on CentOS 7/8).

Uses better-sqlite3's nativeBinding option — no extra dependencies needed.

- db-base.ts: findPrebuildPath() + SQLiteBase constructor fallback
- CI workflow builds on manylinux2014 (glibc 2.17), Alpine, macOS, Windows
- prebuilds/ included in npm package files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Lets7512
Copy link
Copy Markdown
Contributor Author

Lets7512 commented Mar 6, 2026

@mksglu Please let me know if the test plan is sufficient.
Thanks

@mksglu
Copy link
Copy Markdown
Owner

mksglu commented Mar 6, 2026

Thanks for this — the approach is technically sound. nativeBinding is better-sqlite3's official API, and targeting manylinux2014 (glibc 2.17) is the canonical solution for CentOS/Alpine compatibility. The code change in db-base.ts is clean.

However, there are two architectural issues we need to resolve before merging:

1. Package size: ship only Linux prebuilds

The glibc/musl mismatch problem is Linux-specific. macOS and Windows users never hit this — better-sqlite3's default prebuild-install works fine there. Shipping 5 platform binaries (~15MB) penalizes all users for a Linux-only problem.

Ask: Drop darwin-x64, darwin-arm64, and win32-x64 from the prebuild matrix. Ship only:

  • linux-x64 (glibc 2.17) — CentOS 7+, all modern distros
  • linux-x64-musl — Alpine, Docker Alpine images

This cuts package size from ~15MB to ~6MB and targets the actual problem.

2. Publish workflow integration

The prebuild CI workflow produces artifacts, but there's no path from artifacts → npm package. Currently I publish from local (npm publish), which means prebuilds/ would be empty.

This needs a publishing strategy. Options we're considering:

A) CI publish (likely direction): Tag push → CI builds prebuilds → CI runs npm publish. Standard approach for native packages (esbuild, swc, turbo all do this). Needs NPM_TOKEN as GitHub secret.

B) Local publish with artifact download: CI builds prebuilds → before publish, a script downloads them from GitHub Actions artifacts → then npm publish locally.

I need to think through which approach fits our release workflow. Putting this PR on hold until we decide on the publish pipeline — the code itself is ready, it's the delivery mechanism that needs design.

Will revisit soon. Thanks for the solid work here.

@flightlesstux
Copy link
Copy Markdown
Contributor

Hey @Lets7512 — sorry for leaving your test plan question hanging. To answer it directly: yes, the 5 tests in prebuild-fallback.test.ts are sufficient for the code changes in this PR. They cover loading, FTS5, and the nativeBinding path cleanly.

We've decided on the publish pipeline direction — Option A (CI publish). Here's what we need to land this:

Changes needed

1. Drop macOS and Windows from the prebuild matrix

The glibc/musl mismatch is Linux-only. macOS and Windows users are never affected. Please keep only:

  • linux-x64 (manylinux2014 / glibc 2.17)
  • linux-x64-musl (Alpine)

2. Wire up CI publish on tag push

Add a publish job to the workflow that triggers on v* tag push:

  • Builds all Linux prebuilds
  • Runs npm publish with NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

The NPM_TOKEN secret will be set on the repo before the next release.

Once these two changes are in, this is ready to merge. The code in db-base.ts is solid — no other changes needed.

The glibc/musl mismatch is Linux-only — macOS and Windows users are
unaffected. Ship only linux-x64 and linux-x64-musl prebuilds to cut
package size. Add publish job triggered on v* tags using NPM_TOKEN.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@Lets7512
Copy link
Copy Markdown
Contributor Author

Hey @mksglu, @flightlesstux — I've addressed both requested changes:

1. Dropped macOS and Windows from the prebuild matrix
Removed prebuild-macos (darwin-x64, darwin-arm64) and prebuild-windows (win32-x64) jobs. Only linux-x64 (glibc 2.17) and linux-x64-musl (Alpine) remain. Package job updated accordingly.

2. Added CI publish job on v* tag push
New publish job that downloads the consolidated prebuilds and runs npm publish using NODE_AUTH_TOKEN from secrets.NPM_TOKEN.

@mksglu
Copy link
Copy Markdown
Owner

mksglu commented Mar 15, 2026

Hey @Lets7512, thanks for the effort on this — you identified a real problem and took the time to build a solution. I appreciate that.

After deep research, we've gone with a documentation-first approach instead. Here's what we found:

better-sqlite3 already handles this automatically. Its install script is prebuild-install || node-gyp rebuild --release — when prebuilds don't match the platform (glibc < 2.31), it falls back to compiling from source. No code changes needed on our side.

Alpine is already solved. better-sqlite3 v12.8.0 ships musl prebuilds (linuxmusl-x64, linuxmusl-arm64). Our ^12.6.2 range resolves to 12.8.0 on fresh install — Alpine just works.

CentOS/RHEL just needs build tools. The auto-fallback triggers, and compiling from source works with GCC 10+ (devtoolset-10 / gcc-toolset-10) and Python setuptools. We've verified the exact commands on CentOS 7 (glibc 2.17), CentOS 8 (glibc 2.28), and Alpine.

We've added a Build Prerequisites section to the README on the next branch (616c3e4) with verified install commands for each platform. All glibc versions, GCC versions, and package names were cross-referenced against distro repositories and npm registry data.

This avoids the version coupling risk (semver range + fixed prebuilt binary = potential ABI mismatch) and the ~6MB binary bloat from vendoring.

Thanks again for flagging this — the documentation wouldn't exist without your PR. 🙏

@mksglu mksglu closed this Mar 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants