Skip to content

Commit d75bafc

Browse files
jurerotargithub-actions[bot]tomayac
authored
feat: reworked build process and package structure (#131)
* feat: reworked build process and package structure (#3) * added a Dockerfile to create sqlite build environment * added a sqlite build script * added a GitHub Actions workflow which runs tests, check types and validates formatting * added a GitHub Actions workflow which builds Docker image, builds sqlite for wasm, extracts wasm file and bindings and creates a PR to main branch with the updated bindings and wasm file * added Vitest with browser tests for sahpool and oo1 and node tests for sqlite3-node * added tsdown for dead-code-elimination and basic build steps * added lefthook for formatting on push * fix: added sqlite3-opfs-async-proxy.js & sqlite3-worker1.mjs to /dist * chore: update SQLite Wasm binaries from master (9ab2a2af1bb1c7d9777b17d6893c5352ef87424f) (#4) Co-authored-by: jurerotar <28565137+jurerotar@users.noreply.github.com> * chore: update SQLite Wasm binaries from master (428c2656ff1b0023d06b600809227a099caecf2b) (#5) Co-authored-by: jurerotar <28565137+jurerotar@users.noreply.github.com> * chore: updated rolldown outputOptions.legalComments to preserve license and version comments * chore: removed fetch patching * chore: update SQLite Wasm binaries from master (bc2101b44ea60a737a5911aef970db9de6d0dc21) (#7) Co-authored-by: jurerotar <28565137+jurerotar@users.noreply.github.com> * chore: update SQLite Wasm binaries from master (35b9709d4675eb1ff9a8d51525b62f33e68e576f) (#8) Co-authored-by: jurerotar <28565137+jurerotar@users.noreply.github.com> * chore: added wasm-strip dependency to dockerfile * fix: fixed fetching sqlite by commit hash * chore: updated GitHub actions to make node tests run against all supported node versions * chore: cached Playwright browsers to prevent constant re-download * chore: update SQLite Wasm binaries from master (43eb1d4d22f1644a4fd05651301c03a64ecd2fd6) (#12) Co-authored-by: jurerotar <28565137+jurerotar@users.noreply.github.com> * chore: update SQLite Wasm binaries from master (90942efd218fe3e0a46e9d2c9a2afc194e214fba) (#13) Co-authored-by: jurerotar <28565137+jurerotar@users.noreply.github.com> * chore: update SQLite Wasm binaries from master (4b9def19035dfabe6272cb26e776d6ed2f55d517) (#14) Co-authored-by: jurerotar <28565137+jurerotar@users.noreply.github.com> * chore: update SQLite Wasm binaries from master (8c4e315e19a36afc85e018fc7de98ae91ff7967f) (#15) Co-authored-by: jurerotar <28565137+jurerotar@users.noreply.github.com> * chore: removed version checks from tests * chore: update .github/workflows/build-wasm.yml Co-authored-by: Thomas Steiner <tomac@google.com> * chore: update SQLite Wasm binaries from master (f4a613aa4631a6730ac0d8c31070cadf4a4a5905) (#16) Co-authored-by: jurerotar <28565137+jurerotar@users.noreply.github.com> --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Thomas Steiner <tomac@google.com>
1 parent d36dfac commit d75bafc

45 files changed

Lines changed: 65119 additions & 29834 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.editorconfig‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
root = true
2+
3+
[*]
4+
ij_html_space_inside_empty_tag = true
5+
end_of_line = lf
6+
insert_final_newline = true
7+
charset = utf-8
8+
indent_style = space
9+
indent_size = 2
10+
trim_trailing_whitespace = true
11+
12+
# quote style
13+
ij_javascript_force_quote_style = true
14+
ij_typescript_force_quote_style = true
15+
ij_javascript_use_double_quotes = false
16+
ij_typescript_use_double_quotes = false
17+
18+
# bracket spacing
19+
ij_javascript_spaces_within_object_literal_braces = true
20+
ij_typescript_spaces_within_object_literal_braces = true
21+
ij_javascript_spaces_within_object_type_braces = true
22+
ij_typescript_spaces_within_object_type_braces = true
23+
24+
# imports
25+
ij_javascript_spaces_within_imports = true
26+
ij_typescript_spaces_within_imports = true
27+
ij_javascript_import_merge_members = true
28+
ij_typescript_import_merge_members = true
29+
ij_javascript_import_sort_members = true
30+
ij_typescript_import_sort_members = true

‎.github/actions/node-setup/action.yml‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
inputs:
2+
node-version:
3+
description: 'Node.js version'
4+
required: false
5+
default: 'lts/*'
6+
7+
runs:
8+
using: 'composite'
9+
steps:
10+
- name: Setup Node.js
11+
uses: actions/setup-node@v6
12+
with:
13+
node-version: ${{ inputs.node-version }}
14+
cache: 'npm'
15+
cache-dependency-path: package-lock.json
16+
17+
- name: Install dependencies
18+
shell: bash
19+
run: |
20+
set -euo pipefail
21+
npm ci --prefer-offline --no-audit --no-fund

‎.github/workflows/build-wasm.yml‎

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
name: Build SQLite Wasm
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
sqlite_ref:
7+
description: 'SQLite reference (tag, branch, or commit, or "latest")'
8+
required: true
9+
default: 'master'
10+
11+
jobs:
12+
build:
13+
runs-on: ubuntu-latest
14+
steps:
15+
- name: Checkout repository
16+
uses: actions/checkout@v6
17+
with:
18+
persist-credentials: false
19+
20+
- name: Resolve SQLite reference
21+
id: resolve-ref
22+
run: |
23+
SQLITE_REF="${{ github.event.inputs.sqlite_ref }}"
24+
if [ "$SQLITE_REF" = "latest" ]; then
25+
echo "Fetching latest tag..."
26+
LATEST_TAG=$(git ls-remote --tags --sort="v:refname" https://github.com/sqlite/sqlite.git "refs/tags/version-*" | tail -n 1 | cut -f 2 | sed 's/refs\/tags\///')
27+
echo "Latest tag found: $LATEST_TAG"
28+
SQLITE_REF="$LATEST_TAG"
29+
fi
30+
31+
# Get the full commit SHA
32+
SQLITE_SHA=$(git ls-remote https://github.com/sqlite/sqlite.git "$SQLITE_REF" | head -n 1 | cut -f 1)
33+
if [ -z "$SQLITE_SHA" ]; then
34+
# If not found, maybe it's a tag that needs refs/tags/ prefix or it's already a SHA
35+
SQLITE_SHA=$(git ls-remote https://github.com/sqlite/sqlite.git "refs/tags/$SQLITE_REF" | head -n 1 | cut -f 1)
36+
fi
37+
38+
if [ -z "$SQLITE_SHA" ]; then
39+
# Fallback: assume it's a SHA if ls-remote didn't find it as a ref
40+
SQLITE_SHA="$SQLITE_REF"
41+
fi
42+
43+
echo "sqlite_ref=$SQLITE_REF" >> $GITHUB_OUTPUT
44+
echo "sqlite_sha=$SQLITE_SHA" >> $GITHUB_OUTPUT
45+
echo "branch_name=update-sqlite-wasm-${SQLITE_SHA}" >> $GITHUB_OUTPUT
46+
47+
- name: Check if branch exists
48+
id: check-branch
49+
run: |
50+
BRANCH_NAME="${{ steps.resolve-ref.outputs.branch_name }}"
51+
if git ls-remote --exit-code --heads origin "$BRANCH_NAME"; then
52+
echo "Branch $BRANCH_NAME already exists. Skipping build."
53+
echo "skip=true" >> $GITHUB_OUTPUT
54+
else
55+
echo "skip=false" >> $GITHUB_OUTPUT
56+
fi
57+
58+
- name: Set up Buildx Docker CLI plugin
59+
if: steps.check-branch.outputs.skip != 'true'
60+
uses: docker/setup-buildx-action@v3
61+
62+
- name: Build Docker image
63+
if: steps.check-branch.outputs.skip != 'true'
64+
uses: docker/build-push-action@v6
65+
with:
66+
context: .
67+
load: true
68+
tags: sqlite-wasm-builder:env
69+
cache-from: type=gha
70+
cache-to: type=gha,mode=max
71+
72+
- name: Run build
73+
if: steps.check-branch.outputs.skip != 'true'
74+
run: |
75+
mkdir -p out src/bin
76+
docker run --rm \
77+
-e SQLITE_REF="${{ steps.resolve-ref.outputs.sqlite_ref }}" \
78+
-e HOST_UID="$(id -u)" \
79+
-e HOST_GID="$(id -g)" \
80+
-v "$(pwd)/out":/out \
81+
-v "$(pwd)/src/bin":/src/bin \
82+
sqlite-wasm-builder:env build
83+
# Fallback fix for permissions if chown inside docker failed or was incomplete
84+
sudo chown -R $(id -u):$(id -g) out src/bin
85+
86+
- name: Check for changes
87+
if: steps.check-branch.outputs.skip != 'true'
88+
id: git-check
89+
run: |
90+
# Add files that might be newly created or modified
91+
git add src/bin
92+
git status --porcelain
93+
if [ -n "$(git status --porcelain)" ]; then
94+
echo "changes=true" >> $GITHUB_OUTPUT
95+
else
96+
echo "changes=false" >> $GITHUB_OUTPUT
97+
fi
98+
99+
- name: Create Pull Request
100+
if:
101+
steps.check-branch.outputs.skip != 'true' &&
102+
steps.git-check.outputs.changes == 'true'
103+
uses: peter-evans/create-pull-request@v6
104+
with:
105+
token: ${{ secrets.GITHUB_TOKEN }}
106+
commit-message:
107+
'chore: update SQLite Wasm binaries from ${{
108+
steps.resolve-ref.outputs.sqlite_ref }} (${{
109+
steps.resolve-ref.outputs.sqlite_sha }})'
110+
title:
111+
'chore: update SQLite Wasm binaries from ${{
112+
steps.resolve-ref.outputs.sqlite_ref }}'
113+
body: |
114+
This PR updates the SQLite Wasm binaries in `src/bin` by building them from SQLite reference `${{ steps.resolve-ref.outputs.sqlite_ref }}` (commit `${{ steps.resolve-ref.outputs.sqlite_sha }}`).
115+
116+
Triggered by manual workflow dispatch.
117+
branch: ${{ steps.resolve-ref.outputs.branch_name }}
118+
base: main
119+
delete-branch: true

‎.github/workflows/ci.yml‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
name: CI
2+
3+
permissions:
4+
contents: read
5+
pull-requests: read
6+
7+
on:
8+
push:
9+
branches: [main]
10+
pull_request:
11+
branches: [main]
12+
13+
concurrency:
14+
group: ${{ github.workflow }}-${{ github.ref }}
15+
cancel-in-progress: true
16+
17+
defaults:
18+
run:
19+
shell: bash
20+
21+
jobs:
22+
type-check:
23+
name: Type check
24+
runs-on: ubuntu-latest
25+
steps:
26+
- name: Checkout repo
27+
uses: actions/checkout@v6
28+
with:
29+
fetch-depth: 0
30+
31+
- name: Set up Node.js
32+
uses: ./.github/actions/node-setup
33+
34+
- name: Run type check
35+
run: npm run check-types
36+
37+
format:
38+
name: Format check
39+
runs-on: ubuntu-latest
40+
steps:
41+
- name: Checkout repo
42+
uses: actions/checkout@v6
43+
with:
44+
fetch-depth: 0
45+
46+
- name: Setup environment (node + install)
47+
uses: ./.github/actions/node-setup
48+
49+
- name: Run format check
50+
run: npx prettier . --check
51+
52+
test-browser:
53+
name: Run browser tests
54+
runs-on: ubuntu-latest
55+
steps:
56+
- name: Checkout repo
57+
uses: actions/checkout@v6
58+
with:
59+
fetch-depth: 0
60+
61+
- name: Setup environment (node + install)
62+
uses: ./.github/actions/node-setup
63+
64+
- name: Get Playwright version
65+
id: playwright-version
66+
run:
67+
echo "PLAYWRIGHT_VERSION=$(node -e
68+
"console.log(require('./package-lock.json').packages['node_modules/playwright'].version)")"
69+
>> $GITHUB_OUTPUT
70+
71+
- name: Cache Playwright browsers
72+
uses: actions/cache@v4
73+
id: playwright-cache
74+
with:
75+
path: ~/.cache/ms-playwright
76+
key:
77+
${{ runner.os }}-playwright-${{
78+
steps.playwright-version.outputs.PLAYWRIGHT_VERSION }}
79+
restore-keys: |
80+
${{ runner.os }}-playwright-
81+
82+
- name: Install Playwright browsers
83+
if: steps.playwright-cache.outputs.cache-hit != 'true'
84+
run: npx playwright install --with-deps
85+
86+
- name: Install Playwright dependencies
87+
if: steps.playwright-cache.outputs.cache-hit == 'true'
88+
run: npx playwright install-deps
89+
90+
- name: Run browser tests
91+
run: npm run test:browser
92+
93+
test-node:
94+
name: Run node tests (Node ${{ matrix.node-version }})
95+
runs-on: ubuntu-latest
96+
strategy:
97+
fail-fast: false
98+
matrix:
99+
node-version: [20, 22, 23, 24, 25]
100+
steps:
101+
- name: Checkout repo
102+
uses: actions/checkout@v6
103+
with:
104+
fetch-depth: 0
105+
106+
- name: Setup environment (node + install)
107+
uses: ./.github/actions/node-setup
108+
with:
109+
node-version: ${{ matrix.node-version }}
110+
111+
- name: Run node tests
112+
run: npm run test:node

‎.prettierignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
/src/bin
2+
/dist
3+
/node_modules

‎Dockerfile‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# syntax=docker/dockerfile:1
2+
FROM emscripten/emsdk:latest
3+
4+
ARG DEBIAN_FRONTEND=noninteractive
5+
6+
# runtime / build deps
7+
RUN apt-get update \
8+
&& apt-get install -y --no-install-recommends \
9+
ca-certificates \
10+
git \
11+
build-essential \
12+
autoconf \
13+
automake \
14+
libtool \
15+
m4 \
16+
python3 \
17+
nodejs \
18+
npm \
19+
zip \
20+
unzip \
21+
curl \
22+
wabt \
23+
&& rm -rf /var/lib/apt/lists/*
24+
25+
# create useful dirs
26+
RUN mkdir -p /build /out /src/bin /work
27+
28+
# copy helper script
29+
COPY scripts/build-sqlite3-wasm.sh /usr/local/bin/build-sqlite3-wasm.sh
30+
RUN chmod +x /usr/local/bin/build-sqlite3-wasm.sh
31+
32+
WORKDIR /work
33+
34+
ENTRYPOINT ["/usr/local/bin/build-sqlite3-wasm.sh"]
35+
CMD ["build"]

0 commit comments

Comments
 (0)