From 4ec36fd1f85348035523f3981ae4e3b628042466 Mon Sep 17 00:00:00 2001 From: andreatp Date: Thu, 5 Mar 2026 12:22:43 +0000 Subject: [PATCH 1/6] Tight up the repo --- .github/workflows/release.yml | 4 +- INVESTIGATION.md | 179 ---------- PLAN.md | 621 ---------------------------------- README.md | 59 ++-- core/pom.xml | 20 +- pom.xml | 1 - 6 files changed, 32 insertions(+), 852 deletions(-) delete mode 100644 INVESTIGATION.md delete mode 100644 PLAN.md diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index f0b315b..4ab00a7 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -21,7 +21,7 @@ jobs: steps: - name: Checkout - uses: actions/checkout@v6 + uses: actions/checkout@v4 - name: Build pglite WASM working-directory: wasm-build @@ -32,7 +32,7 @@ jobs: run: make unpack - name: Setup Java - uses: actions/setup-java@v5 + uses: actions/setup-java@v4 with: java-version: 11 distribution: 'temurin' diff --git a/INVESTIGATION.md b/INVESTIGATION.md deleted file mode 100644 index 6c4412c..0000000 --- a/INVESTIGATION.md +++ /dev/null @@ -1,179 +0,0 @@ -# pglite4j — Current Investigation Notes - -## Current State (2026-02-23) - -We are running **without wizer and without wasi-vfs**. The raw WASM binary -(~24 MB, linked without `libwasi_vfs.a`) is at `wasm-build/output/pglite.wasi`. -All PostgreSQL files (share/, lib/, bin/, base/) are provided via ZeroFS from -classpath resources at runtime. - -The Java init sequence calls `_start` (which runs `main()` → `main_pre()` → -returns because `REPL=N`), then `pgl_initdb()`, then `pgl_backend()`. - -**SELECT 1 used to work with the wizer'd binary. We are now debugging the -raw binary (no wizer) to get DDL (CREATE TABLE) working.** - ---- - -## RESOLVED: `pgl_backend()` now works (2026-02-23) - -The `pgl_backend()` call previously hit `abort()` → WASM trap during -initialization. This was fixed by: - -1. Removing the `--no-stack-first` linker flag (`build.sh.diff`) -2. Building without wizer and wasi-vfs (raw binary) -3. Adding debug prints in `pgl_mains.c.diff` and enhanced error reporting in - `elog.c.diff` - -### Current working output -``` -pgl_initdb returned 14 -PGLite: pgl_backend returned normally -SELECT 1 => 1 -CREATE TABLE works -INSERT + SELECT works -Tests run: 3, Failures: 0, Errors: 0, Skipped: 0 — BUILD SUCCESS -``` - -### Why we can't see the actual error message - -`whereToSendOutput` is `DestNone` during initialization (it only gets set to -`DestDebug` or `DestRemote` later). So `EmitErrorReport()` has nowhere to -send the message — it's silently lost. - ---- - -## Root Cause Analysis - -### Path taken through the C code - -1. `pgl_initdb()` finds existing PGDATA with `PG_VERSION` → sets `async_restart=0` -2. `pgl_backend()` with `async_restart=0` → calls `main_post()` then - `AsyncPostgresSingleUserMain()` (pgl_mains.c:264) -3. `AsyncPostgresSingleUserMain` does full backend init: `InitStandaloneProcess`, - `InitializeGUCOptions`, `SelectConfigFiles`, `checkDataDir`, - `CreateDataDirLockFile`, `LocalProcessControlFile`, - `CreateSharedMemoryAndSemaphores`, `InitProcess`, `BaseInit` -4. Something between `CreateSharedMemoryAndSemaphores` and `InitPostgres` - raises `ereport(ERROR, ...)` - -### Where the abort comes from - -File: `wasm-build/patches/postgresql-pglite/src-backend-utils-error-elog.c.diff` - -```c -// In errfinish(), when edata->elevel >= ERROR: -recursion_depth--; -#if defined(__wasi__) - fprintf(stderr, "# 547: PG_RE_THROW(ERROR : %d) ignored\r\n", recursion_depth); - abort(); // ← THIS CAUSES THE WASM TRAP -#else - PG_RE_THROW(); // ← Normal path: siglongjmp to error handler -#endif -``` - -### Why `pgl_sjlj.c` is disabled on WASI - -File: `wasm-build/pglite-wasm/pgl_sjlj.c` - -On WASI, the entire `sigsetjmp`/`siglongjmp` error handler is disabled: -```c -#if defined(__wasi__) - PDEBUG("# 2:" __FILE__ ": sjlj exception handler off"); -#else - if (sigsetjmp(local_sigjmp_buf, 1) != 0) { - // error recovery code... - } - PG_exception_stack = &local_sigjmp_buf; -#endif -``` - -This means `PG_exception_stack` is never set. When `PG_RE_THROW()` would -try `siglongjmp(*PG_exception_stack, 1)`, it would crash (NULL deref). -The `abort()` replaces this crash with a controlled trap. - ---- - -## Strategy: Fix Root Causes, NOT Re-enable Exception Handling - -**DO NOT re-enable sigsetjmp/siglongjmp exception handling** unless -absolutely necessary. Instead: - -1. Find which specific function raises the `ereport(ERROR)` during init -2. Fix the root cause so the error doesn't occur on the happy path -3. Adapt the C code to handle the situation gracefully - -Re-enabling exception handling would mask bugs and add complexity. The goal -is a clean happy path where no errors are raised during initialization. - ---- - -## How to Build / Test - -### Build the WASM binary (Docker) -```bash -cd wasm-build -./build.sh -make unpack -``` - -### Run the test -```bash -mvn test -pl core -Dtest="PGLiteTest#selectOne" -Dsurefire.useFile=false -``` - -### Rebuild just pglite.o + relink (faster iteration) - -The C files in `wasm-build/pglite-wasm/` are root-owned from Docker. -**Do not modify them directly.** Instead: - -1. Create patches in `wasm-build/patches/pglite-wasm/` (applied by - `docker/entrypoint.sh` during build) -2. Or override files through Docker commands - -To recompile just the pglite component: -```bash -docker run --rm \ - -v "$(pwd):/workspace:rw" \ - -v "$(pwd)/wasm-build/output/sdk-build:/tmp/sdk/build:rw" \ - -v "$(pwd)/wasm-build/output/sdk-dist:/tmp/sdk/dist:rw" \ - -v "$(pwd)/wasm-build/output/pglite:/tmp/pglite:rw" \ - --entrypoint bash pglite-wasi-builder:latest -c ' - . /tmp/sdk/wasisdk/wasisdk_env.sh - . /tmp/pglite/pgopts.sh - # ... compile and link commands ... - ' -``` - -### Key files - -| File | Description | -|------|-------------| -| `wasm-build/pglite-wasm/pg_main.c` | Main entry, includes all other .c files | -| `wasm-build/pglite-wasm/pgl_mains.c` | `AsyncPostgresSingleUserMain`, `RePostgresSingleUserMain` | -| `wasm-build/pglite-wasm/pgl_sjlj.c` | Error handler (disabled on WASI) | -| `wasm-build/pglite-wasm/interactive_one.c` | Wire protocol handling, `interactive_one()` | -| `wasm-build/patches/postgresql-pglite/src-backend-utils-error-elog.c.diff` | Error reporting patch | -| `core/src/main/java/.../PGLite.java` | Java init sequence | - ---- - -## Next Steps - -1. ~~Find and fix root cause of `ereport(ERROR)` during init~~ **DONE** — removed `--no-stack-first` -2. ~~Get CREATE TABLE working~~ **DONE** — all 3 tests pass -3. Revisit wizer + wasi-vfs integration for faster startup -4. Expand JDBC driver coverage (more SQL types, prepared statements, etc.) -5. Clean up debug prints once stable - ---- - -## Previous Findings (wasi-vfs) - -- wasi-vfs intercepts ALL WASI filesystem calls regardless of which - directories are mapped — it's all-or-nothing, never delegates to host WASI -- Specific subdirectory mapping (`--dir share::/tmp/pglite/share`) does NOT - help — `is_embed=1` confirms global interception -- This blocks PGDATA access (ENOENT for `/tmp/pglite/base/global/pg_control`) -- Three potential fixes documented in PLAN.md; currently bypassed by not - linking `libwasi_vfs.a` diff --git a/PLAN.md b/PLAN.md deleted file mode 100644 index 1768eef..0000000 --- a/PLAN.md +++ /dev/null @@ -1,621 +0,0 @@ -# pglite4j — Embedded PostgreSQL for Java via WebAssembly - -## Vision - -A Java library that embeds PostgreSQL (via PGlite compiled to WASI) so that any Java project can add it as a **test dependency** and get a real PostgreSQL instance without spinning up containers or external processes. Standard JDBC drivers (pgjdbc) connect to it transparently. - -```xml - - com.dylibso.chicory - pglite4j - ... - test - -``` - -Then just use `jdbc:pglite:` as your connection URL — that's it: - -```java -// Zero-config: the driver auto-starts an embedded PostgreSQL instance -Connection conn = DriverManager.getConnection("jdbc:pglite:memory://"); -conn.createStatement().execute("CREATE TABLE foo (id serial, name text)"); - -// Spring Boot — just set the URL in application-test.properties: -// spring.datasource.url=jdbc:pglite:memory:// -``` - -No `try`-with-resources, no annotations, no extensions. The custom JDBC driver: -1. Auto-registers via `META-INF/services/java.sql.Driver` (ServiceLoader) -2. On first `connect()`, boots PGlite WASM, opens a local socket, performs the PG handshake -3. Delegates to pgjdbc pointing at `localhost:` -4. Reuses the same PGlite instance for subsequent connections to the same URL -5. Shuts down via JVM shutdown hook - ---- - -## Proof of Concept — What's Already Working - -A working test exists on branch `pglite-ai-support` of chicory2: - -- **File:** `chicory2/machine-tests/src/test/java/com/dylibso/chicory/testing/MachinesTest.java` (lines 266-634) -- **Test:** `shouldRunPGLite` - -### What the test does - -1. Extracts `pglite-wasi.tar.xz` (24 MB PostgreSQL 17 compiled to WASI) -2. Sets up an in-memory filesystem (ZeroFs) with PGDATA, config, extensions -3. Instantiates the WASM module via Chicory (AOT-compiled to JVM bytecode) -4. Calls `pgl_initdb()` to create the database cluster -5. Calls `pgl_backend()` to initialize the backend (traps on OpenPipeStream — expected, safe to ignore) -6. Enables wire protocol via `use_wire(1)` -7. Performs the full PostgreSQL wire protocol handshake (StartupMessage, MD5 auth, ReadyForQuery) -8. Sends `SELECT 1;` as a wire protocol Query message -9. Receives and parses the DataRow response via CMA shared memory - -### How to run it - -```bash -cd chicory2 - -# 1. Build core modules -./mvnw install -pl annotations/annotations,annotations/processor,wasm,runtime,wasi,compiler,log,wasm-tools,wasm-corpus,compiler-maven-plugin -DskipTests - -# 2. Run the PGLite test -./mvnw test -pl machine-tests -Dtest=MachinesTest#shouldRunPGLite -``` - -### WASM archive location - -``` -/home/andreatp/workspace/pglite4j/wasm-build/output/sdk-dist/pglite-wasi.tar.xz -``` - -Contains: `pglite.wasi` (24 MB), postgres/initdb binaries, share/postgresql configs, extensions (plpgsql, dict_snowball), timezone data. - ---- - -## Key Insight: PGlite Speaks the Wire Protocol - -PGlite **does** implement the PostgreSQL wire protocol natively inside the WASM module. The bytes going in and out are standard PostgreSQL v3 protocol messages. This means a Java socket server only needs to act as a **transparent byte shuttle** — no protocol parsing required in the Java layer. - -### WASM Exports - -| Export | Signature | Description | -|--------|-----------|-------------| -| `pgl_initdb()` | `() -> i32` | Create database cluster | -| `pgl_backend()` | `() -> void` | Initialize backend (may trap — expected) | -| `use_wire(state)` | `(i32) -> void` | Enable (1) or disable (0) wire protocol mode | -| `interactive_write(size)` | `(i32) -> void` | Signal input message size; clears output (`cma_wsize=0`) | -| `interactive_read()` | `() -> i32` | Returns output message size (`cma_wsize`) | -| `interactive_one()` | `() -> void` | Process one interaction cycle | -| `get_channel()` | `() -> i32` | Get transport mode: `>=0` = CMA, `<0` = file | -| `get_buffer_addr(fd)` | `(i32) -> i32` | Buffer address: `1 + (buffer_size * fd)` | -| `get_buffer_size(fd)` | `(i32) -> i32` | Buffer size: `(CMA_MB * 1024 * 1024) / CMA_FD` | -| `clear_error()` | `() -> void` | Clear PostgreSQL error state, reset transaction | -| `pgl_shutdown()` | `() -> void` | Shutdown PostgreSQL | -| `pgl_closed()` | `() -> i32` | Check if closed | - -### CMA (Channel Memory Access) Communication Flow - -``` -JDBC Driver <--> Java Socket <--> WASM Linear Memory <--> PostgreSQL (in WASM) - - wireSendCma: - 1. use_wire(1) - 2. memory.write(addr, rawBytes) - 3. interactive_write(rawBytes.length) - - interactive_one() <-- process the message - - wireRecvCma: - 1. len = interactive_read() - 2. response = memory.read(addr + pendingLen + 1, len) - 3. interactive_write(0) <-- clear state -``` - -**Memory layout:** -- Input written at: `buffer_addr` (where `buffer_addr = 1 + (buffer_size * fd)`) -- Output read from: `buffer_addr + pending_wire_len + 1` -- Buffer size: 12 MB (`CMA_MB=12`, `CMA_FD=1`) -- Max single message: 16 KB (`FD_BUFFER_MAX=16384`) — larger messages overflow to file transport - -### CMA vs File Transport - -Both modes use identical PostgreSQL wire protocol bytes; they differ only in how bytes move between host and WASM: - -| Mode | When | How | -|------|------|-----| -| **CMA** (`channel >= 0`) | Messages fit in buffer | Direct WASM memory read/write | -| **File** (`channel < 0`) | CMA buffer overflow | Pseudo-socket files: `.s.PGSQL.5432.in` / `.out` | - -The C code in `interactive_one.c` (lines 669-712) determines which mode to use based on whether `sockfiles` is true and whether `SOCKET_DATA > 0`. - ---- - -## Reference Implementations - -### 1. pglite-oxide (Rust) - -**Location:** `/home/andreatp/workspace/pglite-oxide/src/interactive.rs` - -The Rust reference implementation is the most complete and closest to what pglite4j should do. - -**Key types:** -```rust -enum Transport { - Cma { pending_wire_len: usize }, - File { sinput: PathBuf, slock: PathBuf, cinput: PathBuf, clock: PathBuf }, -} -``` - -**Key functions:** - -- `InteractiveSession::new()` (lines 582-690) — Full init sequence: `_start()`, `pgl_initdb()`, `pgl_backend()`, get channel/buffer, select transport -- `send_wire(payload)` (lines 746-778) — Write to WASM memory (CMA) or file, call `interactive_write(len)` -- `try_recv_wire()` (lines 780-798) — Dispatcher: CMA or file receive -- `try_recv_wire_cma()` (lines 800-852) — Read from `buffer_addr + pending + 1`, validate bounds, clear state via `interactive_write(0)` -- `forward_wire(payload)` (lines 886-903) — Main loop: send, then `collect_replies()` / `run_once()` up to 256 ticks until no more data -- `drain_wire()` (lines 865-884) — Drain pending data (128 ticks) -- `ensure_handshake()` (lines 968-988) — Full startup: clear pending, send StartupMessage, handle auth (MD5/cleartext), wait for ReadyForQuery ('Z') -- `collect_replies()` — Calls `try_recv_wire()` and appends to reply vec - -**forward_wire pattern (the core loop):** -```rust -fn forward_wire(&mut self, payload: &[u8]) -> Result>> { - if !payload.is_empty() { - self.use_wire(true)?; - self.send_wire(payload)?; - } - let mut replies = Vec::new(); - for _ in 0..256 { - let produced_before = self.collect_replies(&mut replies)?; - self.run_once()?; - let produced_after = self.collect_replies(&mut replies)?; - if !produced_before && !produced_after { break; } - } - Ok(replies) -} -``` - -### 2. pglite-socket (TypeScript) - -**Location:** `/home/andreatp/workspace/pglite/packages/pglite-socket/src/index.ts` - -A Node.js TCP server that bridges sockets to PGlite. This is the exact pattern pglite4j should replicate in Java. - -**Core of `PGLiteSocketHandler.handleData()`** (line 185): -```typescript -const result = await this.db.execProtocolRaw(new Uint8Array(data)); -this.socket.write(Buffer.from(result)); -``` - -That's it — receive raw bytes from socket, pass to PGlite, write response back. The `execProtocolRaw()` method internally handles CMA/file transport, the `interactive_write`/`interactive_one`/`interactive_read` cycle, and the handshake. - -**`PGLiteSocketServer`** manages: -- TCP `ServerSocket` on configurable host:port or Unix socket -- Connection queuing (PGlite is single-connection, like SQLite) -- Exclusive locking via `db.runExclusive()` -- Timeout for queued connections (default 60s) - -**Server script** (`/home/andreatp/workspace/pglite/packages/pglite-socket/src/scripts/server.ts`): -```bash -pglite-server --db memory:// --port 5432 --host 127.0.0.1 -``` - -### 3. PGlite C internals - -**Location:** `/home/andreatp/workspace/pglite/postgres-pglite/pglite-wasm/interactive_one.c` - -Key details from the C code: - -- `interactive_write(size)` sets `cma_rsize = size` and clears `cma_wsize = 0` -- `interactive_read()` returns `cma_wsize` (the output size) -- `use_wire(state)` toggles between wire mode (`is_wire=true, is_repl=false`) and REPL mode -- `interactive_one()` handles: - - Startup packet detection (`peek == 0` -> `startup_auth()`) - - Password packet detection (`peek == 112` aka `'p'` -> `startup_pass(true)`) - - Normal wire messages via `SocketBackend(inBuf)` - - ReadyForQuery emission - - CMA vs file output routing -- Auth uses hardcoded MD5 salt: `{0x01, 0x23, 0x45, 0x56}` — password checking is effectively skipped (`recv_password_packet` reads it but the TODO for `CheckMD5Auth` is not implemented) -- Channel assignment after processing: `channel = cma_rsize + 2` for CMA, `channel = -1` for file transport - ---- - -## Architecture Plan - -### Single Module - -``` -pglite4j/ - src/main/java/com/dylibso/pglite4j/ - PgLiteDriver.java # JDBC Driver — the public API (just this) - src/main/resources/ - META-INF/services/java.sql.Driver - pglite-wasi.tar.xz # bundled PostgreSQL WASM binary - wasm-build/ # PGlite WASI build scripts (already exists) -``` - -Everything is internal to the driver. No separate modules, no public API beyond the JDBC URL. The WASM lifecycle, CMA communication, TCP socket bridge — all private implementation details inside `PgLiteDriver`. - -### How it works - -``` -DriverManager.getConnection("jdbc:pglite:memory://") - │ - ▼ -PgLiteDriver.connect(url, props) - │ - ├── First call for this URL? - │ ├── Extract pglite-wasi.tar.xz from classpath to temp dir - │ ├── Boot WASM instance (pgl_initdb, pgl_backend, handshake) - │ ├── Start internal ServerSocket on auto-selected free port - │ ├── Register JVM shutdown hook for cleanup - │ └── Cache instance keyed by URL - │ - ├── Delegate to pgjdbc: - │ return pgjdbcDriver.connect( - │ "jdbc:postgresql://localhost:/template1", - │ props - │ ); - │ - └── Subsequent calls → reuse cached instance, just delegate to pgjdbc -``` - -**URL scheme:** `jdbc:pglite:` -- `jdbc:pglite:memory://` — in-memory database (fresh each JVM) -- `jdbc:pglite:/tmp/mydb` — file-backed persistent data directory (future) - -> **TODO:** Currently only `memory://` is supported. We want to also support file-backed PostgreSQL where the PGDATA directory is persisted to disk, so that data survives JVM restarts. This would require changes to how ZeroFS is configured (or using real filesystem paths instead of ZeroFS for PGDATA) and potentially adjusting the wizer snapshot to handle pre-existing PGDATA on startup. - -**ServiceLoader registration:** `META-INF/services/java.sql.Driver` containing: -``` -com.dylibso.pglite4j.PgLiteDriver -``` - -**Driver sketch:** - -```java -public class PgLiteDriver implements java.sql.Driver { - - static { - try { - DriverManager.registerDriver(new PgLiteDriver()); - } catch (SQLException e) { - throw new RuntimeException(e); - } - } - - private static final ConcurrentHashMap instances = new ConcurrentHashMap<>(); - - static { - Runtime.getRuntime().addShutdownHook(new Thread(() -> - instances.values().forEach(ManagedInstance::close) - )); - } - - @Override - public boolean acceptsURL(String url) { - return url != null && url.startsWith("jdbc:pglite:"); - } - - @Override - public Connection connect(String url, Properties info) throws SQLException { - if (!acceptsURL(url)) return null; - String dataPath = url.substring("jdbc:pglite:".length()); - ManagedInstance managed = instances.computeIfAbsent(dataPath, ManagedInstance::boot); - return DriverManager.getConnection(managed.getJdbcUrl(), info); - } - - // --- everything below is private implementation detail --- - - private static class ManagedInstance { - // Bundles: WASM instance + CMA bridge + TCP socket server - // All the code from PgLiteInstance/PgLiteServer lives here as private methods - } -} -``` - -**Usage — just swap the JDBC URL:** - -```java -// Plain JDBC -Connection conn = DriverManager.getConnection("jdbc:pglite:memory://"); - -// Spring Boot (application-test.properties) -spring.datasource.url=jdbc:pglite:memory:// - -// HikariCP -HikariConfig config = new HikariConfig(); -config.setJdbcUrl("jdbc:pglite:memory://"); -DataSource ds = new HikariDataSource(config); - -// Quarkus (application.properties) -quarkus.datasource.jdbc.url=jdbc:pglite:memory:// - -// Flyway, Liquibase, jOOQ, MyBatis — all work transparently -``` - ---- - -## Implementation Steps - -### Phase 1: Get the driver working end-to-end - -- [x] Create Maven project with chicory dependencies (core module exists) -- [x] Move the wire protocol / CMA logic into `PGLite.java` + `PgWireCodec.java` -- [x] Implement `execProtocolRaw()` following the pglite-oxide pattern -- [x] Wire init sequence: extract pgdata, setup ZeroFS, WASM instance, `pgl_initdb`, `pgl_backend`, handshake -- [x] Bundle pgdata as classpath resources, auto-extract to ZeroFS -- [x] Test: `PGLite.query("SELECT 1")` works end-to-end -- [x] WASM build pipeline: wasmtime + wizer + wasi-vfs + wasm-opt (single `make build`) -- [ ] **JDBC driver**: `PgLiteDriver` class with `jdbc:pglite:` URL, ServiceLoader registration -- [ ] **JDBC driver**: Internal `ServerSocket` on auto-selected free port, raw byte pass-through -- [ ] **JDBC driver**: Delegation to pgjdbc for the actual JDBC implementation -- [ ] **JDBC driver**: Lazy singleton boot, JVM shutdown hook cleanup -- [ ] Test: `DriverManager.getConnection("jdbc:pglite:memory://")` -> `SELECT 1` via pgjdbc - -### Phase 2: Validate with real-world usage - -- [ ] Test with `psql` connecting to the internal socket -- [ ] Test with HikariCP connection pool -- [ ] Test with Flyway migrations -- [ ] Handle CMA overflow -> file transport fallback -- [ ] Connection queuing (serialize access — PGlite is single-connection) - -### Phase 3: Polish - -- [ ] Error handling & recovery (`clear_error()`) -- [ ] Performance: AOT compilation of the WASM module for fast startup -- [ ] Consider: bypass TCP entirely with a custom `java.net.SocketImpl` that calls `forwardWire` in-process - ---- - -## Open Questions - -1. **File transport fallback:** The current test only uses CMA. Need to verify file transport works when messages exceed the CMA buffer (16 KB single message / 12 MB total). The pglite-oxide code handles this transparently — we should too. - -2. **Socket read framing:** When reading from the TCP socket, we need to figure out when a complete PG message has arrived before passing it to PGlite. The wire protocol has length-prefixed messages, so we may need minimal framing logic (read tag byte + 4-byte length, then read that many bytes). Alternatively, we might be able to pass partial data and let PGlite handle buffering internally — needs investigation. The TypeScript `pglite-socket` seems to just pass whatever bytes arrive from the socket directly to `execProtocolRaw`, suggesting PGlite handles partial reads internally. - -3. **Concurrency model:** PGlite is single-threaded/single-connection. The socket server should accept connections sequentially. For test use cases this is fine, but worth documenting. - -4. **WASM binary size & startup time:** The 24 MB WASM binary compiles to JVM bytecode via Chicory's AOT compiler. Some functions exceed JVM method size limits and fall back to the interpreter. Need to measure startup time and consider caching the compiled module. - -5. **Extensions:** PGlite ships with plpgsql and dict_snowball. Adding more extensions means rebuilding the WASM binary. - ---- - -## File References - -| What | Path | -|------|------| -| Working Java test | `chicory2/machine-tests/src/test/java/com/dylibso/chicory/testing/MachinesTest.java:266-634` | -| Build/run instructions | `chicory2/machine-tests/PGLITE-STATUS.md` | -| WASM binary archive | `pglite4j/wasm-build/output/sdk-dist/pglite-wasi.tar.xz` | -| PGlite C internals | `pglite/postgres-pglite/pglite-wasm/interactive_one.c` | -| pglite-oxide (Rust ref) | `pglite-oxide/src/interactive.rs` | -| pglite-socket (TS ref) | `pglite/packages/pglite-socket/src/index.ts` | -| pglite-socket server | `pglite/packages/pglite-socket/src/scripts/server.ts` | - -All paths relative to `/home/andreatp/workspace/`. - - -Additional notes: - -- Try to keep all the resources self-contained in the JAR -- use ZeroFS instead of the tmp folder so that everything is fully portable -- Automate the unzip via `make unpack` in wasm-build/, and find a way to automate including needed files in the bundle -- Follow sqlite4j code structure when in doubt — keep it familiar -- follow up: optimize startup times with wasm-opt and wizer, and try wasi-sdk? - ---- - -## Design Decisions - -### Embedded resources — no external pgDistDir - -All PGlite resources (WASM binary, share/, extensions) must be embedded -in the JAR. There is no user-facing configuration for a distribution -directory. The builder takes no path argument — everything is loaded from -the classpath at build time (via Chicory's AOT compiler) and from -embedded resources at runtime (via ZeroFS populated from classpath). - ---- - -## Startup Optimization Plan — wasi-vfs + wizer - -### Problem - -Current startup is slow (~58 s in tests) because: -1. Filesystem setup: copying the PG distribution into ZeroFS at runtime -2. `pgl_initdb()`: creates the database cluster from scratch every boot -3. `pgl_backend()`: initializes the PostgreSQL backend -4. Wire protocol handshake (auth, ReadyForQuery) - -Steps 1-4 are identical every time for a fresh in-memory database. They -can all be done once at build time and snapshotted. - -### Solution: two build-time tools - -#### 1. wasi-vfs — embed the filesystem into the WASM binary - -**What:** [wasi-vfs](https://github.com/kateinoigakukun/wasi-vfs) packs -a host directory tree into a WASM binary as a read-only virtual -filesystem. At runtime the WASM module sees normal files via WASI -fd_read/path_open — no host directory mapping needed. - -**How it helps:** Eliminates the runtime ZeroFS copy of share/, -extensions, timezone data, config files. These become part of the WASM -binary itself. - -**Build step:** -```bash -wasi-vfs pack pglite.wasi \ - --mapdir /tmp/pglite::wasm-build/output/tmp/pglite \ - -o pglite-packed.wasi -``` - -**Constraint:** wasi-vfs produces a read-only filesystem. PGDATA (the -mutable database cluster) still needs a writable filesystem at runtime -(ZeroFS). This is fine — wasi-vfs handles the static resources, ZeroFS -handles the mutable PGDATA. - -#### 2. wizer — pre-initialize the WASM module - -**What:** [Wizer](https://github.com/bytecodealliance/wizer) executes a -WASM module's initialization function, then snapshots the entire memory -state (globals, linear memory) into a new WASM binary. The next -instantiation starts from the warm snapshot. - -**How it helps:** Runs `pgl_initdb()`, `pgl_backend()`, and the wire -protocol handshake once at build time. The resulting binary boots -directly into a ReadyForQuery state. - -**Build step:** -```bash -wizer pglite-packed.wasi \ - --allow-wasi \ - --init-func _start \ - --keep-init-func false \ - -o pglite-initialized.wasi -``` - -This requires an initialization entry point in PGlite that: -1. Calls `pgl_initdb()` -2. Calls `pgl_backend()` (catches the expected trap) -3. Enables wire mode, performs the handshake -4. Returns — wizer snapshots everything at this point - -### Reference implementations - -| Project | What they do | Reference | -|---------|-------------|-----------| -| **trino-wasm-python** | wasi-vfs packs Python stdlib into WASM, then wizer snapshots the initialized CPython interpreter. Pipeline: compile → `wasi-vfs pack` → `wizer` → deploy. Uses `--allow-wasi --init-func _start --keep-init-func false`. | [trinodb/trino-wasm-python](https://github.com/trinodb/trino-wasm-python) | -| **lumis4j** | Rust syntax highlighter compiled to WASM. Wizer pre-loads all language parsers via `wizer.initialize()` export. Then Chicory AOT compiles to JVM bytecode. Pipeline: `cargo build` → `wizer --allow-wasi` → `wasm-opt -Oz` → `chicory-compiler-maven-plugin`. | [roastedroot/lumis4j](https://github.com/roastedroot/lumis4j) | - -### Proposed build pipeline - -``` -1. Docker build (existing) - └── pglite.wasi (24 MB, raw) - -2. wasi-vfs pack - ├── Embed: share/postgresql/, lib/postgresql/, password, timezonesets - ├── Mount point: /tmp/pglite - └── Output: pglite-packed.wasi - -3. wizer pre-initialize - ├── Run: pgl_initdb + pgl_backend + handshake - ├── Snapshot: warm memory state with initialized cluster - └── Output: pglite-initialized.wasi - -4. wasm-opt (optional, binaryen) - ├── Flags: -Oz --strip-debug - └── Output: pglite-opt.wasi (size-optimized) - -5. Chicory AOT compile (existing, maven plugin) - ├── Input: pglite-opt.wasi - └── Output: PGLiteModule.java (JVM bytecode) -``` - -### Impact on Java code - -With wasi-vfs + wizer, the Java `PGLite` constructor simplifies to: -- No more archive extraction -- No more copying files into ZeroFS (static resources are in the binary) -- ZeroFS only needed for PGDATA (mutable, writable) -- No `pgl_initdb()` / `pgl_backend()` / handshake calls — already done -- Constructor just creates the Instance and it's ready for queries - -### Status: WORKING — wasi-vfs + wizer + wasmtime + wasm-opt - -The full build pipeline is implemented and working in a single -`make build` run. The key insight was separating initdb (which traps -via `pg_proc_exit(66)`) from the wizer snapshot step. - -**Build pipeline (in Docker):** -1. Compile PostgreSQL to WASI → `pglite.wasi` -2. **wasmtime pre-init**: runs `main()` with `INITDB_ONLY=1` to - populate `/pgdata` via initdb. `pg_proc_exit(66)` terminates - wasmtime gracefully (unlike wizer which needs a clean return). -3. **wizer snapshot**: runs `wizer_initialize()` which detects - `/pgdata/PG_VERSION`, skips initdb, calls `pgl_backend()`, and - returns cleanly. Wizer snapshots the warm memory state. -4. **wasi-vfs pack**: embeds `/tmp/pglite/share` and `/tmp/pglite/lib` - as read-only filesystem inside the WASM binary. -5. **wasm-opt**: optimizes with `${WASM_OPT_FLAGS}` (default `-Oz --strip-debug`). -6. **tar.xz**: packages `pglite.wasi` + `/pgdata` into archive. - -**PGDATA handling**: wasi-vfs only embeds `share/` and `lib/` (read-only -PostgreSQL distribution files). The mutable PGDATA cluster is NOT -embedded in wasi-vfs — it's packaged separately in the tar.xz archive -and provided at runtime via ZeroFS. This avoids the earlier problem -where wasi-vfs intercepted all filesystem calls including PGDATA paths. - -**TODO — verify DDL/DML**: The wasi-vfs approach now only embeds -`share/` and `lib/` subdirectories (not the whole `/tmp/pglite` tree). -Need to verify that DDL (CREATE TABLE, etc.) works correctly with -PGDATA served via ZeroFS while share/lib come from wasi-vfs. - -### Hardcoded variables - -The following values are hardcoded in three places and **must stay in -sync**: `pg_main.c` (`wizer_initialize`), `PGLite.java` (constants), -and `build.sh` (wizer `env -` block). - -| Variable | Value | Used by | -|----------|-------|---------| -| `PREFIX` | `/tmp/pglite` | Install prefix for PG binaries, share, libs | -| `PGDATA` | `/pgdata` | Mutable database cluster directory (wizer uses `/pgdata`, Java may use `/pgdata` or `/tmp/pglite/base`) | -| `PGUSER` | `postgres` | Superuser name | -| `PGDATABASE` | `template1` | Default database | -| `PATH` | `/tmp/pglite/bin` | Binary search path | -| `PGSYSCONFDIR` | `/tmp/pglite` | Server config directory | -| `PGCLIENTENCODING` | `UTF8` | Client encoding | -| `LC_CTYPE` | `en_US.UTF-8` | Locale for character classification | -| `TZ` / `PGTZ` | `UTC` | Timezone | -| `REPL` | `N` | Disable interactive REPL | -| `ENVIRONMENT` | `wasm32_wasi_preview1` | WASI platform identifier | -| `MODE` | `REACT` | PGlite operational mode | - -**Why hardcoded:** The `WASM_PREFIX` macro in `wasm_common.h` has a bug -where `#undef PG_PREFIX` runs before lazy macro expansion of -`WASM_PREFIX`, causing it to resolve to the literal string `"PG_PREFIX"` -instead of `"/tmp/pglite"`. Additionally, wizer doesn't forward host -environment variables to the WASI module by default (requires -`--inherit-env true`). Hardcoding avoids both issues. - -### Key implementation details - -- **wizer replaces `_start` with `unreachable`**: After pre-initialization, - `_start`/`main()` must not be called. Java uses `withStart(false)` on - the Chicory Instance.Builder. -- **Memory size**: The wizer snapshot expands initial memory to ~4029 pages - (~252 MB). Java uses `withMemoryLimits(new MemoryLimits(4029))`. -- **8 functions use interpreter fallback**: Functions that exceed JVM method - size limits fall back to the Chicory interpreter via - `WARN`. -- **wizer needs writable `/tmp`**: The initdb `pgl_popen` writes to - `/tmp/initdb.boot.txt` and `/tmp/initdb.single.txt`. The wizer - invocation maps `--mapdir /tmp::/tmp/wizer-tmp` for this. - -### Resolved open questions - -1. **wizer + PGlite compatibility:** Resolved. `pgl_backend()` does NOT - trap (the `proc_exit(66)` is a fake shutdown that returns cleanly). - A `wizer.initialize` export was added to `pg_main.c` that runs the - full init sequence: `pgl_initdb()`, `pgl_backend()`, - `interactive_write(0)`, `interactive_one()`. - -2. **PGDATA separation:** Resolved. wasi-vfs embeds the static - distribution (share/, lib/, password). The writable PGDATA is - provided at runtime via ZeroFS. The wizer snapshot includes a - pre-initialized PGDATA that is also packed into the tar archive. - -3. **wasi-vfs + wasi-sdk compatibility:** Resolved. wasi-sdk 25 + - wasi-vfs 0.6.2 work together. `libwasi_vfs.a` is linked at compile - time, and `wasi-vfs pack` embeds the filesystem post-link. - -4. **Binary size:** The wizer-initialized binary is ~126 MB (includes - the full memory snapshot). After wasm-opt, this is reduced somewhat. - The tar.xz archive compresses well. diff --git a/README.md b/README.md index 02f6a25..3ce8e97 100644 --- a/README.md +++ b/README.md @@ -3,21 +3,11 @@ Embedded PostgreSQL in plain Java bytecode. No containers, no native binaries, no external processes: just add a dependency and use a JDBC URL. > **⚠️ Warning** -> This project is highly experimental and in a very early stage of development. APIs may change, features are incomplete, and it is not yet recommended for production use. Feedback and contributions are very welcome! +> PGLite is a patched version of PostgreSQL, which was not designed for embedding. Do not use it in production! This driver is intended for testing and demo applications only. ## How it works -`pglite4j` bundles a full PostgreSQL 17 instance compiled to WebAssembly (WASI) and runs it directly inside the JVM via [Chicory](https://github.com/dylibso/chicory) (a pure-Java WebAssembly runtime). The JDBC driver opens an internal loopback socket and transparently bridges TCP to the WASM module's CMA (Contiguous Memory Allocator) shared memory, no network traffic ever leaves the process. - -The build pipeline that produces the WASM binary runs inside Docker and chains several tools: - -1. **wasi-sdk** — cross-compiles PostgreSQL + [PGlite](https://github.com/electric-sql/postgres-pglite) patches to a WASI target -2. **wasmtime** — runs `initdb` to create the database cluster -3. **Wizer** — snapshots the fully initialized PostgreSQL state (post-initdb, post-backend-start) so runtime startup skips all of that -4. **wasi-vfs** — embeds the read-only PostgreSQL distribution (`share/`, `lib/`) directly into the WASM binary -5. **wasm-opt** — optimizes the final binary for size - -At build-time the Chicory compiler translates the WASM module to JVM bytecode. The JDBC driver (`PgLiteDriver`) opens a `ServerSocket` on a random loopback port, delegates to [pgjdbc](https://jdbc.postgresql.org/), and acts as a transparent byte shuttle between the TCP socket and PostgreSQL's wire protocol on the WASM memory. +`pglite4j` bundles a full PostgreSQL 17 instance compiled to WebAssembly (WASI) and runs it directly inside the JVM via [Chicory](https://github.com/dylibso/chicory) (a pure-Java WebAssembly runtime). The JDBC driver opens an internal loopback socket and transparently bridges TCP to PostgreSQL's wire protocol running in WASM linear memory — no network traffic ever leaves the process. ``` DriverManager.getConnection("jdbc:pglite:memory://") @@ -30,7 +20,7 @@ ServerSocket(127.0.0.1:) | ^ | | raw PG wire protocol bytes v | - CMA shared memory <──> PostgreSQL (in WASM linear memory) + WASM shared memory <──> PostgreSQL (in WASM linear memory) | v pgjdbc connects to localhost: ──> returns java.sql.Connection @@ -53,7 +43,6 @@ Add the JDBC driver dependency: ### Plain JDBC ```java -// In-memory (ephemeral) — data is lost when the JVM exits Connection conn = DriverManager.getConnection("jdbc:pglite:memory://"); conn.createStatement().execute("CREATE TABLE demo (id serial PRIMARY KEY, name text)"); conn.createStatement().execute("INSERT INTO demo (name) VALUES ('hello')"); @@ -61,9 +50,9 @@ conn.createStatement().execute("INSERT INTO demo (name) VALUES ('hello')"); ### Persistent storage -Point the JDBC URL to a file path and `pglite4j` will periodically snapshot the entire in-memory database to a zip file on disk. On the next JVM startup, the database is restored from that snapshot. +Point the JDBC URL to a file path and `pglite4j` will periodically snapshot the in-memory database to a zip file on disk. On the next JVM startup, the database is restored from that snapshot. -> **Note:** This is **not** traditional disk-backed storage. PostgreSQL runs entirely in memory (ZeroFS). The driver takes periodic snapshots (backup/restore), similar to Redis RDB persistence. Data written between the last snapshot and a crash will be lost. This is suitable for demo apps, prototyping, and development — not for production workloads that require durability guarantees. +> **Note:** This is **not** traditional disk-backed storage. PostgreSQL runs entirely in memory. The driver takes periodic snapshots (backup/restore), similar to Redis RDB persistence. Data written between the last snapshot and a crash will be lost. ```java // File-backed — data survives JVM restarts @@ -93,7 +82,6 @@ quarkus.datasource.db-kind=postgresql quarkus.datasource.jdbc.url=jdbc:pglite:memory:// # or persistent: jdbc:pglite:/var/data/myapp.zip quarkus.datasource.jdbc.driver=io.roastedroot.pglite4j.jdbc.PgLiteDriver -quarkus.datasource.jdbc.max-size=5 quarkus.devservices.enabled=false ``` @@ -104,7 +92,6 @@ quarkus.devservices.enabled=false spring.datasource.url=jdbc:pglite:memory:// # or persistent: jdbc:pglite:/var/data/myapp.zip spring.datasource.driver-class-name=io.roastedroot.pglite4j.jdbc.PgLiteDriver -spring.datasource.hikari.maximum-pool-size=5 spring.jpa.hibernate.ddl-auto=create-drop spring.jpa.database-platform=org.hibernate.dialect.PostgreSQLDialect ``` @@ -118,31 +105,25 @@ config.setMaximumPoolSize(5); DataSource ds = new HikariDataSource(config); ``` +## Known limitations + +- **No connection isolation** — PostgreSQL runs in single-user mode; all connections share the same session state. Queries are serialized via a lock, so there is no data corruption, but concurrent transactions are not isolated from each other. +- **Server-side prepared statements disabled** — all connections share a single backend, so named prepared statements would collide. The driver sets `prepareThreshold=0` to always use unnamed statements. This has no functional impact. +- **Limited extensions** — only `plpgsql` and `dict_snowball` are bundled; adding more requires rebuilding the WASM binary. +- **Binary size** — the WASM binary and pgdata resources add ~10 MB to the classpath. + +If any of these are limiting your use of the library, please [file an issue](https://github.com/roastedroot/pglite4j/issues) to discuss. + ## Project structure ``` pglite4j/ - core/ Core module — WASM lifecycle, CMA transport, wire protocol bridge - jdbc/ JDBC driver — PgLiteDriver, ServiceLoader registration, socket bridge - it/ Integration tests (Quarkus pet-clinic app with Hibernate + Panache) + core/ Core module — WASM lifecycle, wire protocol bridge + jdbc/ JDBC driver — PgLiteDriver, socket bridge, ServiceLoader registration + it/ Integration tests — Quarkus and Spring Boot sample apps wasm-build/ Dockerized build pipeline for the PostgreSQL WASM binary ``` -## Status and known limitations - -- [x] ~~**Only `memory://` is supported**~~ — file-backed storage is now supported via periodic snapshots. The database runs entirely in memory; the driver takes a full snapshot (zip of pgdata) on a configurable schedule and on shutdown. On restart the snapshot is restored. This is backup/restore-style persistence (like Redis RDB), not write-ahead logging — data between the last snapshot and a crash is lost -- [x] ~~**Single connection only**~~ — multiple JDBC connections are now supported per database instance; requests are serialized through a single PGLite backend via a lock, so connection pools with `max-size > 1` work correctly (queries execute one at a time, not in parallel) -- [x] ~~**Error recovery**~~ — both simple and extended query protocol errors are handled correctly; PostgreSQL errors trap the WASM instance and are caught by the Java side, which resets the backend state and drains stale protocol buffers so subsequent queries work cleanly -- [ ] **No connection isolation** — PostgreSQL runs in single-user mode with one session; all connections share the same session state (transactions, session variables). Queries are serialized, so there is no data corruption, but concurrent transactions are not isolated from each other. This is fine for connection pools that use connections sequentially (borrow, use, return). -- [ ] **Server-side prepared statements disabled** — because all connections share a single PostgreSQL backend, named prepared statements (`S_1`, `S_2`, …) would collide across connections. The driver sets `prepareThreshold=0` so pgjdbc always uses the unnamed prepared statement. This has no functional impact but means PostgreSQL cannot cache query plans across executions. -- [ ] **Limited extensions** — only `plpgsql` and `dict_snowball` are bundled; adding more requires rebuilding the WASM binary -- [ ] **Startup time** — first connection has some overhead that can be optimized further -- [ ] **Binary size** — the WASM binary + pgdata resources add several MBs to the classpath - -### CMA (Contiguous Memory Allocator) - -CMA is a preallocated contiguous region at the start of WASM linear memory used for zero-copy data transfer between Java and the PostgreSQL backend (similar concept to [Linux CMA](https://developer.toradex.com/software/linux-resources/linux-features/contiguous-memory-allocator-cma-linux/)). Messages that fit within the CMA buffer (default 12 MB) are transferred directly via shared memory. For responses that exceed the CMA buffer, the C code automatically falls back to file-based transport (`/pgdata/.s.PGSQL.5432.out`), which the Java side reads transparently. - ## Building from source ```bash @@ -161,6 +142,6 @@ mvn install ## Acknowledgements Special thanks to: -- **[PGLite](https://github.com/electric-sql/postgres-pglite)** - The Postgres build already mostly patched for Wasm -- **[PGLite-build](https://github.com/electric-sql/pglite-build)** - Spearheaded the build for the WASI target -- **[Chicory](https://github.com/dylibso/chicory)** - Pure Java WebAssembly runtime that makes this possible +- **[PGLite](https://github.com/electric-sql/postgres-pglite)** — the PostgreSQL build patched for Wasm +- **[PGLite-build](https://github.com/electric-sql/pglite-build)** — spearheaded the WASI build target +- **[Chicory](https://github.com/dylibso/chicory)** — pure-Java WebAssembly runtime that makes this possible diff --git a/core/pom.xml b/core/pom.xml index 26ea990..a321837 100644 --- a/core/pom.xml +++ b/core/pom.xml @@ -71,17 +71,17 @@ io.roastedroot.pglite4j.core.PGLiteModule ../wasm-build/output/pglite.wasi - WARN - + + 2919 4336 - 4494 - 4551 - 6394 - 6397 - 11038 - --> + 4337 + 4495 + 4552 + 6395 + 6398 + 11040 + diff --git a/pom.xml b/pom.xml index db78f31..f8dbb92 100644 --- a/pom.xml +++ b/pom.xml @@ -44,7 +44,6 @@ UTF-8 - UTF-8 UTF-8 11 true From 1c2d3245d4414a4e9a563158da4f090435980f98 Mon Sep 17 00:00:00 2001 From: andreatp Date: Thu, 5 Mar 2026 12:26:11 +0000 Subject: [PATCH 2/6] better coverage --- .github/workflows/ci.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index bfaede5..851539a 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -41,8 +41,8 @@ jobs: strategy: fail-fast: false matrix: - os: [ ubuntu-latest ] - java: [ 21 ] + os: [ ubuntu-latest, macos-latest, windows-latest ] + java: [ 17, 21, 25 ] steps: - uses: actions/checkout@v4 From 40da454f279ec98810dfcb20feda8ad8ca521b99 Mon Sep 17 00:00:00 2001 From: andreatp Date: Thu, 5 Mar 2026 13:04:34 +0000 Subject: [PATCH 3/6] more --- core/pom.xml | 13 ------------- pom.xml | 4 ++-- 2 files changed, 2 insertions(+), 15 deletions(-) diff --git a/core/pom.xml b/core/pom.xml index a321837..3dc9178 100644 --- a/core/pom.xml +++ b/core/pom.xml @@ -115,19 +115,6 @@ - - org.apache.maven.plugins - maven-checkstyle-plugin - - - checkstyle - - check - - validate - - - org.apache.maven.plugins diff --git a/pom.xml b/pom.xml index f8dbb92..1371c89 100644 --- a/pom.xml +++ b/pom.xml @@ -132,9 +132,9 @@ - java17 + java21 - [17,) + [21,) From bfdd7828d5193e6d0c271fe299dd77930918f73d Mon Sep 17 00:00:00 2001 From: andreatp Date: Thu, 5 Mar 2026 13:05:18 +0000 Subject: [PATCH 4/6] surefire 1 gig --- pom.xml | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/pom.xml b/pom.xml index 1371c89..0193f5f 100644 --- a/pom.xml +++ b/pom.xml @@ -99,6 +99,14 @@ true + + org.apache.maven.plugins + maven-surefire-plugin + ${surefire-plugin.version} + + -Xmx1g + + org.codehaus.mojo templating-maven-plugin From db8ca92d9cfa7d03375eb8f5e03172b51df2a3e6 Mon Sep 17 00:00:00 2001 From: andreatp Date: Thu, 5 Mar 2026 13:17:22 +0000 Subject: [PATCH 5/6] more --- .github/workflows/ci.yml | 2 ++ README.md | 1 + core/pom.xml | 1 - pom.xml | 8 -------- 4 files changed, 3 insertions(+), 9 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 851539a..43fd6bf 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -61,6 +61,8 @@ jobs: - name: Build and test run: mvn -B install + env: + MAVEN_OPTS: -Xmx2g - name: Publish Test Report if: always() diff --git a/README.md b/README.md index 3ce8e97..ad4aa8c 100644 --- a/README.md +++ b/README.md @@ -111,6 +111,7 @@ DataSource ds = new HikariDataSource(config); - **Server-side prepared statements disabled** — all connections share a single backend, so named prepared statements would collide. The driver sets `prepareThreshold=0` to always use unnamed statements. This has no functional impact. - **Limited extensions** — only `plpgsql` and `dict_snowball` are bundled; adding more requires rebuilding the WASM binary. - **Binary size** — the WASM binary and pgdata resources add ~10 MB to the classpath. +- **High memory usage** — each PGLite instance runs a full PostgreSQL backend in WASM linear memory. Expect significant heap consumption (1 GB+); make sure to size `-Xmx` accordingly. If any of these are limiting your use of the library, please [file an issue](https://github.com/roastedroot/pglite4j/issues) to discuss. diff --git a/core/pom.xml b/core/pom.xml index 3dc9178..27f13a2 100644 --- a/core/pom.xml +++ b/core/pom.xml @@ -115,7 +115,6 @@ - org.apache.maven.plugins maven-compiler-plugin diff --git a/pom.xml b/pom.xml index 0193f5f..1371c89 100644 --- a/pom.xml +++ b/pom.xml @@ -99,14 +99,6 @@ true - - org.apache.maven.plugins - maven-surefire-plugin - ${surefire-plugin.version} - - -Xmx1g - - org.codehaus.mojo templating-maven-plugin From 262740dd85f3698d7234640b3292633c0ef27bed Mon Sep 17 00:00:00 2001 From: andreatp Date: Thu, 5 Mar 2026 13:36:37 +0000 Subject: [PATCH 6/6] bigger macos runners --- .github/workflows/ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 43fd6bf..93fc9e0 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -41,7 +41,7 @@ jobs: strategy: fail-fast: false matrix: - os: [ ubuntu-latest, macos-latest, windows-latest ] + os: [ ubuntu-latest, macos-26-intel, windows-latest ] java: [ 17, 21, 25 ] steps: