Skip to content

Add sensible SQLite default Pragmas (with ability to override)#21

Open
cllns wants to merge 9 commits into
mainfrom
add-sqlite-pragma-with-sensible-defaults
Open

Add sensible SQLite default Pragmas (with ability to override)#21
cllns wants to merge 9 commits into
mainfrom
add-sqlite-pragma-with-sensible-defaults

Conversation

@cllns
Copy link
Copy Markdown
Member

@cllns cllns commented May 12, 2026

Summary

Adds Hanami::DB::SQLite::Pragmas, a small value class that holds a set of SQLite pragmas with sensible defaults, accepts user overrides, validates pragma names against the linked SQLite, and exposes #connect_sqls, which is an array of PRAGMA statements suitable for Sequel's per-connection :connect_sqls hook.

This matches @fractaledmind's recommendations, which landed in Rails:

pragma value
foreign_keys true
journal_mode :wal
synchronous :normal
mmap_size 128 MiB
journal_size_limit 64 MiB
cache_size 2000 pages

Usage

# Just the defaults
pragmas = Hanami::DB::SQLite::Pragmas.new

# Override individual values; defaults still apply for the rest
pragmas = Hanami::DB::SQLite::Pragmas.new(overrides: {synchronous: :full})

# Drop the defaults entirely
pragmas = Hanami::DB::SQLite::Pragmas.new(clear_defaults: true, overrides: {})

# Wire into Sequel
Sequel.connect(url, connect_sqls: pragmas.connect_sqls)

To try these out in a full-stack hanami app, use:

config.gateway(:default) do |gateway|
  gateway.connection_options(connect_sqls: Hanami::DB::SQLite::Pragmas.new.connect_sqls)
end

And verify with Hanami.app["db"].fetch("PRAGMA journal_mode").first.

In my app, benchmarking the 23 endpoints I have with a 'realistic' read-heavy load (on a file-based SQLite db), reads improved ~10% on average, and 3-25% faster for p95 (mostly 10-15%). For writes, most writes were much faster (30-50%), while some degraded slightly (by a small amount).

Importantly, there were some really bad p95 values of 5s+ which are gone now, which matters even more than the modest/significant performance increase above.

I'll open a PR to make this the default experience in hanami/hanami once this lands, so all SQLite users will benefit from it.

Design notes

  • Name validation runs against the linked SQLite's own pragma_list (via the pragma_pragma_list table-valued function), so newer pragmas are recognised automatically without a gem upgrade.
  • busy_timeout was included in @fractaledmind's conference talk. Will add this a follow-on PR to update Sequel config for SQLite alongside other driver concerns (e.g. integer_booleans), not in a class named Pragmas.
  • Per-connection by design. PRAGMA state lives on the connection, not the database. #connect_sqls returns an array of SQL strings that Sequel applies via its :connect_sqls option to every new physical connection the pool opens. An :after_connect callback would have been the wrong abstraction: Sequel passes the raw adapter handle (SQLite3::Database), not a Sequel::Database, and a one-shot call against a single Sequel::Database would only configure whichever pool member happened to handle that call.
  • Always validates names. I originally had an option where users could pass in validate: false, based on when I was validating against the sqlite3 gem, but I switched the method to call a query, and I can't think of any reason why anyone would want to skip validation, except one: using a very old SQLite version. The PRAGMA LISt was added in SQLite 3.20.0 which was released August 2017. I highly doubt anyone will be connecting to a DB that old with Hanami, but I'm open to putting it back in as an escape valve. It does open a connection to :memory: on call but that's in every SQLite compilation and is extremely fast, plus it's only run once.

cllns added 6 commits May 11, 2026 00:30
Bootstraps the SQLite namespace and a dedicated exception class for
the validation case: when an override names a pragma the linked
SQLite doesn't recognise. Carries the offending names as structured
data alongside the message and freezes its inputs so callers can
hold references safely. The error message tells the user how to
opt out (`validate: false`).
…t_sqls

A small value class that holds a set of SQLite pragmas. Defaults are
chosen to match the broadly-accepted production-friendly set (WAL,
synchronous=normal, 128MiB mmap, 64MiB journal cap, 2k-page cache,
foreign keys on); user overrides merge on top, with `clear_defaults`
for callers who want to start empty.

Name validation runs against the linked SQLite's own `pragma_list`
(via `pragma_pragma_list`), reflecting whatever the runtime supports
rather than a hardcoded allowlist that would rot. The lookup is
class-level, lazy, mutex-guarded, and skipped entirely when
`validate: false` — so loading the file never opens a SQLite handle,
and concurrent first-callers from a thread-pool boot can't race.

`#connect_sqls` returns an array of `PRAGMA name = value` strings
suitable for Sequel's per-connection `:connect_sqls` option. That
hook fires for every new physical connection the pool opens, which
is what pragma state needs — `:after_connect` would have been the
wrong abstraction since it receives the raw adapter connection
(SQLite3::Database), not a Sequel::Database, and connection-pool
semantics mean only one pool member would have seen any one-shot
side-effect.

String keys in overrides are normalised to symbols so the canonical
form (matching DEFAULTS) is the only thing that reaches the rest of
the class.
Unit specs exercise the SQL-string shape; this exercises the contract
end-to-end. Opens a file-backed SQLite via Sequel with
`connect_sqls: pragmas.connect_sqls`, then reads back each PRAGMA to
confirm the value SQLite actually committed — including overrides
taking precedence and unrelated defaults still applying.

Also asserts that pool members beyond the first see the same pragma
state, which is the load-bearing property of the per-connection hook
(and the reason an :after_connect-style one-shot would have been the
wrong design).

Adds a `sqlite_file_database_url` helper alongside the existing memory
URL helper, so tests can spin up disposable file-backed databases on
both MRI and JRuby.
Drops the `validate:` kwarg. Validation now runs unconditionally on
every `Pragmas.new`, so a typo'd pragma name always raises rather
than depending on how the caller constructed the instance. The
bypass guidance is removed from `UnknownPragmaError`'s message and
the corresponding `validate: false` test contexts go with it.
@cllns cllns force-pushed the add-sqlite-pragma-with-sensible-defaults branch from 9e9e7ea to 034ee07 Compare May 12, 2026 06:10
cllns added 3 commits May 12, 2026 00:23
JRuby installs `jdbc-sqlite3`, not the `sqlite3` gem, so the eager
`require "sqlite3"` and the direct `SQLite3::Database` reference broke
spec loading entirely on JRuby (LoadError before any example ran).

Sequel is already a transitive runtime dependency via `rom-sql` and
abstracts the adapter difference behind a URL scheme. Opening a
`:memory:` connection through Sequel uses sqlite3-ruby on MRI and
jdbc-sqlite3 on JRuby, both yielding the same pragma_list contents.
RUBY_PLATFORM describes the underlying machine ("x86_64-linux",
"x86_64-darwin", "java"); JRuby reporting "java" there is an
implementation quirk, not the field's purpose. RUBY_ENGINE names
the implementation directly ("ruby", "jruby", "truffleruby"),
which is what every check in this repo actually means.
The :memory: SQLite URL isn't a Pragmas concern — it's a SQLite-level
fact about which adapter scheme this Ruby implementation supports.
Hoisting it makes it reachable for any other internal SQLite code
that needs a transient in-memory connection (e.g. future schema
inspection or feature detection), and shrinks the Pragmas surface.

Lexical constant lookup still resolves `MEMORY_URL` cleanly from
within Pragmas, so no qualifier needed at the call site.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant