Add sensible SQLite default Pragmas (with ability to override)#21
Open
cllns wants to merge 9 commits into
Open
Add sensible SQLite default Pragmas (with ability to override)#21cllns wants to merge 9 commits into
cllns wants to merge 9 commits into
Conversation
Bootstraps the SQLite namespace and a dedicated exception class for the validation case: when an override names a pragma the linked SQLite doesn't recognise. Carries the offending names as structured data alongside the message and freezes its inputs so callers can hold references safely. The error message tells the user how to opt out (`validate: false`).
…t_sqls A small value class that holds a set of SQLite pragmas. Defaults are chosen to match the broadly-accepted production-friendly set (WAL, synchronous=normal, 128MiB mmap, 64MiB journal cap, 2k-page cache, foreign keys on); user overrides merge on top, with `clear_defaults` for callers who want to start empty. Name validation runs against the linked SQLite's own `pragma_list` (via `pragma_pragma_list`), reflecting whatever the runtime supports rather than a hardcoded allowlist that would rot. The lookup is class-level, lazy, mutex-guarded, and skipped entirely when `validate: false` — so loading the file never opens a SQLite handle, and concurrent first-callers from a thread-pool boot can't race. `#connect_sqls` returns an array of `PRAGMA name = value` strings suitable for Sequel's per-connection `:connect_sqls` option. That hook fires for every new physical connection the pool opens, which is what pragma state needs — `:after_connect` would have been the wrong abstraction since it receives the raw adapter connection (SQLite3::Database), not a Sequel::Database, and connection-pool semantics mean only one pool member would have seen any one-shot side-effect. String keys in overrides are normalised to symbols so the canonical form (matching DEFAULTS) is the only thing that reaches the rest of the class.
Unit specs exercise the SQL-string shape; this exercises the contract end-to-end. Opens a file-backed SQLite via Sequel with `connect_sqls: pragmas.connect_sqls`, then reads back each PRAGMA to confirm the value SQLite actually committed — including overrides taking precedence and unrelated defaults still applying. Also asserts that pool members beyond the first see the same pragma state, which is the load-bearing property of the per-connection hook (and the reason an :after_connect-style one-shot would have been the wrong design). Adds a `sqlite_file_database_url` helper alongside the existing memory URL helper, so tests can spin up disposable file-backed databases on both MRI and JRuby.
Drops the `validate:` kwarg. Validation now runs unconditionally on every `Pragmas.new`, so a typo'd pragma name always raises rather than depending on how the caller constructed the instance. The bypass guidance is removed from `UnknownPragmaError`'s message and the corresponding `validate: false` test contexts go with it.
9e9e7ea to
034ee07
Compare
JRuby installs `jdbc-sqlite3`, not the `sqlite3` gem, so the eager `require "sqlite3"` and the direct `SQLite3::Database` reference broke spec loading entirely on JRuby (LoadError before any example ran). Sequel is already a transitive runtime dependency via `rom-sql` and abstracts the adapter difference behind a URL scheme. Opening a `:memory:` connection through Sequel uses sqlite3-ruby on MRI and jdbc-sqlite3 on JRuby, both yielding the same pragma_list contents.
RUBY_PLATFORM describes the underlying machine ("x86_64-linux",
"x86_64-darwin", "java"); JRuby reporting "java" there is an
implementation quirk, not the field's purpose. RUBY_ENGINE names
the implementation directly ("ruby", "jruby", "truffleruby"),
which is what every check in this repo actually means.
The :memory: SQLite URL isn't a Pragmas concern — it's a SQLite-level fact about which adapter scheme this Ruby implementation supports. Hoisting it makes it reachable for any other internal SQLite code that needs a transient in-memory connection (e.g. future schema inspection or feature detection), and shrinks the Pragmas surface. Lexical constant lookup still resolves `MEMORY_URL` cleanly from within Pragmas, so no qualifier needed at the call site.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
Hanami::DB::SQLite::Pragmas, a small value class that holds a set of SQLite pragmas with sensible defaults, accepts user overrides, validates pragma names against the linked SQLite, and exposes#connect_sqls, which is an array ofPRAGMAstatements suitable for Sequel's per-connection:connect_sqlshook.This matches @fractaledmind's recommendations, which landed in Rails:
foreign_keystruejournal_mode:walsynchronous:normalmmap_sizejournal_size_limitcache_sizeUsage
To try these out in a full-stack
hanamiapp, use:And verify with
Hanami.app["db"].fetch("PRAGMA journal_mode").first.In my app, benchmarking the 23 endpoints I have with a 'realistic' read-heavy load (on a file-based SQLite db), reads improved ~10% on average, and 3-25% faster for p95 (mostly 10-15%). For writes, most writes were much faster (30-50%), while some degraded slightly (by a small amount).
Importantly, there were some really bad p95 values of 5s+ which are gone now, which matters even more than the modest/significant performance increase above.
I'll open a PR to make this the default experience in
hanami/hanamionce this lands, so all SQLite users will benefit from it.Design notes
pragma_list(via thepragma_pragma_listtable-valued function), so newer pragmas are recognised automatically without a gem upgrade.busy_timeoutwas included in @fractaledmind's conference talk. Will add this a follow-on PR to update Sequel config for SQLite alongside other driver concerns (e.g.integer_booleans), not in a class namedPragmas.#connect_sqlsreturns an array of SQL strings that Sequel applies via its:connect_sqlsoption to every new physical connection the pool opens. An:after_connectcallback would have been the wrong abstraction: Sequel passes the raw adapter handle (SQLite3::Database), not aSequel::Database, and a one-shot call against a singleSequel::Databasewould only configure whichever pool member happened to handle that call.validate: false, based on when I was validating against the sqlite3 gem, but I switched the method to call a query, and I can't think of any reason why anyone would want to skip validation, except one: using a very old SQLite version. The PRAGMA LISt was added in SQLite 3.20.0 which was released August 2017. I highly doubt anyone will be connecting to a DB that old with Hanami, but I'm open to putting it back in as an escape valve. It does open a connection to:memory:on call but that's in every SQLite compilation and is extremely fast, plus it's only run once.